Docker networking

Once you install Docker in Linux, a ‘default’ networking configuration is applied. Here is what happens…

Docker adds a bridge to the Linux OS named ‘docker0‘ and that bridge is an isolated network defined in software.

slade@linux-home:/etc/iptables$ ifconfig docker0
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:e5ff:fef0:dbc0  prefixlen 64  scopeid 0x20<link>
        ether 02:42:e5:f0:db:c0  txqueuelen 0  (Ethernet)
        RX packets 17842  bytes 2257624 (2.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 24480  bytes 174222061 (174.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Your Linux host running the Docker engine can see this interface, and machines attached to your LAN are able to access ports you’ve exposed on your containers via iptables NAT functionality.

Docker will also make the required iptables changes such that traffic arriving at the primary interface on any listening port attached to a Docker container is then forwarded to that Docker container via the bridge it’s attached to. Also, any outbound traffic leaving a container is forwarded via the bridge to the primary network interface on the host.

-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT

Examples of a few rules that were created in response to Docker containers spinning up

-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 5432 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8444 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8443 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8001 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8000 -j ACCEPT
-A DOCKER -d 172.17.0.4/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8080 -j ACCEPT

Note that everything revolves around the docker0 bridge.

This is all good, but the default bridge has limitations to be aware of and it is recommended to build your own user-defined bridge to avoid them.

Configure your networking manually…

Add a Docker bridge

docker network create -d bridge --subnet 172.172.0.1/16 docker1

Check your Docker networking

slade@linux-home:~$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
69ce3fd4683d        bridge              bridge              local
d1c6e30494d3        docker1             bridge              local
7f7e80588cd4        host                host                local
5a9009966282        none                null                local

Show your Linux bridges

slade@linux-home:~$ brctl show
bridge name	bridge id		STP enabled	interfaces
br-d1c6e30494d3		8000.02429b725669	no
docker0		8000.0242edf05746	no		veth76b8a00

Note that the Docker output shows a bridge with a network ID ending in 94d3 and that same ID appears in the output of brctl show.

Now check your Linux interfaces again.

slade@linux-home:~$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 60:a4:4c:53:3e:18 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.10/24 brd 10.0.0.255 scope global eno1
       valid_lft forever preferred_lft forever
    inet6 fe80::62a4:4cff:fe53:3e18/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ed:f0:57:46 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:edff:fef0:5746/64 scope link
       valid_lft forever preferred_lft forever
17: br-d1c6e30494d3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:9b:72:56:69 brd ff:ff:ff:ff:ff:ff
    inet 172.172.0.1/16 brd 172.172.255.255 scope global br-d1c6e30494d3
       valid_lft forever preferred_lft forever

You can see the bridge with id br-d1c6e30494d3 has the IP we assigned when we created the Docker bridge using Docker cli; however, it does show a state of DOWN.

Now you can run containers and attach them to this network. Start a container and run it on the new docker1 bridge

docker run -d --name kong-database --network docker1 \
	-p 5432:5432 \
	-e "POSTGRES_USER=kong" \
	-e "POSTGRES_PASSWORD=secretface" \
	-e "POSTGRES_DB=kong" \
	postgres:9.6

Check your Linux interfaces again

slade@linux-home:~/kong$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 60:a4:4c:53:3e:18 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.10/24 brd 10.0.0.255 scope global eno1
       valid_lft forever preferred_lft forever
    inet6 fe80::62a4:4cff:fe53:3e18/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:ed:f0:57:46 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:edff:fef0:5746/64 scope link
       valid_lft forever preferred_lft forever
17: br-d1c6e30494d3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:9b:72:56:69 brd ff:ff:ff:ff:ff:ff
    inet 172.172.0.1/16 brd 172.172.255.255 scope global br-d1c6e30494d3
       valid_lft forever preferred_lft forever
    inet6 fe80::42:9bff:fe72:5669/64 scope link
       valid_lft forever preferred_lft forever

The bridge with id br-d1c6e30494d3 is UP.

Continue running a few containers, attaching them to the new bridge.

docker run --rm --network docker1 \
    --link kong-database:kong-database \
    -e "KONG_DATABASE=postgres" \
    -e "KONG_PG_USER=kong" \
    -e "KONG_PG_PASSWORD=secretface" \
    -e "KONG_PG_HOST=kong-database" \
    -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \
    kong:0.15.0 kong migrations bootstrap
docker run -d --name kong --network docker1 \
    --link kong-database:kong-database \
    -e "KONG_DATABASE=postgres" \
    -e "KONG_PG_USER=kong" \
    -e "KONG_PG_PASSWORD=secretface" \
    -e "KONG_PG_HOST=kong-database" \
    -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \
    -e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
    -e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
    -e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
    -e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
    -e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" \
    -p 8000:8000 \
    -p 8443:8443 \
    -p 8001:8001 \
    -p 8444:8444 \
    kong:0.15.0

You should now see these containers running

slade@linux-home:~/kong$ docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                                                NAMES
a11bacb99169        pgbi/kong-dashboard   "./docker/entrypoint…"   18 minutes ago      Up 18 minutes       0.0.0.0:8080->8080/tcp                                               hopeful_bose
068d82543001        kong:0.15.0           "/docker-entrypoint.…"   19 minutes ago      Up 19 minutes       0.0.0.0:8000-8001->8000-8001/tcp, 0.0.0.0:8443-8444->8443-8444/tcp   kong
46f19f0da69b        postgres:9.6          "docker-entrypoint.s…"   26 minutes ago      Up 26 minutes       0.0.0.0:5432->5432/tcp                                               kong-database

Notice that iptables has been updated automatically. As with the rules that were created for the default docker0 bridge, we now have those same rules on our br-d1c6e30494d3 bridge.

-A FORWARD -o br-d1c6e30494d3 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-d1c6e30494d3 -j DOCKER
-A FORWARD -i br-d1c6e30494d3 ! -o br-d1c6e30494d3 -j ACCEPT
-A FORWARD -i br-d1c6e30494d3 -o br-d1c6e30494d3 -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT

And these

-A DOCKER -d 172.172.0.2/32 ! -i br-d1c6e30494d3 -o br-d1c6e30494d3 -p tcp -m tcp --dport 5432 -j ACCEPT
-A DOCKER -d 172.172.0.3/32 ! -i br-d1c6e30494d3 -o br-d1c6e30494d3 -p tcp -m tcp --dport 8444 -j ACCEPT
-A DOCKER -d 172.172.0.3/32 ! -i br-d1c6e30494d3 -o br-d1c6e30494d3 -p tcp -m tcp --dport 8443 -j ACCEPT
-A DOCKER -d 172.172.0.3/32 ! -i br-d1c6e30494d3 -o br-d1c6e30494d3 -p tcp -m tcp --dport 8001 -j ACCEPT
-A DOCKER -d 172.172.0.3/32 ! -i br-d1c6e30494d3 -o br-d1c6e30494d3 -p tcp -m tcp --dport 8000 -j ACCEPT
-A DOCKER -d 172.172.0.4/32 ! -i br-d1c6e30494d3 -o br-d1c6e30494d3 -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-d1c6e30494d3 ! -o br-d1c6e30494d3 -j DOCKER-ISOLATION-STAGE-2

You should now be able to access these running containers from machines on your LAN.

An aside… I ran in to an issue where Linux persisted the bridge br-d1c6e30494d3 on a reboot, but Docker did not persist the docker1 bridge. When I re-created the docker1 bridge, it then created a new Linux bridge that directly overlapped br-d1c6e30494d3.

slade@linux-home:~/kong$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 60:a4:4c:53:3e:18 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.10/24 brd 10.0.0.255 scope global eno1
       valid_lft forever preferred_lft forever
    inet6 fe80::62a4:4cff:fe53:3e18/64 scope link
       valid_lft forever preferred_lft forever
8: br-d1c6e30494d3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:56:59:40:fe brd ff:ff:ff:ff:ff:ff
    inet 172.172.0.1/16 brd 172.172.255.255 scope global br-d1c6e30494d3
       valid_lft forever preferred_lft forever
25: br-cbe891bbef2b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:89:98:23:18 brd ff:ff:ff:ff:ff:ff
    inet 172.172.0.1/16 brd 172.172.255.255 scope global br-cbe891bbef2b
       valid_lft forever preferred_lft forever
    inet6 fe80::42:89ff:fe98:2318/64 scope link
       valid_lft forever preferred_lft forever

The fix was to remove bridge br-d1c6e30494d3. This resulted in containers attaching to the new Docker bridge, but hosts on the network not being able to actually reach those containers.

slade@linux-home:~/kong$ sudo ip link set br-d1c6e30494d3 down
slade@linux-home:~/kong$ sudo brctl delbr br-d1c6e30494d3

Problem solved.

Leave a comment

Close Bitnami banner
Bitnami