An article explains the principles of Docker network

Overview

Docker native network is based on Linux network namespace and virtual network device (veth pair). When the Docker process is started, a virtual bridge named docker0 is created on the host, and the Docker container started on the host will be connected to this virtual bridge.

 $ ifconfig

 # Output is as follows

 docker0 : ... mtu 1500
 inet 172.17 .0 .1 netmask 255.255 .0 .0 broadcast 172.17 .255 .255
 ...

The working mode of the virtual bridge is similar to that of a physical switch. All containers on the host are connected to a Layer 2 network through the virtual bridge.

Assign an IP from the docker0 subnet to the container and set the docker0 IP address as the default gateway for the container. Create a pair of virtual network cards veth pair devices on the host. Docker puts one end of the veth pair device in the newly created container and names it eth0 (the container's network card), and the other end in the host, naming it something like vethxxx, and connects this network device to the docker0 bridge.

Docker will automatically configure iptables rules and NAT to connect to the docker0 bridge on the host. After completing these operations, the container can use its eth0 virtual network card to connect to other containers and access the external network.

The network interfaces in Docker are virtual interfaces by default. Linux implements data transmission between interfaces through data replication in the kernel, which can give full play to the efficiency of data forwarding between different Docker containers or between containers and hosts. The data packets in the sending interface's sending cache will be directly copied to the receiving interface's cache without the need for exchange through physical network devices.

 # Query the veth device on the host
 $ ifconfig | grep veth *

 veth06f40aa :
 ...
 vethfdfd27a : 

Image source: https://www.suse.com/c/rancher_blog/introduction-to-container-networking/

The virtual bridge docker0 is connected to the network card on the host machine through iptables configuration. All qualified requests are forwarded to docker0 through iptables and then distributed to the corresponding containers.

 # View docker's iptables configuration
 $ iptables -t nat -L

 # Output is as follows

 Chain PREROUTING ( policy ACCEPT )
 target prot opt ​​source destination
 DOCKER all -- anywhere anywhere ADDRTYPE match dst-type LOCAL

 Chain DOCKER ( 2 references )
 target prot opt ​​source destination
 RETURN all -- anywhere anywhere

Network Driver

Docker's network subsystem supports pluggable drivers, has multiple drivers by default, and provides core network functions.

Image source: Docker - Containers and Container Cloud

Docker daemon completes network creation and management by calling the API provided by libnetwork. CNM is used in libnetwork to complete network functions. CNM mainly has three components: sandbox, endpoint, and network.

• Sandbox: A sandbox contains information about a container's network stack. A sandbox can have multiple endpoints and multiple networks. The sandbox can manage the container's interfaces, routing, and DNS settings. The sandbox can be implemented as a Linux network namespace, FreeBSD Jail, or similar mechanisms.

• Endpoint: An endpoint can join one sandbox and one network. An endpoint belongs to only one network and one sandbox. The implementation of an endpoint can be a veth pair, an Open vSwitch internal port, or a similar device.

• Network: A network is a group of endpoints that can directly connect to each other. A network can contain multiple endpoints. The network can be implemented by Linux bridge, VLAN, etc.

Bridge Mode

Bridge is the default network mode, which creates an independent network namespace for the container. The container has an independent network card and all network stacks. All containers using this mode are connected to the docker0 bridge, which acts as a virtual switch to allow containers to communicate with each other. However, since the IP address of the host machine and the IP address of the container veth pair are not in the same network segment, in order to communicate with networks outside the host machine, Docker uses port binding, that is, forwarding the port traffic on the host machine to the container through iptables NAT.

The bridge mode can already meet the most basic usage requirements of Docker containers, but it uses NAT when communicating with the outside world, which increases the complexity of communication and has limitations in complex scenarios.

 $ docker network inspect bridge

 # The output is as follows ( selected information )
 [
 {
 "Name" : "bridge" ,
 "Scope" : "local" ,
 "Driver" : "bridge" ,
 "EnableIPv6" : false ,
 "IPAM" : {
 "Driver" : "default" ,
 "Options" : null ,
 "Config" : [
 {
 "Subnet" : "172.17.0.0/16" ,
 "Gateway" : "172.17.0.1"
 }
 ]
 } ,
 "Containers" : {
 # List of containers using the bridge network
 } ,
 }
 ]

From the above output, you can see that the IP address of the virtual bridge is the gateway address of the bridge network type.

We can find a container from the output Containers container list and view its network type and configuration:

 $ docker inspect container ID

 # The output is as follows ( selected information )
 [
 ...

 "NetworkSettings" : {
 "Bridge" : "" ,
 "Gateway" : "172.17.0.1" ,
 "IPAddress" : "172.17.0.4" ,
 "Networks" : {
 "bridge" : {
 "Gateway" : "172.17.0.1" ,
 "IPAddress" : "172.17.0.4" ,
 }
 }

 ...
 ]

From the above output, you can see that the IP address of the virtual bridge is the gateway address of the container of the bridge network type.

Implementation Mechanism

DNAT rules are set in iptables to implement port forwarding function:

 # iptables configuration view
 $ iptables -t nat -vnL

 # Output is as follows
 Chain PREROUTING ( policy ACCEPT 37 M packets , 2210 M bytes )

 ...

 0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.17.0.4:80

When a container needs to map a port to the host, Docker automatically assigns an IP address to the container and adds a new iptables rule.

Host Mode

The container does not obtain an independent network namespace, but shares one with the host. The container does not virtualize its own network card, configure its own IP, etc., but directly uses the host. However, other aspects of the container, such as the file system and process list, are still isolated from the host. The container is completely open to the outside world. If you can access the host, you can access the container.

The host mode reduces the network isolation between containers and between containers and the host. Although it has performance advantages, it causes competition and conflicts for network resources. Therefore, it is suitable for scenarios with smaller container clusters.

Start an Nginx container with a network type of host:

 $ docker run -d --net host nginx

 Unable to find image 'nginx:latest' locally
 latest : Pulling from library / nginx
 ...
 f202870092fc40bc08a607dddbb2770df9bb4534475b066f45ea35252d6e76e2

View the list of containers with a network type of host:

 $ docker network inspect host

 # The output is as follows ( selected information )
 [
 {
 "Name" : "host" ,
 "Scope" : "local" ,
 "Driver" : "host" ,
 "Internal" : false ,
 "Attachable" : false ,
 "Ingress" : false ,
 "ConfigFrom" : {
 "Network" : ""
 } ,
 "Containers" : {
 # List of containers using host network
 "f202870092fc40bc08a607dddbb2770df9bb4534475b066f45ea35252d6e76e2" : {
 "Name" : "frosty_napier" ,
 "EndpointID" : "7306a8e4103faf4edd081182f015fa9aa985baf3560f4a49b9045c00dc603190" ,
 "MacAddress" : "" ,
 "IPv4Address" : "" ,
 "IPv6Address" : ""
 }
 } ,
 }
 ]

View the Nginx container network type and configuration:

 $ docker inspect f202870092fc4

 # The output is as follows ( selected information )
 [
 ...

 "NetworkSettings" : {
 "Bridge" : "" ,
 "Gateway" : "" ,
 "IPAddress" : "" ,
 "Networks" : {
 "host" : {
 "Gateway" : "" ,
 "IPAddress" : "" ,
 }
 }
 ...
 ]

From the above output, we can see that the network type used by the Nginx container is host and there is no independent IP.

View the Nginx container IP address:

 # Enter the shell inside the container

 $ docker exec -it f202870092fc4 / bin / bash

 # Install ip command

 $ apt update && apt install -y iproute2

 # View IP address
 $ ip a

 # Output is as follows
 1 : lo : < LOOPBACK , UP , LOWER_UP > mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
 ...
 2 : docker0 : < NO - CARRIER , BROADCAST , MULTICAST , UP > mtu 1500 qdisc noqueue state DOWN group default
 ...
 10 : eth0@if11 : < BROADCAST , MULTICAST , UP , LOWER_UP > mtu 1500 qdisc noqueue state UP group default
 ...

 # Exit the container and view the host IP address
 $ exit
 $ ip a

 # Output is as follows
 1 : lo : < LOOPBACK , UP , LOWER_UP > mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
 ...
 2 : docker0 : < NO - CARRIER , BROADCAST , MULTICAST , UP > mtu 1500 qdisc noqueue state DOWN group default
 ...
 10 : eth0@if11 : < BROADCAST , MULTICAST , UP , LOWER_UP > mtu 1500 qdisc noqueue state UP group default
 ...

From the above output, we can see that there is no independent IP inside the Nginx container, but the IP of the host machine is used.

Check the port monitoring status of the host:

 $ sudo netstat -ntpl

 # Output is as follows
 Proto Recv - Q Send - Q Local Address Foreign Address State PID / Program name
 tcp 0 0 0.0 .0 .0 : 80 0.0 .0 .0 : * LISTEN 1378 / nginx : master
 tcp6 0 0 ::: 80 ::: * LISTEN 1378 / nginx : master

From the above output, we can see that the process listening on port 80 is nginx, not docker-proxy.

None mode

The container has its own Network Namespace, but does not perform any network configuration. This means that the container has no network card, IP, routing and other information. You need to manually add a network card and configure the IP for the container. The container in none mode is completely isolated, and only the loopback network card lo is used for process communication.

The none mode provides minimal network settings for the container and provides the highest flexibility by customizing the container's network without network configuration.

Start an Nginx container with a network type of host:

 $ docker run -d --net none nginx

 Unable to find image 'nginx:latest' locally
 latest: Pulling from library/nginx
 ...
 d2d0606b7d2429c224e61e06c348019b74cd47f0b8c85347a7cdb8f1e30dcf86

View the list of containers with a network type of none:

 $ docker network inspect none

 # The output is as follows ( selected information )
 [
 {
 "Name" : "none" ,
 "Scope" : "local" ,
 "Driver" : "null" ,
 "Internal" : false ,
 "Attachable" : false ,
 "Ingress" : false ,
 "ConfigFrom" : {
 "Network" : ""
 } ,
 "Containers" : {
 # List of containers using the none network
 "d2d0606b7d2429c224e61e06c348019b74cd47f0b8c85347a7cdb8f1e30dcf86" : {
 "Name" : "hardcore_chebyshev" ,
 "EndpointID" : "b8ff645671518e608f403818a31b1db34d7fce66af60373346ea3ab673a4c6b2" ,
 "MacAddress" : "" ,
 "IPv4Address" : "" ,
 "IPv6Address" : ""
 }
 } ,
 }
 ]

View the Nginx container network type and configuration:

 $ docker inspect d2d0606b7d242

 # The output is as follows ( selected information )
 [
 ...

 "NetworkSettings" : {
 "Bridge" : "" ,
 "Gateway" : "" ,
 "IPAddress" : "" ,
 "Networks" : {
 "none" : {
 "Gateway" : "" ,
 "IPAddress" : "" ,
 }
 }
 ...
 ]

From the above output, you can see that the network type used by the Nginx container is none, and there is no independent IP.

View the Nginx container IP address:

 # Enter the shell inside the container

 $ docker exec -it d2d0606b7d242 / bin / bash

 # Access the public network link

 $ curl -I "https://www.docker.com"

 curl : ( 6 ) Could not resolve host : www .docker .com

 # Why is there an error? This is because the current container has no network card, IP, routing and other information. It is a completely independent operating environment, so there is no way to access the public network link.

 # View IP address
 $ hostname -I

 # No output, the container has no IP address

Check the port monitoring status of the host:

 $ docker port d2d0606b7d242

 or

 $ sudo netstat - ntpl | grep : 80

 # No output, the Nginx process is running in the container, and the port is not mapped to the host

Container Mode

Similar to the host mode, the container shares the network namespace with the specified container. There is no network isolation between the two containers, but they are network isolated from the host and other containers. Containers in this mode can access other containers in the same network namespace through localhost, which has high transmission efficiency and saves certain network resources. It is very useful in some special scenarios, such as k8s Pod.

Other modes

Due to space considerations, other network modes will not be discussed here. Interested readers can read them on their own according to the reference link at the end of the article.

Network Driver Overview

• • When multiple containers need to communicate on the same host, use bridge
• • Use host when the network stack should not be isolated from the host, but you want other aspects of the container to be isolated.
• • When containers running on different hosts need to communicate, use overlay
• • When migrating from a VM or needing to make a container look like a physical host, use Macvlan, where each container has a unique MAC address
• • When you need to integrate Docker with a specialized network stack, use a third-party

Docker and iptables

If you run Docker on a server that is accessible from the public Internet, you will need corresponding iptables rules to restrict access to containers or other services on the host.

Add iptables rules before Docker rules

Docker installs two custom iptables chains named DOCKER-USER and DOCKER to ensure that incoming packets are always checked by these two chains first.

 # You can view it through this command

 $ iptables - L - n - v | grep - i docker

All iptables rules for Docker are added to the Docker chain. Do not modify this chain manually (it may cause problems). If you need to add some rules that are loaded before Docker, add them to the DOCKER-USER chain. These rules are applied before any rules automatically created by Docker.

Rules added to the FORWARD chain are checked after these chains, which means that if a port is exposed via Docker, it will be exposed regardless of what rules are configured in the firewall. If you want these rules to still apply when exposing ports via Docker, you must add them to the DOCKER-USER chain.

Limiting connections to the Docker host

By default, all external IPs are allowed to connect to the Docker host. To allow only specific IPs or networks to access the container, insert a rule at the top of the DOCKER-USER filter chain.

For example, to only allow 192.168.1.1 to access:

 # Assume the input interface is eth0
 $ iptables - I DOCKER - USER - i eth0 ! - s 192.168 .1 .1 - j DROP

You can also allow connections from the source subnet, for example, to allow users from the 192.168.1.0/24 subnet to access:

 # Assume the input interface is eth0
 $ iptables - I DOCKER - USER - i eth0 ! - s 192.168 .1 .0 / 24 - j DROP

Prevent Docker from manipulating iptables

Set the iptables value to false in the Docker engine configuration file /etc/docker/daemon.json, but it is best not to modify it because it is likely to destroy the container network of the Docker engine.

Set the default binding address for the container

By default, the Docker daemon will expose ports on the 0.0.0.0 address, which is any address on the host machine. If you want to change that behavior to only expose ports on internal IP addresses, you can specify a different IP address using the --ip option.

Integrate into firewall

If you are running Docker 20.10.0 or later and iptables is enabled on your system, Docker automatically creates a firewall zone named docker and joins all network interfaces it creates (e.g. docker0 ) to the docker zone to allow seamless networking.

Run the command to remove the docker interface from the firewall zone:

 firewall - cmd --znotallow=trusted --remove-interface=docker0 --permanent
 firewall - cmd --reload

Reference

• Networking overview[1]

• Networking tutorials[2]

• Docker and iptables[3]

• Detailed explanation of Docker container[4]

• Introduction to Container Networking[5]

• Docker container network solution: calico network model[6]

• Docker - containers and container clouds[7]

Reference Links

[1]​ Networking overview: ​​https://docs.docker.com/network/[2]​​ Networking tutorials: ​​https://docs.docker.com/network/network-tutorial-standalone/[3]​​ Docker and iptables: ​​https://docs.docker.com/network/iptables/​​

[4]​ Docker container detailed explanation: ​​https://juejin.cn/post/6844903766601236487​​

[5] Introduction to Container Networking: https://www.suse.com/c/rancher_blog/introduction-to-container-networking/

[6]​ Docker container network solution: calico network model: ​​https://cizixs.com/2017/10/19/docker-calico-network/​​

[7] Docker - Containers and Container Clouds: https://book.douban.com/subject/26894736/