A Preliminary Study on Kubernetes Network Concepts

Kubernetes network is a core concept in Kubernetes. In short, the Kubernetes network model ensures that all Kubernetes pods on the cluster can communicate. In addition, based on the Kubernetes network model, Kubernetes has other core concepts, namely Kubernetes Services and Kubernetes Ingress.

[[393012]]

In this article, we will explore Kubernetes networking using a system model approach. We will develop a simple model to understand container-to-container communication and Pod-to-Pod communication.

How to view the network

Networking is undoubtedly a vast and complex field that requires years of theoretical and practical experience to master. In this article, we will review networking at a conceptual level without going into implementation details.

Ideal network model

The above diagram describes a network as a Network Graph, which consists of a set of nodes and links between nodes. A node can exchange information with another node if and only if there is a connection between the nodes.

Message exchange framework

A node, the source, exchanges messages with another node, the target, by placing messages on the target's input queue. The message exchange is represented by a Send Event, Send·M, observed by the source node and a corresponding Receive Event, Recv·M, observed at the target node.

Message exchange behavior

A node in a network is either a process or a switch. A process generates and consumes messages, and a switch processes messages according to its forwarding information base (FIB).

Forwarding Information Base (FIB) of S1 and S2

The figure above describes the forwarding information base (FIB) S1 and S2 of the switch. When receiving a message, each switch will query its forwarding information base to decide whether to send (deliver), forward (forward) or discard (discard) the message.

Switch:

• Match the request header of the information, i.e. the source address, source port, destination address and destination port, with its forwarding information base,
• Execute related operations, the default is discard

Kubernetes Network Model

The Kubernetes network model is a descriptive network model, that is, any network that meets the Kubernetes network model specification is a Kubernetes network.

However, Kubernetes does not specify how to implement the network model. In fact, there are many alternative implementations on the market, called network plugins.

This section describes the Kubernetes network model using a set of constraints on message exchanges.

Constraints: Network-addressable entities

The Kubernetes network model defines three addressable entities: K8S pod, K8S node, and K8S Service. Each entity is assigned a different IP address.

 ∧ (K8s-Pod(E₁) ∨ K8s-Node(E₁) ∨ K8s-Service(E₁)) ∧ (K8s-Pod(E₂) ∨ K8s-Node(E₂) ∨ K8s-Service(E₂)):
 addr(E₁, a) ∧ addr(E₂, a)₂
 ⟺ E₁ = E₂

However, the network model does not make any further claims about these IP addresses. For example, the Kubernetes network model does not make any further claims about the IP address space extracted from these IP addresses.

Limitation: Inter-container communication

The Kubernetes networking model requires that container C1 executing in the context of a Pod P can communicate with other containers C2 executing in the context of P over localhost.

 K8s-Pod(P) ∧ K8s-Container(C₁, P) ∧ K8s-Container(C₂, P): open (C₂, p)
 ⟹
 Send(e, C₁, 127.0.0.1, _, 127.0.0.1, p)
 ⟹
 Recv(e, C₂, 127.0.0.1, _, 127.0.0.1, p)

Constraints: Pod to Pod

The Kubernetes network model requires that container C1 executing in the context of Pod P1 can communicate with other containers C2 executing in the context of P2 through the address of P2.

 ∧ K8s-Pod(P₁) ∧ K8s-Container(C₁, P₁) ∧ K8s-Pod(P₂) ∧ K8s-Container(C2, P₂):
 addr(P₁, sa) ∧ addr(P₁, ta) ∧ open (C₂, tp)
 ⟹
 Send(e, C₁, sa, sp, ta, tp)
 ⟹
 Recv(e, C₂, sa, sp, ta, tp)

Constraints: Process to Pod

The Kubernetes network model requires that a process hosted on a node N, called Daemon D, can communicate with any container C executing in the context of a Pod P hosted on N through P's address.

 Container(C, P): host(N, D) ∧ host(N, P) ∧ addr(P, a) ∧ open (C, p)
 ⟹
 Send(e, D, _, _, a, p)
 ⟹
 Recv(e, C, _, _, a, p)

Kubernetes Networking as a Network Graph

This section uses the Kubernetes Network Graph, an ideal model, to describe the Kubernetes network model.

The following diagram describes the use case in this section: The Kubernetes cluster K1 consists of 2 nodes. Each node hosts 2 Pods. Each Pod executes 2 containers, one container listening on port 8080 and one container listening on port 9090. In addition, each node hosts 1 Daemon.

We can model a Kubernetes cluster network as a Graph with a set of nodes and a set of links.

node

Each K8S container C is mapped to a network Process C

 K8s-Pod(P) ∧ K8s-Container(C, P): Process(C)

Each Daemon D is mapped to a network Process C

 K8s-Daemon(D): Process(D)

Each K8s Pod P is mapped to the network Switch P, the Pod's Switch

 K8s-Pod(P): Switch(P)

Each K8S node N is mapped to network Switch N, the node's Switch:

 K8s-Pod(N): Switch(N)

Link

Each container C will be linked to its Pod Switch P

 K8s-Pod(P) ∧ K8s-Container(C, P): link(C, P)

Each Daemon D will be linked to its node Switch N

 K8s-Node(N) ∧ K8s-Daemon(D): host(N, D)  
 ⟹  
 link(D, N)

Each Pod Switch P will be connected to its node Switch N

 K8s-Node(N) ∧ K8s-Pod(P): host(N, P)  
 ⟹  
 link(P, N)

Each node Switch N1 will be connected to other nodes Switch N2

 K8s-Node(N₁) ∧ K8s-Node(N₂): N₁ ≠ N₂
 ⟹
 link(N₁, N₂)

In the forwarding information base of Pod Switch

P2 forwarding information base

 1. Delivery on localhost K8s-Pod(P) ∧ K8s-Container(C, P):
 open (C, p)
 ⟹
 [* * 127.0.0.1 p Deliver(C)] in FIB[P]
 2. Delivery on Pod Address
 K8s-Pod(P) ∧ K8s-Container(C, P):
 addr(P, a) ∧ open (C, p)
 ⟹
 [* * a p Deliver(C)] in FIB[P]
 3. Local Forwarding Rule  
 K8s-Node(N) ∧ K8s-Pod(P):
 host(N, P)
 ⟹
 [* * * * Forward (N)] in FIB[P]

In the forwarding information base of the node Switch

Forwarding information base N2

 {{{1. Node to Pod Forwarding Rule  
 K8s-Node(N) ∧ K8s-Pod(P):
 host(N, P) ∧ addr(P, a)
 ⟹
 [* * a * Forward (P)] in FIB[N]
 2. Node to Node Forwarding Rule  
 K8s-Node(N₁) ∧ K8s-Node(N₂) ∧ K8s-Pod(P):
 N₁ ≠ N₂ ∧ host(N₂, P) ∧ addr(P, a)
 ⟹
 [* * a * Forward (N₂)] in FIB[N₁] 
 
 }}}

Example

This section will explain the Life of a Message in the Kubernetes cluster network K1 through some examples.

Container to container

Container C1.1 needs to communicate with container C1.2:

• C1.1 is executed in the context of P1
• C1.2 Execution in the context of P1
• C₁.₁ connects to C₁.₂ via 127.0.0.1:9090

Intra-node Pod-to-Pod communication

Container C 1.1 needs to communicate with C 3.1:

• C 1.1 is executed in the context of P1 on the N1 node
• C 3.1 is executed in the context of P3 on the N1 node

• C 1.1 to C 3.1 via 10.1.1.2:8080

Pod-to-Pod Communication Between Nodes

Container C 1.1 needs to communicate with container C 2.1:

• C1.1 is executed in the context of P1 hosted on the N1 node
• C2.1 is executed in the context of P2 on node N2

• C1.1 to C2.1 via 10.1.2.1:8080

Daemon to Pod Communication

Daemon D1 needs to communicate with container C 1.1:

• D1 is hosted on node N1
• C 1.1 is executed in the context of Pod P1, which is hosted on Node N1

• D1 reaches C1.1 via 10.1.1.1:8080

Summarize

The Kubernetes network model is a permissive network model, that is, any network that satisfies the constraints of the Kubernetes network model is a valid Kubernetes network.

Mapping the Kubernetes network model to the Network Graph enables us to reason about the network at a conceptual level and skip a range of details required to reason at an implementation level.

In subsequent articles, we will use this Network Graph to discuss Kubernetes services, Kubernetes Ingress, and Kubernetes policies.