A brief analysis of traffic management of Istio component Envoy

Background

The development convenience brought by the microservice architecture has significantly shortened the development cycle of business functions. Through the native optimization of the cloud computing platform architecture, the continuous integration and delivery of business functions have become more agile. However, the microservice architecture has also introduced many problems in service governance: an application consists of multiple services, each service has several instances, and the running status of each instance changes in real time. These have given rise to the emergence of the inter-service communication layer. The communication layer is not coupled with the application code, but can capture the dynamic changes of the underlying environment and make appropriate adjustments to avoid single point failures in the business.

1. Introduction to ServiceMesh

1.1 Introduction to Service Mesh

The service mesh is an infrastructure layer that handles inter-service communication. Cloud-native applications have complex service topologies, and the service mesh is responsible for implementing reliable delivery of requests in these topologies. In practice, the service mesh is usually implemented as a set of lightweight network proxies that are deployed with the application but are transparent to the application.

From a local perspective, service mesh technology is to deploy a proxy on the application node. The application sends the request to the proxy, and the proxy completes the point-to-point routing forwarding.

In the above figure, if we remove the applications on the left, only the proxies and the calling relationships between them are shown (i.e. the right figure). At this time, the concept of Service Mesh will become clear: the proxies and calling relationships form a complete network, representing the complex calling relationships between services, and carrying all applications in the system.

1.2 Characteristics and advantages of service network architecture

1) Point-to-point communication: no central bottleneck.

2 ) No intrusion to applications: It can support the integration of heterogeneous technology products. At the same time, it is transparent to applications, and application developers no longer need to worry about complex network communication implementations, and can focus on the implementation of business logic.

2. Introduction to Istio and Envoy

Istio is an open source Service Mesh framework developed by Google, IBM, and Lyft. It has become the de facto technical standard for Service Mesh and is widely used in IT architectures in various industries.

Envoy is a high-performance proxy developed in C++. It has built-in service discovery, load balancing, TLS termination, HTTP/2, GRPC proxy, fuse, health check, grayscale release based on percentage traffic splitting, fault injection and other functions. It is used to coordinate the inbound and outbound traffic of all services in the service mesh.

3. The principle of Envoy traffic management

3.1 Introduction to Iptables

Istio calls iptables in Linux for traffic management. Iptables is an application software running in user space. It manages the flow and forwarding of network packets by controlling the netfilter module of the Linux kernel. In fact, netfilter is the real security framework of the firewall. Netfilter is the cornerstone of Linux network security. It provides a complete set of hook function mechanisms. The five hook points of the IP layer correspond to the five built-in chains of iptables:

• PREROUTING: In this DNAT.
• POSTROUTING: In this SNAT.
• INPUT: Processes packets input to the local process.
• OUTPUT: Processes packets output by local processes.
• FORWARD: Handles packets forwarded to other machines and other network namespaces.

3.2 About inbound IP packets

IP packets coming in from the network first enter the TREOUTING chain, and then perform routing judgment:

1) If the packet routing destination is the local machine: it enters the INPUT chain and then sends it to the local process.

2 ) If the packet routing destination is not the local machine and IP forwarding is enabled, it enters the FORWARD chain, then passes through the POSTROUTING chain, and finally is sent out through the network interface.

3 ) For packets sent from the local process to the protocol stack, they first pass through the OUTPUT chain, then through the POSTROUTING chain, and finally through the network interface.

3.3 About custom chains

In addition, we can also customize chains, but customized chains can only work if they are called as actions by a default chain.

In Kubernetes, Istio automatically injects Envoy Sidecar through the Admission webhook mechanism and runs it in the same Pod as the application container. In this case, they will share the network namespace and therefore use the same network protocol stack.

The configurations that Istio injects into application Pods mainly include:

1) Init container istio-init: used to set iptables port forwarding in Pod.

2) Sidecar container istio-proxy: runs the Envoy Sidecar proxy.

3.4Iptables configuration rules

After the container is initialized, we enter the Sidecar container and switch to the root user to view the configured iptables rules.

 iptables -t nat -S 

ISTIO_INBOUND chain: redirects all traffic entering the Pod but not the specified port (such as 22) to port 15006 (Envoy inlet traffic port) for interception.

ISTIO_OUTPUT chain: redirect all Pod outflow traffic sent by istio-proxy user space and whose destination is not localhost to port 15001 (envoy egress traffic port). All other traffic is directly released to the next POSTROUTING chain without being intercepted and processed by Envoy.

The overall traffic flow diagram is shown in the following figure:

1) Inbound traffic entering the Pod is first intercepted and processed by the PREROUTING chain.

2 ) When TCP requests enter the PREROUTING chain, they are all handed over to ISTIO_INBOUND for processing.

 - A PREROUTING - p tcp - j ISTIO_INBOUND

3 ) All TCP requests with target ports other than 15008/22/15090/15021/15020 are handled by ISTIO_IN_REDIRECT.

 - A ISTIO_INBOUND - p tcp - j ISTIO_IN_REDIRECT

4 ) Redirect all TCP requests sent here to port 15006 (Envoy inlet traffic port)

 - A ISTIO_IN_REDIRECT - p tcp - j REDIRECT --to-ports 15006

5) After being processed internally by Envoy, it decides to forward the data packet to the application. This step is considered egress traffic for Envoy and will be intercepted by netfilter and forwarded to the egress traffic OUTPUT chain.

6 ) Outbound requests, when TCP requests enter the OUTPUT chain, they are all handed over to ISTIO_OUTPUT for processing.

 - A OUTPUT - p tcp - j ISTIO_OUTPUT

7 ) Match the outbound request corresponding rule, request the local service, the exit is the lo network card and comes from the istio-proxy user space, and the traffic returns to the next chain in its call point, that is, the POSTROUTING chain.

 -A ISTIO_OUTPUT -m owner --uid-owner 1337 -j RETURN -A ISTIO_OUTPUT -m owner
 --gid-owner 1337 -j RETURN

8 ) The request sent by Sidecar reaches the target application.

9 ) After the target application processes the business logic, it responds to the Sidecar. This step belongs to the egress traffic for the application and is intercepted by netfilter again and forwarded to the egress traffic OUTPUT chain.

10 ) Outbound requests, when TCP requests enter the OUTPUT chain, they are all handed over to ISTIO_OUTPUT for processing.

 - A OUTPUT - p tcp - j ISTIO_OUTPUT

11 ) Request the next service/response request, that is, request a non-local service and not from the istio-proxy user space, and the traffic is forwarded to the ISTIO_REDIRECT chain.

 - A ISTIO_OUTPUT - j ISTIO_REDIRECT

12 ) Redirect all TCP protocol request traffic redirected here to port 15001 (Envoy egress traffic port).

 - A ISTIO_REDIRECT - p tcp - j REDIRECT --to-ports 15001

13 ) After being processed internally by Envoy, it is decided to forward the data packet externally. This step is considered egress traffic for Envoy and will be intercepted by netfilter and forwarded to the egress traffic OUTPUT chain.

14 ) Outbound requests, when TCP requests enter the OUTPUT chain, they are all handed over to ISTIO_OUTPUT for processing.

 - A OUTPUT - p tcp - j ISTIO_OUTPUT

15 ) Requesting non-local services, the export is not the lo network card and comes from the istio-proxy user space, then the ISTIO_REDIREC process is skipped and RETURN is directly sent to the next chain, the POSTROUTING chain

 -A ISTIO_OUTPUT -m owner --uid-owner 1337 -j RETURN -A ISTIO_OUTPUT -m owner
 --gid-owner 1337 -j RETURN

16 ) After the POSTROUTING chain is processed, the appropriate network card is selected according to the routing table to send outbound traffic.

4. Summary

The core work of Envoy is to intercept business-transparent requests and manage all inbound and outbound traffic. After certain rules are applied to the intercepted requests for security access control, access control, traffic control, and many other aspects, they are sent to the application. By using Envoy, developers can focus on the development of application functions without having to consider complex network communications.