The fatal factor affecting TCP connection throughput: HOL

1. What is HOL

HOL means Head of line blocking. In the Internet field, it includes HTTP head of line blocking and TCP head of line blocking.

1. HTTP Head of line blocking

Wikipedia explains HTTP HOL as follows:

Although HTTP/2 solves the HOL problem of HTTP/1.1, since it is built on TCP, the HOL problem at the TCP level still exists.

2. TCP Head of line blocking

TCP HOL is defined as follows:

For example, in the TCP receive buffer, if the first data packet of the request has not arrived for some reason, the subsequent data packets in the buffer cannot be passed to the upper-layer application and can only wait until the first data packet arrives, and then they can be passed to the upper-layer application together.

The problems caused by TCP HOL are universal. As long as TCP is used, there will be such problems, so the impact is very wide.

2. Case Analysis

1. Network switches

The above figure shows how different input queues in a network switch wait for HOL.

The first and third input queues are competing for the use of Output 4. If the switch chooses to pass the packet from the third input queue, the packet from the first input queue can only wait. The packet with sequence number 3 in the first input queue can only wait because of HOL, even though Output 3 is idle.

2. Single connection vs multiple connection experiments

Below is the throughput with concurrency set to 1, which is 153.60 reqs/sec.

Below is the throughput with a concurrency of 10, which is 145.08 reqs/sec.

Here, the throughput of 10 concurrent connections is lower than that of 1 concurrent connection because the test environment is a Docker environment with poor hardware configuration.

The above test was conducted in a network environment with no packet loss. What happens in the case of network packet loss?

We simulate a network environment with a packet loss rate of 1%.

The throughput with concurrency of 1 drops to 37.41 reqs/sec.

The throughput with a concurrency of 10 is 132.20 reqs/sec.

We found that multiple concurrency is more effective in the case of packet loss. This is because the HOL blocking problem is alleviated to a certain extent in the case of multiple concurrency, while the HOL blocking problem is obvious in the case of a single connection.

3. Interaction between tcpcopy and intercept

For a long time, the interaction between tcpcopy and intercept only used one connection. When the test pressure is relatively large, the network will deteriorate, and it is easy to lose packets or receive data in time, resulting in a large amount of information being blocked.

Later, in order to solve the problem of HOL caused by a single connection, multiple connections were used, and the performance was greatly improved.

4. Java netty game framework

The following figure shows a Java netty framework.

A thread is used to process both read data and write data. The thread is busy processing read events for a while, and processing write events for a while. When processing read events intensively, the data to be written is blocked, and when processing write events intensively, the read events are blocked. When the pressure is high, the delay will become very large.

3. Problems caused by HOL blocking

IV. Conclusion

In the program architecture, try to use multiple connections to handle various events, otherwise it is easy to encounter TCP HOL blocking problems.