Why can TFO reduce TCP to 0 handshakes?

1. Overview

In the previous article, why TCP needs a three-way handshake to establish a connection, simple answers are given to the following three questions:

• Can application data be carried during the first handshake?
• Can application data be carried during the second handshake?
• Can application data be carried during the third handshake?

Simply put, traditional TCP requires three handshakes to establish a connection, and these three handshakes only send simple SYN and ACK messages.

From the perspective of network bandwidth resource utilization, the TCP header of the transport layer + the IP header of the network layer are at least 40 bytes. In order to send a message data packet of several bytes, an additional 40-byte header is assembled. This is somewhat similar to the so-called "confused window syndrome" mentioned earlier when talking about the TCP sliding window.

From the perspective of application optimization, because application layer data can only be sent after TCP has established a connection through a three-way handshake, there will be a certain delay when the application sends data for the first time. This side effect will be exacerbated in scenarios such as short connections and mobile devices.

So how do you solve this problem? Use the TFO solution.

📝 PS: Because there are differences between different versions of the TCP protocol stack, the premise of this article is that no data is transmitted during the TCP three-way handshake, that is, the traditional TCP three-way handshake. In fact, many Linux distributions provided by cloud computing service providers have optimized the network protocol stack and will directly send data during the third TCP handshake. When readers capture packets for verification, there may be some differences from the results of this article.

2. TFO

TCP Fast Open (TFO) is an optimization based on the traditional three-way handshake, allowing data to be sent during the handshake process, thereby reducing the delay in sending data for the first time and improving network application performance.

1. Implementation principle:

The core principle of TFO is to introduce a Cookie mechanism in the communication between the sender and the receiver, so that the sender can simplify the TCP three-way handshake when reconnecting to the receiver later.

As the name implies, the cookies in TFO Cookies are the same as the Cookie mechanism in the Web application layer. When you visit for the first time, you need to log in for verification. After verification by the server, you can carry it directly in subsequent visits without logging in again.

(1) First connection

• When the sender establishes a TCP connection with the receiver for the first time, it sends a SYN message.
• The receiver returns a SYN-ACK message with a randomly generated identifier called TFO Cookie to the sender.
• After receiving the SYN-ACK message, the sender saves the TFO Cookie and sends an ACK message to the receiver, completing the three-way handshake and starting data transmission.

(2) Subsequent connections

• When the sender connects to the same receiver again, it can carry the last saved TFO Cookie in the SYN message, and at the same time attach the application layer data in the SYN message (that is, send data directly during the first handshake)
• After the receiver verifies the sender's TFO Cookie, it sends the data to the application layer for processing and returns a SYN-ACK message (it can also send data at the same time)
• After receiving the SYN-ACK message, the sender sends an ACK message to the receiver to complete the three-way handshake.

2. Advantages

Through TFO, the sender can directly carry data when sending a SYN message, and the receiver can directly process the data during the first handshake and directly send the data during the second handshake. Finally:

• The sender sends data for the first time, reducing the delay by 1.5 RTT
• The receiver sends data for the first time, reducing the delay by 1 RTT

The sender sends data for the first time, reducing the delay by 1.5 RTT

The receiver sends data for the first time, reducing the delay by 1 RTT

3. Limitations

(1) Compatibility

Both communicating parties need to support TFO. If one of them does not support it, the connection automatically falls back to the traditional TCP connection establishment process. In addition, the forwarding devices (NAT, firewall) in the communication link will also implement this compatibility mechanism.

(2) Security

Although TFO cookies are generated by the receiver and sent to the sender, and each cookie is associated with the sender, it increases the security attack surface of the receiver and may cause security risks such as "TCP SYN Flood" amplification attacks.

If an attacker obtains a valid TFO Cookie from the compromised host and then forges a large number of data-carrying packets, the receiver will need a large amount of memory to temporarily store application data, eventually leading to memory exhaustion.

(3) Deployment environment requirements

There are requirements for the kernel version and the kernel parameters need to be modified.

(4) Application data is too large

If the data that the sender wants to send for the first time is larger than the MSS of TCP, it still needs to be unpacked and sent multiple times. When the application data is too large, the advantage of TCP Fast Open (RTT reduction) can be almost ignored.

3. Simulation Environment

TFO requires support from both the sender and the receiver. If either party does not support TFO, the connection will automatically fall back to the traditional three-way handshake.

To demonstrate the effect, the author used two Linux servers as the communication sender and receiver. The corresponding distribution version and kernel version parameters are as follows.

1.TFO requires the Linux kernel version: >= 3.7.

 # 发送方## 发行版本(WSL2 环境) $ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian ## 内核版本$ uname -r 5.10.0-21-amd64 # 接收方## 发行版本$ cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ## 内核版本$ uname -r 3.10.0-1160.53.1.el7.x86_64

2. Kernel parameter adjustment

TFO enabling requires modifying the default kernel parameters:

• 0: Disable TFO
• 1: Enable sender mode TFO
• 2: Enable receiver mode TFO
• 3: Enable TFO in both sender and receiver modes

 # 发送方启用TFO $ echo 1 | sudo tee /proc/sys/net/ipv4/tcp_fastopen # 接收方启用TFO # 写入3 表示既启用发送方TFO 也启用接收方TFO $ echo 3 | sudo tee /proc/sys/net/ipv4/tcp_fastopen

As a simulation experiment, the author only temporarily modified the parameters. You can use the following steps to make the configuration permanent:

(1) Edit the /etc/sysctl.conf file and add configuration items

 net.ipv4.tcp_fastopen=3

(2) Run the sysctl -p command to make the setting effective. It is still effective after reboot.

4. Program Code

If the reader uses a higher version of curl on the host, he can directly enable TFO mechanism access using the following method:

 $ curl --tcp-fastopen http://example.com # 可以使用如下方式确认curl 版本是否支持TFO $ curl -V | grep -i TFO

Because the curl version in the server I use is low, I write Python script code here. The core code is actually just the setting of the parameters of 2 sockets.

1. Receiver (server) code

The receiver is implemented as a server program, binds/listens to a specified port, and then receives TCP connections from the sender (client).

 # service.py import socket def listen(): # 初始化服务端监听对象listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # 参数32 表示TCP 协议栈# 未完成的Fast Open 队列长度listener.setsockopt(socket.SOL_TCP, socket.TCP_FASTOPEN, 32) # 监听12345 端口号# 为了模拟，所以不用主流端口号了:-) listener.bind(('0.0.0.0', 12345)) # 最大连接数设置为1024 listener.listen(1024) print("Server is listening on port 12345...") # 轮询接收新的TCP 连接while True: conn, addr = listener.accept() print(f"Accepted connection from {addr}") print(f"Received data: {conn.recv(1024)}") conn.send(b"Hello, Client") conn.close() print(f"Closed connection with {addr}") if __name__ == "__main__": try: # 启动监听listen() except KeyboardInterrupt: # 捕获Ctrl + C 终止程序print("Server shutting down...")

2. Sender (client) code

 # client.py import socket # 初始化客户端监听对象sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # 参数32 表示TCP 协议栈# 未完成的Fast Open 队列长度sock.setsockopt(socket.SOL_TCP, socket.TCP_FASTOPEN, 32) # 向服务端发送数据时# 设置Fast Open 选项sock.sendto(b"Hello, Server", socket.MSG_FASTOPEN, ("104.21.71.166", 12345)) print(f"Received data: {conn.recv(1024)}") sock.close()

5. Run the program experiment

The core code of the program (2 lines in total) is ready. Next, start running the program to verify the TCP Fast Open process.

Server public IP: 104.21.71.166

1. Start the server program and confirm the monitoring status

 # 在1 个终端启动服务端程序$ python3 server.py # 在另外1 个终端查看程序监听状态是否正常$ netstat -ant | grep 12345 | grep LISTEN tcp 0 0 0.0.0.0:12345 0.0.0.0:* LISTEN

2. The client starts capturing packets

Open WireShark and monitor the corresponding network card device.

3. Run the client program

 # 为了验证效果，这里可以连续运行几次# 每次运行间隔3 - 5 秒即可$ python3 client.py # 输出省略...

4. Check the client TCP connection status

 netstat -ant | grep 12345 | grep TIME_WAIT # 输出如下# 连续运行了多少次client.py # 就会产生多少TIME_WAIT 状态的TCP 连接# 10.0.0.53 为客户端的内网IP 地址tcp 0 0 10.0.0.53:38084 104.21.71.166:12345 TIME_WAIT tcp 0 0 10.0.0.53:37530 104.21.71.166:12345 TIME_WAIT tcp 0 0 10.0.0.53:37528 104.21.71.166:12345 TIME_WAIT tcp 0 0 10.0.0.53:38076 104.21.71.166:12345 TIME_WAIT tcp 0 0 10.0.0.53:38078 104.21.71.166:12345 TIME_WAIT ...

Everything is running normally. Next, you can go to see the packet capture results of WireShark.

6. WireShark packet capture results

First, use the tcp.options.tfo filter condition to quickly filter out TCP packets related to TCP Fast Open.

The following is an analysis of the WireShark packet capture results.

1. Establishing a connection for the first time

When the sender establishes a TCP connection with the receiver for the first time, it sends a SYN message and sets the TCP Options field to TCP Fast Open.

No data is sent at this time, so Len = 0 in the WireShark packet capture result.

When the receiver returns a SYN-ACK message, it also attaches a randomly generated identifier called TFO Cookie to the sender.

After receiving the SYN-ACK message, the sender saves the TFO Cookie and sends an ACK message to the receiver, completing the three-way handshake.

The value of the TFO cookie is: d82d9074a6105a13.

After the three-way handshake is completed, data transmission begins.

2. Establish a connection later

From the screenshot, we can see that when the client and server establish a TCP connection, they carry the FTO Cookie during the first handshake and send data directly, so Len = 13 in the WireShark packet capture result.

So what is this 13? It is the data sent by the client, which is exactly 13 bytes.

 conn.sendto(b"Hello, Server", ...)

When the subsequent TCP connection is established (first handshake), data can be sent directly (due to space limitations, only two data capture details are captured here):

The value of the TFO Cookie in each data packet is d82d9074a6105a13, which is the value carried by the server when sending the SYN-ACK message when the TCP connection is first established.