Four waves, closing the TCP connection

[[428968]]

This article is reprinted from the WeChat public account "Xiaocai Learns Programming", written by fasionchan. Please contact Xiaocai Learns Programming public account for reprinting this article.

In the previous section, we conducted an experiment to deeply study the process of establishing a connection using the TCP three-way handshake.

After we exit the telnet command, TCP will close the connection. At the same time, we also observe the communication process of TCP closing the connection through tcpdump. In this section, we continue to study the communication details of TCP closing the connection.

The communication process in the previous experiment has been captured and saved. We can directly open it with the tcpdump command (tcp.pcap):

 root@client [ ~ ] ➜ tcpdump -nr tcp.pcap
 reading from file tcp.pcap, link-type LINUX_SLL (Linux cooked v1)
 17:31:57.624391 ARP, Request who-has 10.0.0.2 tell 10.0.0.3, length 28
 17:31:57.624439 ARP, Reply 10.0.0.2 is - at 0e:bd:60:1c:69:9d, length 28
 17:31:57.624450 IP 10.0.0.3.55692 > 10.0.0.2.22: Flags [S], seq 386101196, win 29200, options [mss 1460, sackOK, TS val 811948031 ecr 0, nop, wscale 7], length 0
 17:31:57.624495 IP 10.0.0.2.22 > 10.0.0.3.55692: Flags [S.], seq 1155103769, ack 386101197, win 28960, options [mss 1460, sackOK, TS val 3541712191 ecr 811948031,nop,wscale 7], length 0
 17:31:57.624522 IP 10.0.0.3.55692 > 10.0.0.2.22: Flags [.], ack 1, win 229, options [nop,nop,TS val 811948031 ecr 3541712191], length 0
 17:31:57.635739 IP 10.0.0.2.22 > 10.0.0.3.55692: Flags [P.], seq 1:42, ack 1, win 227, options [nop,nop,TS val 3541712202 ecr 811948031], length 41
 17:31:57.635778 IP 10.0.0.3.55692 > 10.0.0.2.22: Flags [.], ack 42, win 229, options [nop,nop,TS val 811948042 ecr 3541712202], length 0
 17:31:59.808411 IP 10.0.0.3.55692 > 10.0.0.2.22: Flags [F.], seq 1, ack 42, win 229, options [nop,nop,TS val 811950215 ecr 3541712202], length 0
 17:31:59.809175 IP 10.0.0.2.22 > 10.0.0.3.55692: Flags [.], ack 2, win 227, options [nop,nop,TS val 3541714376 ecr 811950215], length 0
 17:31:59.809464 IP 10.0.0.2.22 > 10.0.0.3.55692: Flags [F.], seq 42, ack 2, win 227, options [nop,nop,TS val 3541714376 ecr 811950215], length 0
 17:31:59.809483 IP 10.0.0.3.55692 > 10.0.0.2.22: Flags [.], ack 43, win 229, options [nop,nop,TS val 811950216 ecr 3541714376], length 0

Obviously, the last four packets are the process of waving four times to close the connection:

When we press Ctrl-D to exit the telnet command, the client sends a FIN packet to the server. This is a TCP packet with the FIN flag set, which tells the server that the data on the client side has been sent and is ready to close the connection:

After receiving the FIN packet, the server will reply with an ACK to confirm. Note that the confirmation number is based on the FIN packet sequence number plus one, because FIN also occupies a sequence number, just like SYN.

At the same time, the server notifies the upper-layer application, the SSH service process, of the closure of the connection read end. At this point, the transmission direction from the client to the server in the TCP connection has been closed, and the connection is in a semi-closed state.

The gray part in the figure above is the closed transmission direction, which is the write end for the client and the read end for the server. Data transmission in the other direction can still proceed normally, so the server can continue to send data (write end) and the client can also receive data sent by the server (read end).

After the SSH service process learns that the client has closed the connection, it prepares to end the service and close the connection. If it still has data to send, it can still send it to the client through the half-open connection. When all data has been sent, the server also sends a FIN packet to tell the client that the connection is closed.

A TCP connection contains two transmission channels in two directions, so two pairs of FIN/ACK packets are required, each responsible for closing the corresponding direction. Therefore, these two pairs of FIN/ACK interactions are also figuratively called four waves.

State Transition

TCP requires three handshakes to establish a connection, and four handshakes to close the connection, which is a relatively cumbersome process. This means that a TCP connection should have many intermediate states. Let's take a closer look:

The above figure is a timing diagram of the entire life cycle of a TCP connection. The left side is the client's timeline, and the right side is the server's timeline. The different colors on the timeline represent the status of the client and server connections:

1. The client sends a SYN packet and the connection enters the SYN_SENT state;
2. The server receives the SYN packet from the client, it replies with a SYN/ACK packet, and the connection enters the SYN_RECV state;
3. The client receives the SYN/ACK packet from the server, it replies with an ACK packet, and the connection enters the ESTABLISHED state;
4. The server receives the ACK packet from the client, and the server connection also enters the ESTABLISHED state;
5. When the connection is in the ESTABLISHED state, the client and server can transfer data to each other;
6. The data packet in the middle of the timing diagram and the ACK packet after it are the version information returned by the SSH service to the client in the experiment (this part of the data is directly output to the screen by the telnet command);
7. When the client is ready to exit, it notifies the server through a FIN packet, and the connection enters the FIN_WAIT1 state;
8. The server receives the FIN packet from the client, it replies with an ACK packet to confirm, and the connection enters the CLOSE_WAIT state;
9. The client receives the ACK packet from the server and the connection enters the FIN_WAIT2 state;
10. At this time, the connection is in a semi-closed state, and the server can still send data to the client;
11. After the server sends the remaining data, it sends a FIN packet to the client to notify the client to close the connection, and the server connection enters the LAST_ACK state;
12. The client receives the FIN packet from the server and replies with an ACK packet for confirmation. The client connection enters the TIME_WAIT state;
13. After the server receives the ACK packet, the connection is completely closed;
14. Since the last ACK packet may be lost, the client must wait for a while in the TIME_WAIT state to confirm the FIN packet retransmitted by the server;

At this point, we can get a complete TCP state transition diagram:

The state transition paths of the TCP active connection party (client) and the passive connection party (server) are different: the green path in the figure is the state transition path of the client under normal circumstances; and the red path is the transition path of the server under normal circumstances. As can be seen from the figure, the state transition of the active closing party is also much more complicated than that of the passive closing party.

According to the introduction of TCP/IP Detailed Explanation, the TCP protocol also supports establishing a connection from the server. However, since this implementation is actually very rare, it will not be introduced in depth here.

State transition is an important knowledge point of the TCP protocol, especially the understanding of the TIME_WAIT state, which is often tested in backend technical interviews.

TIME_WAIT state

The TCP active closing party will eventually enter the TIME_WAIT state and maintain a duration of 2MSL. Why?

As shown in the figure above, if the last ACK packet in the four waves is lost in the network, what will happen? At this time, the passively closed server will retransmit the FIN packet. Therefore, the active closing party cannot close directly, but should maintain the TIME_WAIT state for a period of time to reply the ACK packet to the retransmitted FIN packet. Otherwise, the other end will still retransmit the FIN packet and give up after retrying several times. At this time, the connection can only be closed abnormally.

On the other hand, a TCP connection is uniquely identified by a 4-tuple consisting of the IP addresses and ports of both communicating parties. What will happen if the active closing party does not wait for a while in the TIME_WAIT state, but quickly closes and releases resources? At this time, the original 4-tuple may be reused by the new connection, and the retransmitted FIN packet of the old connection may be delayed due to network reasons, eventually causing conflict interference to the new connection.

So, why does it maintain the TIME_WAIT state for 2MSL?

MSL is short for maximum segment lifetime, which is the maximum time a TCP packet can exist in the network before being discarded. This time is definitely limited because the IP packet responsible for transmitting the TCP packet has a TTL field that limits the survival time. Since the IP packet's limit on survival time is based on the number of hops, the two are not equivalent. Different network protocol stack implementations have different MSL values: 30 seconds, 1 minute, or 2 minutes.

The TIME_WAIT state is maintained for 2MSL for the following reasons:

Assume that the last ACK packet arrives at the peer host just before the survival time expires, and the MSL time has passed. After receiving the ACK, the peer will immediately close the connection and will not be able to send a FIN packet. However, if the peer has just retransmitted a FIN packet before receiving the ACK, the MSL time must pass to ensure that the FIN packet disappears from the network. Therefore, the connection must be maintained in the TIME_WAIT state for 2MSL time before it can be released, otherwise it may interfere with potential new connections.

You may ask, if the last ACK packet is lost, won't the other end continue to retransmit the FIN packet? Doesn't it need to wait for the FIN to fail?

Yes, the other end will definitely retransmit the FIN packet, and usually start retransmitting very quickly. MSL is usually tens of seconds, while the network round trip time is much smaller, usually only in milliseconds, and the TCP retransmission time is of similar order of magnitude. Therefore, within the MSL time, TCP can retransmit the FIN packet several times! If one of the FIN packets can arrive, TCP will reset the timer and maintain the TIME_WAIT state for another 2MSL. If several FIN packets are lost, there is no point in waiting any longer!

• If the last ACK packet can reach the other end, it only needs to wait for 2MSL time at most to ensure that there is no FIN packet retransmitted by the other end in the network;
• If the last ACK packet is lost, the peer has retransmitted it several times within the MSL;
  • If one of the retransmitted FIN packets can reach the local end, TCP will reset the timer after replying ACK and continue to wait for 2MSL in TIME_WAIT;
  • If all retransmitted FIN packets are lost, it means the network quality is very poor and there is no point in waiting any longer;

Therefore, after receiving the FIN packet from the other end, the active closing party must wait for 2MSL in the TIME_WAIT state before releasing the connection. This ensures that the retransmitted FIN packet is replied to and that the retransmitted FIN packet disappears from the network, without causing conflict interference to the new connection of the multiplexed quadruple.

The active close party is usually the client, and the concurrency is generally not high, so the TIME_WAIT state will basically not cause any impact. If a high-concurrency service (such as a Web service) has a large number of short connections, it may leave many connections in the TIME_WAIT state. Since the TIME_WAIT state sockets cannot be recycled immediately, they will occupy a large amount of system resources and have a serious impact on the performance of the service.

At this time, the system administrator can choose to shorten the system's 2MSL duration appropriately to speed up the cleanup of TIME_WAIT connections. In addition, in Linux systems, the tcp_tw_recycle and tcp_tw_reuse kernel options can be enabled to reuse sockets in the TIME_WAIT state. These belong to the scope of TCP and system tuning, and will be introduced in detail later.