A brief discussion on common tunneling technologies: IPSec

IPSec is not a single protocol, but a set of network security protocols, including the network authentication protocol AH (Authentication Header), ESP (Encapsulating Security Payload), the key management protocol IKE (Internet Key Exchange), and some user network authentication and encryption algorithms.

IPSec working modes are divided into two types: transport mode and tunnel mode. In simple terms, the transport mode is that the original layer 2 data packet is no longer attached with the layer 2, 3, and 4 headers, and the tunnel mode is that the original layer 2 data packet is encapsulated through the protocol tunnel and the layer 2, 3, and 4 headers are added. The tunnel mode of IPSec is to encapsulate another layer of IP header outside the original IP data packet, so IPSec is often called the three-layer tunnel protocol. The following will take you to learn about some of these specific protocols in detail, as well as the NAT traversal implementation that occupies an important position in the actual application of the IPSec protocol.

Part 01. Security Protocol - AH Protocol

The AH protocol is an IP-based transmission protocol with a protocol number of 51. The specific working method is to add an AH header after the standard IP header of each data packet:

The sender of the AH protocol will perform hash calculations on the data packet and the authentication key. After receiving the message, the receiver will perform hash calculations according to the same algorithm and compare the results with the original calculations. If they are inconsistent, it can be inferred that the data packet has been modified or destroyed during transmission. In this way, data source authentication and data integrity verification can be provided. It is worth mentioning that the integrity verification scope of the AH protocol is the entire IP message.

There are several important fields in the AH message header that deserve attention: the Security Parameter Index (SPI) is used to uniquely identify the IPSec security association, and the sequence number uniquely identifies each data packet and can be used to prevent replay attacks.

Part 02, Security Protocol - ESP Protocol

Like the AH protocol, the ESP protocol is also an IP-based transport layer protocol with a protocol number of 50. The specific working method is to add an ESP header after the IP header of each data packet. It is worth noting that an ESP tail (ESP Tail and ESP Auth Data) is also added to the end of the data packet. At the same time, unlike the AH protocol, the ESP protocol only encrypts the payload of the IP data packet, and does not encrypt the IP header.

Similar to the AH protocol, the ESP message header also has two fields: Security Parameter Index (SPI) and Sequence Number. In addition, the authentication algorithms of the AH protocol and the ESP protocol are the same.

The specific comparison between AH protocol and ESP protocol is as follows:

Summary: The AH protocol cannot provide data packet encryption, and the ESP protocol verification scope does not include the IP header. Therefore, in scenarios with higher security requirements, you can consider using the AH protocol and ESP protocol together.

Part 03, IKE Protocol

Simply put, the IKE protocol is a protocol for dynamically negotiating IPSec tunnels. It can complete identity authentication, key exchange, and generate IPSec SA. During the negotiation process, whether the data packet will be encapsulated using the AH protocol or the ESP protocol and identity authentication is determined.

There are currently two versions of the IKE protocol: IKEv1 and IKEv2. Based on v1, IKEv2 not only simplifies the SA negotiation process and improves negotiation efficiency, but also fixes many recognized cryptographic security vulnerabilities and improves security performance. Therefore, IKEv2 is more widely used in practice.

Taking IKEv2 as an example, the first pair of IPSec SA can be negotiated and established through the initial exchange, which includes two exchanges of four messages, including parameter negotiation such as encryption and verification algorithms, generation of shared keys, and completion of identity authentication and message authentication. If multiple pairs of IPSec SA need to be created, they can be negotiated through the creation of sub-SA exchange processes. At the same time, there is some control information transmission during the negotiation process, such as error messages or notification messages, which are completed through notification exchanges.

Part 04, NAT Traversal

The IPSec protocol can be widely used. In addition to providing secure and encrypted transmission, another important reason is that it can achieve NAT traversal, which is extremely important in existing network transmission. Because public network IP resources are limited, most devices share public network IP resources after NAT conversion to transmit and exchange messages, so NAT traversal is extremely important in existing network applications.

As described above, the characteristics of the AH and ESP protocols, we found that the AH protocol cannot traverse NAT because NAT will modify the IP header of the message, but the AH integrity check is based on the entire IP message, so IPSec under the AH protocol cannot traverse NAT. The integrity check of the ESP message does not include the IP header, and the IP address conversion will not destroy the hash value of ESP. Therefore, in the NAT scenario where only IP conversion is performed, the ESP protocol encapsulation supports NAT traversal. However, in many cases, the public network IP is shared, so NAT conversion not only needs to convert the IP, but also the port. However, the ESP protocol encrypts the IP payload, making it impossible to modify the port number. This is also the fundamental reason why many IP in IP tunnels cannot traverse NAT. The solution is to add another UDP message header - NAT-T (NAT Traversal), and the source and destination port numbers are both 4500.

NAT-T tunnel can solve the problem of IPSec traversing NAT, but there are still two problems after traversing NAT: First, the problem of identity authentication and IP address reuse after traversing NAT. In the current IP network, IP is the identity identifier, but IP will change after NAT. At present, the main method used in China is to replace IP address with string as the identity identifier, which is not affected by NAT. Second, the ESP protocol is an IP-based protocol. When there are multiple ESP application terminals behind the NAT gateway and the address is reused, it is impossible to perform reverse mapping based on the IP protocol number alone, and the ESP protocol must be changed. This will be specifically reflected in the NAT-T process. Let's explain the NAT-T process in detail below .

First, determine whether both parties support NAT-T. When NAT traversal is enabled, the vendor ID payload that identifies the NAT-T capability will be sent during the IKE negotiation process to check whether the communicating parties support NAT-T. Only when both parties include this payload in their respective messages will the relevant NAT-T negotiation be carried out later. The second step is to determine whether there is a NAT device on the link. During the IKE negotiation, a NAT-D (NAT Discovery) payload will be sent. This payload is used to detect NAT devices. Both parties will send the hash value of the source and destination IP and port to each other and store it in this payload. If it changes during the transmission process, the hash value will change accordingly. By determining the hash value, it can be determined whether there is NAT conversion in the link. Finally, ESP processing is performed . Under the premise of the first two steps, when the NAT gateway is found, the port number of the subsequent IKE message is converted to 4500.

Although the above technology can achieve NAT traversal, it also has certain limitations: using the NAT-T function will increase the overhead of about 200 bytes during the IKE negotiation process and the overhead of about 20 bytes during data transmission; at the same time, the AH protocol cannot be used, which will reduce security to a certain extent; the NAT device cannot guarantee that the fixed IP and fixed port always provide access mapping for internal hosts, so IPSec must be able to automatically detect changes to ensure that communication is always smooth.

Part 05, Postscript

Since IPSec was developed in 1990, it has been used for more than 30 years. The development of technology has become more mature after generations of accumulation, and the basic exchange process is relatively complete. The future development direction is likely to be in the authentication and encryption algorithms. The US NIST also released a special publication "IPSec VPNs Guide" in 2020. The article pointed out that IETF is focusing on studying various IKE and IPSec extension topics, and also introduced efforts in multicast and group authentication, implicit IV in ESP, and post-quantum key exchange. In the future, we will pay attention to the latest IPSec technology in a timely manner, apply it in actual production and work, and contribute to China Mobile's new network architecture.