What exactly is HTTP3 that international companies such as Google are beginning to support?

There has been a lot of news about HTTP/3 in recent times, and more and more large international companies have begun to use HTTP/3.

Therefore, HTTP/3 is already on the way, and it is only a matter of time before it is fully used. So, as front-line developers, it is time for us to understand what HTTP/3 is and why we need it.

So, this article will explain what exactly is HTTP/3? What technologies does it use? What problems does it solve?

Problems with HTTP/2

Before writing this article, I wrote a special article titled "What did HTTP/2 do wrong? Is it going to be abandoned after just two years of glory?" to analyze the problems with HTTP/2 and the reasons behind them.

I won’t go into detail here. I strongly recommend that you read this article first, which will help you learn this article.

In the previous article, we mentioned that HTTP/2 has problems such as TCP head-of-line blocking, long TCP handshake delay, and protocol rigidity because the underlying transport layer protocol is still TCP.

This means that although HTTP/2 uses technologies such as multiplexing and binary framing, there is still room for optimization.

QUIC Protocol

We know that the reason why HTTP/2 is "deprecated" is that the transport layer protocol it uses is still TCP, so the first problem that HTTP/3 solves is to bypass TCP.

If a new protocol is developed, it will still be affected by the rigidity of the intermediate devices and cannot be widely used. Therefore, the researchers came up with a way to implement it based on UDP.

So, Google was the first to adopt this approach and put it into practice. In 2013, they launched a protocol called QUIC, the full name of which is Quick UDP Internet Connections.

As the name suggests, this is a protocol that is completely based on UDP.

At the beginning of the design, Google hoped to use this protocol to replace the HTTPS/HTTP protocol to speed up the transmission of web pages. In June 2015, the QUIC network draft was officially submitted to the Internet Engineering Task Force. In October 2018, the Internet Engineering Task Force HTTP and QUIC Working Group officially renamed HTTP over QUIC protocol to HTTP/3.

Therefore, the HTTP/3 we are talking about now is actually HTTP over QUIC, that is, HTTP implemented based on the QUIC protocol.

So, if you want to understand the principles of HTTP/3, you only need to understand QUIC.

The QUIC protocol has the following characteristics:

• UDP-based transport layer protocol: It uses UDP port numbers to identify a specific server on a specific machine.
• Reliability: Although UDP is an unreliable transport protocol, QUIC has made some modifications based on UDP, making it provide reliability similar to TCP. It provides packet retransmission, congestion control, adjustment of transmission rhythm and other features that exist in TCP.
• Implemented unordered, concurrent byte streams: QUIC's single data stream can guarantee ordered delivery, but multiple data streams may be out of order, which means that the transmission of a single data stream is in order, but the order received by the receiver in multiple data streams may be different from the order sent by the sender!
• Fast handshake: QUIC provides 0-RTT and 1-RTT connection establishment
• Use TLS 1.3: TLS 1.3 has many advantages over earlier TLS versions, but the main reason to use it is that its handshake takes fewer round trips, which can reduce the latency of the protocol.

So, which layer of the TCP/IP protocol suite does QUIC belong to? We know that QUIC is implemented based on UDP and is the protocol that HTTP/3 depends on. So, according to the TCP/IP layering, it belongs to the transport layer, that is, the same layer as TCP and UDP.

If we look at it in more detail, because QUIC not only assumes the responsibilities of the transport layer protocol, but also has the security-related capabilities of TLS, the following figure can be used to understand the position of QUIC in the implementation of HTTP/3.

Next, let's analyze the QUIC protocol separately. First, let's look at how it establishes a connection.

QUIC connection

We know that TCP, a reliable transmission protocol, requires a three-way handshake. It is precisely because of the three-way handshake that an additional 1.5 RTT is consumed. If TLS is added, 3-4 RTT connections will be consumed.

So, how does QUIC establish a connection? How to reduce RTT?

QUIC proposes a new connection establishment mechanism. Based on this connection mechanism, a fast handshake function is implemented. A QUIC connection establishment can use 0-RTT or 1-RTT to establish a connection.

• QUIC uses the Diffie-Hellman algorithm during the handshake process to ensure the security of data interaction and combines its encryption and handshake processes to reduce the number of round trips during the connection establishment process.
• Diffie–Hellman (DH) key exchange is a special method for exchanging keys. It is one of the earliest key exchange methods put into practice in the field of cryptography. DH allows two parties to reach a shared key through an insecure channel without any (private) information about the other party. This key is used to symmetric encrypt subsequent information exchanges.
• The overall process of establishing a QUIC connection is as follows: QUIC uses the Diffie-Hellman algorithm to negotiate the initial key during the handshake process. The initial key depends on a set of configuration parameters stored by the server, which are updated periodically. After the initial key negotiation is successful, the server will provide a temporary random number, and both parties will regenerate the session key based on this number. The client and server will use the newly generated key to encrypt and decrypt data.

The above process is mainly divided into two steps: Initial handshake and Final (and repeat) handshake. The following introduces these two processes respectively.

Initial handshake

When the connection is established, the client sends a hello message to the server (inchoate client hello (CHLO)). Because it is the first time to establish a connection, the server returns a rejection message (REJ), indicating that the handshake is not established or the key has expired.

However, this rejection message will also contain more information (configuration parameters), mainly:

• Server Config: A server configuration, including the server-side Diffie-Hellman algorithm long-term public key (long term Diffie-Hellman public value)
• Certificate Chain: The trust chain used to authenticate the server
• Signature of the Server Config: The signature of the Server Config encrypted with the public key of the leaf certificate of the trust chain
• Source-Address Token: An authenticated, encrypted block containing the client’s publicly visible IP address and the server’s timestamp.

After the client receives the rejection message (REJ), it will perform data parsing, signature verification and other operations, and then cache the necessary configurations.

At the same time, after receiving the REJ, the client will randomly generate a pair of its own short-term key (ephemeral Diffie-Hellman private value) and short-term public key (ephemeral Diffie-Hellman public value) for this connection.

After that, the client will package the short-term public key it just generated into a Complete CHLO message packet and send it to the server. The purpose of this request is to transmit its own short-term key to the server to facilitate forward secrecy, which will be described in detail later.

After sending the Complete CHLO message to the server, in order to reduce RTT, the client will not wait for the server's response, but will immediately transmit data.

To ensure data security, the client will calculate its own short-term key and the long-term public key returned by the server to obtain an initial key.

With this initial key, the client can use this key to encrypt the information it wants to transmit, and then transmit it securely to the server.

On the other hand, after receiving the Complete CHLO request, the server will have both the client's short-term public key and its own long-term key after parsing the request. In this way, through calculation, the server can obtain an initial key that is exactly the same as the client's.

Next, after receiving the data encrypted by the client using the initial key, he can use this initial key to decrypt it, and can encrypt his own response with this initial key and return it to the client.

Therefore, from the beginning of the connection establishment to the data transmission, only 1 RTT of the initial connection establishment is consumed.

Final (and repeated) handshake

So, can subsequent data transmission be encrypted using the initial keys?

In fact, it is not entirely true, because the initial key is generated based on the server's long-term public key after all, and before the public key expires, almost all connections use the same public key, so this actually has certain risks.

Therefore, in order to achieve the security of forward secrecy, the client and server need to use each other's short-term public key and their own short-term secret key to perform operations.

In cryptography, forward secrecy (FS) is a security property of a communication protocol in cryptography, which means that the leakage of a long-term master key will not lead to the leakage of past session keys.

So now the problem is that the client's short-term key has been sent to the server, but the server only gave its long-term key to the client, and did not give its own short-term key.

Therefore, after receiving the Complete CHLO, the server will send a server hello (SHLO) message to the server, which will be encrypted using the initial keys.

This CHLO message package will contain a short-term public key regenerated by the server.

In this way, both the client and the server have each other's short-term public key (ephemeral Diffie-Hellman public value).

In this way, both the client and the server can perform operations based on their own short-term keys and the other party's short-term public key to generate a forward-secure key that is only used for this connection. Subsequent requests can be encrypted and decrypted based on this key.

In this way, the two parties complete the final key exchange, connection handshake and establish a QUIC connection.

When the connection is to be re-established next time, the client will take out the long-term public key of the server that it has previously cached from the cache, recreate a short-term key, regenerate an initial key, and then use this initial key to encrypt the data to be transmitted, and send a Complete CHLO request to the server. In this way, 0 RTT data transmission is achieved.

Therefore, if there is a cached long-term public key, the data transfer will be carried out directly, and the preparation time is 0 RTT

In the above, by using the Diffie-Hellman algorithm to negotiate keys and merging the encryption and handshake processes, the RTT of the connection process is greatly reduced, so that the QUIC-based connection establishment can be reduced to 1 RTT or even 0 RTT.

Below is a flowchart on Google's official website about QUIC connection establishment, which can help you understand this process.

In addition, through the above process of establishing a handshake, we can also know that QUIC ensures security well through encryption and decryption throughout the process.

Multiplexing

One of the biggest problems with HTTP implemented based on TCP is head-of-line blocking. So, how does QUIC solve this problem?

During TCP transmission, data is split into sequentially arranged data packets, which are then transmitted to the receiving end through the network. The receiving end then combines these data packets into original data in sequence, thus completing the data transmission.

However, if one of the packets does not arrive in order, the receiver will keep the connection open and wait for the packet to return, which will block subsequent requests. This is TCP head-of-line blocking.

Similar to HTTP/2, QUIC can have multiple independent logical data streams on the same physical connection. These data streams are transmitted in parallel on the same connection, and there is no timing requirement for the transmission between multiple data streams, nor will they affect each other.

Streams provide a lightweight, ordered byte stream abstraction in QUIC.

QUIC's single data stream can guarantee ordered delivery, but multiple data streams may be out of order. This means that the transmission of a single data stream is in order, but the order received by the receiver in multiple data streams may be different from the order sent by the sender!

That is to say, there is no dependency between multiple data streams on the same connection (they are not required to arrive in order). Even if a data packet fails to arrive, it will only affect its own data stream and will not affect other data streams.

Connection Migration

The identification of TCP connection requires the server and client to pass through the four parameters of IP and port. In the scenario of network switching, such as the mobile phone switching network, its own IP will change. This will cause the previous TCP connection to become invalid and need to be re-established.

This scenario is still quite frequent today when mobile devices are popular.

So, at this point, QUIC is optimized.

The QUIC protocol uses a unique UUID to mark each connection. When the network environment changes, as long as the UUID remains unchanged, data transmission can continue without the need for a handshake.

reliability

The reason why TCP is called a reliable link is not only because it has three handshakes and four closing processes, but also because it makes many reliability guarantees such as flow control, data retransmission, and congestion control.

This is why TCP has always been used as an important protocol for HTTP implementation.

So, if QUIC wants to replace TCP, it needs to make efforts in this regard as well. After all, UDP itself does not have these capabilities.

TCP congestion control is an algorithm used by TCP to avoid network congestion and is a major congestion control measure on the Internet. There are many classic algorithm implementations, such as TCP Tahoe and Reno, TCP Vegas, TCP Hybla, TCP New Reno, TCP Westwood and Westwood+, TCP BIC and CUBIC, etc.

The QUIC protocol also implements congestion control. It does not rely on a specific congestion control algorithm and provides a pluggable interface to allow users to experiment. The Cubic congestion control algorithm of the TCP protocol is used by default.

Regarding flow control, QUIC provides two levels of flow control: stream-based and connection-based. It requires both control of a single stream and overall control of all streams.

QUIC's connection-level flow control is used to limit the total buffer that the QUIC receiver is willing to allocate to the connection, preventing the server from allocating an arbitrarily large buffer to a client. The process of connection-level flow control is basically the same as that of stream-level flow control, but the offset limit for forwarding data and receiving data is the sum of all streams.

Disadvantages

Above, we have introduced many advantages of QUIC compared to TCP. It can be said that this protocol is indeed better than TCP.

Because it is based on UDP, the UDP protocol itself has not been changed, but only some enhancements have been made. Although it can avoid the problem of rigidity of intermediate devices, it is not completely without problems in promotion.

First, many enterprises, operators, and organizations will intercept or limit UDP traffic other than port 53 (DNS) because this traffic has been frequently abused for attacks recently.

In particular, some existing UDP protocols and implementations are vulnerable to amplification attacks, where attackers can control innocent hosts to send large amounts of traffic to victims.

Therefore, the transmission of the QUIC protocol based on UDP may be blocked.

In addition, because UDP has always been positioned as an unreliable connection, many intermediate devices do not support and optimize it to a high degree, so the possibility of packet loss is still relatively high.

Summarize

The following table summarizes the similarities and differences between HTTP/2 and HTTP/3. Some of them have been introduced in this article, and some of them are not mentioned in this article because I think they are not particularly important. If you are interested, you can study them on your own.