What went wrong with HTTP/2? It’s about to be deprecated after just two years of success! ?

[[357747]]

There has been a lot of news about HTTP/3 in recent times, and more and more large international companies have begun to use HTTP/3.

Therefore, HTTP/3 is already on the way, and it is only a matter of time before it is fully used. So, as front-line developers, it is time for us to understand what HTTP/3 is and why we need it.

So, I prepared to start writing this article, but if I want to explain HTTP/3 clearly, the issue that I cannot avoid is HTTP/2. So as I wrote, the length became longer and longer, so I divided them into two parts.

In this article, we will mainly review HTTP/2, and then focus on the problems with HTTP/2 and why it was deprecated.

HTTP/2 is no longer a success?

Although the HTTP/2 standard was officially published as RFC 7540 in May 2015, and most browsers supported it by the end of 2015.

However, it was not until around 2018 that it was truly widely used. But it was also in November 2018 that the IETF gave official approval and recognized HTTP-over-QUIC as HTTP/3.

In 2018, I wrote an article introducing "What is HTTP/2?" At that time, HTTP/2 was still a new technology and had just begun to have software support. Just two years have passed, and now HTTP/3 has quietly arrived.

According to W3Techs data, as of June 2019, only 36.5% of websites in the world support HTTP/2. Therefore, many websites may not have started to support HTTP/2, but HTTP/3 has already arrived.

Therefore, for many websites, upgrading directly to HTTP/3 may be a more correct choice.

HTTP/2 Review

Before reading this article, it is strongly recommended that you read the article "What is HTTP/2?", which introduces the history of HTTP and the background of the birth of various versions of the HTTP protocol.

When you read this, I assume that you have a basic understanding of HTTP/2.

We know that the birth of HTTP/2 is mainly to solve the efficiency problem in HTTP/1.1. The core technology of HTTP/2 is multiplexing technology, which allows multiple request-response messages to be initiated simultaneously through a single HTTP/2.0 connection.

At the same time, technologies such as binary framing, header compression, and server push are also implemented.

From the birth of HTTP/1.0 to HTTP/2, the HTTP protocol has been upgraded three times in the past 24 years, but there is one key technical point that remains unchanged, that is, all these HTTP protocols are implemented based on the TCP protocol.

HTTP is ever-changing, but TCP is unchangeable. This is because TCP is more reliable than UDP.

Although the launch of HTTP/2 based on HTTP/1.1 has greatly improved efficiency, many people still believe that this is only a "temporary solution". This is why the industry began to invest heavily in the research and development and promotion of HTTP/3 shortly after its launch.

The deep-seated reason behind this is that it is still implemented based on the TCP protocol. Although the TCP protocol is more reliable, it still has certain problems. Let's analyze it in detail.

HTTP/2 Issues

Head of Line Blocking

Head-of-line blocking is translated from English head-of-line blocking. The term is not new, because the problem of head-of-line blocking has always existed as early as the HTTP/1.1 era.

However, many people have seen arguments in some materials that HTTP/2 solves the problem of head-of-line blocking. But this statement is only half right.

It can only be said that HTTP/2 solves the HTTP head-of-line blocking problem, but it does not solve the TCP head-of-line blocking problem!

If you have some knowledge of the history of HTTP, you will know that the most important improvement of HTTP/1.1 compared to HTTP/1.0 is the introduction of persistent connections (keep-alive).

The so-called persistent connection means that multiple HTTP requests and responses can be transmitted on one TCP connection, reducing the consumption and delay of establishing and closing connections.

After the introduction of persistent connections, the HTTP protocol has been significantly improved in terms of performance.

In addition, HTTP/1.1 allows the use of request pipelining on persistent connections, which is another performance optimization relative to persistent connections.

The so-called request pipeline means that multiple requests can be queued before the HTTP response arrives. When the first HTTP request flows through the network to the server, the second and third requests can also be sent. In high-latency network conditions, this can reduce the network round-trip time and improve performance.

However, there are still certain restrictions and requirements for pipeline connections. One of the more critical ones is that the server must send back HTTP responses in the same order as the requests.

This means that if a response is delayed, all subsequent responses will be delayed until the head of the line response arrives. This is known as HTTP head of line blocking.

However, the HTTP head-of-line blocking problem has been effectively solved in HTTP/2. HTTP/2 abandons the pipelining method and innovatively introduces the concepts of frames, messages, and data streams. Clients and servers can decompose HTTP messages into independent frames, send them out of order, and finally reassemble them at the other end.

Because there is no order, there is no need to block, which effectively solves the HTTP head-of-line blocking problem.

However, HTTP/2 still has the problem of TCP head-of-line blocking, because HTTP/2 still relies on the TCP protocol.

During TCP transmission, data is split into sequentially arranged data packets, which are then transmitted to the receiving end through the network. The receiving end then combines these data packets into original data in sequence, thus completing the data transmission.

However, if one of the packets does not arrive in order, the receiver will keep the connection open and wait for the packet to return, which will block subsequent requests. This is TCP head-of-line blocking.

The pipelined persistent connection of HTTP/1.1 also allows the same TCP link to be used by multiple HTTPs, but HTTP/1.1 stipulates that a domain name can have 6 TCP connections. In HTTP/2, the same domain name only uses one TCP connection.

Therefore, in HTTP/2, the impact of TCP head-of-line blocking will be greater, because HTTP/2's multiplexing technology makes multiple requests actually based on the same TCP connection. If a request causes TCP head-of-line blocking, multiple requests will be affected.

TCP handshake duration

When mentioning the TCP protocol, the first thing that comes to mind is its three-way handshake and four-way close features.

Because TCP is a reliable communication protocol, and this reliability is achieved through the three-way handshake. Through the three-way handshake, TCP can ensure that the data received by the receiver is complete, orderly, and error-free during the transmission process.

However, the problem is that the three-way handshake takes time. Here is a concept about network latency.

Network latency is also known as RTT (Round Trip Time). It refers to the time it takes for a request to be sent from a client browser to a server and then to receive a response from the server. RTT is an important indicator of network performance.

We know that the TCP three-way handshake process requires three interactions between the client and the server, which means that 1.5 RTT is consumed.

In addition, if the secure HTTPS protocol is used, the TLS protocol must be used for secure data transmission, which consumes another RTT (different versions of TLS have different handshake mechanisms, so the minimum consumption is used here).

This means that a pure HTTP/2 connection consumes 1.5 RTTs, and an HTTPS connection consumes 3-4 RTTs.

The specific time consumed varies depending on the distance between the server and the client. If they are relatively close, the time consumed is within 100ms, which may not be noticeable to the user. However, if the RTT takes 300-400ms, then the total time consumed for a connection establishment process may be about one second. At this time, the user will obviously feel that the web page is loading very slowly.

Is it feasible to upgrade TCP?

Based on the above mentioned problems, many people have proposed: Since TCP has these problems, and we know that these problems exist, and even the solutions are not difficult to think of, why can't we upgrade the protocol itself to solve these problems?

In fact, this involves the issue of "protocol rigidity".

In this way, when we browse data on the Internet, the data transmission process is actually extremely complicated.

As we know, there are several prerequisites for using the Internet at home. First, we need to open the Internet through the operator and use a router, which is an intermediate device in the network transmission process.

Intermediate equipment refers to auxiliary equipment inserted between the data terminal and the signal conversion equipment to complete some additional functions before modulation or after demodulation. For example, hubs, switches and wireless access points, routers, security modems, communication servers, etc. are all intermediate equipment.

There are many such intermediate devices in places we cannot see. A network needs to be forwarded by countless intermediate devices before it can reach the end user.

If the TCP protocol needs to be upgraded, it means that these intermediate devices must be able to support new features. We know that we can replace the router, but what about other intermediate devices? Especially those larger devices? The cost of replacement is huge.

Moreover, in addition to the intermediate devices, the operating system is also an important factor, because the TCP protocol needs to be implemented through the operating system kernel, and the update of the operating system is also very lagging.

Therefore, this problem is called "intermediate device rigidity", which is also an important reason for "protocol rigidity". This is also an important reason that restricts the update of TCP protocol.

Therefore, many new TCP features standardized by the IETF in recent years have not been widely deployed or used due to lack of broad support!

Abandon TCP?

The root cause of the above-mentioned problems is that HTTP/2 is implemented based on TPC, and upgrading the TCP protocol itself is difficult to achieve.

Then, there is only one solution left, which is to abandon the TCP protocol.

If we abandon TCP, we will have two new options: use other existing protocols or create a new protocol?

Seeing this, smart readers must have thought that creating new protocols will also be affected by the rigidity of intermediate devices. In recent years, efforts to create new transport layer protocols have basically failed because of the great difficulties encountered in deploying them on the Internet!

Therefore, if you want to upgrade the new HTTP protocol, there is only one way to go, that is to make some modifications and support based on the existing protocol. UDP is an excellent choice.

Summarize

Because HTTP/2 is implemented using the TCP protocol at the bottom layer, although it solves the HTTP head-of-line blocking problem, it is powerless to solve the TCP head-of-line blocking problem.

During TCP transmission, data is split into sequentially arranged data packets, which are then transmitted to the receiving end through the network. The receiving end then combines these data packets into original data in sequence, thus completing the data transmission.

However, if one of the packets does not arrive in order, the receiver will keep the connection open and wait for the packet to return, which will block subsequent requests. This is TCP head-of-line blocking.

In addition, TCP is a reliable transmission that is achieved through a three-way handshake. During the TCP three-way handshake, the client and the server need to interact three times, which means that 1.5 RTT is consumed. If it is HTTPS, the RTT consumed is even higher.

However, because many intermediate devices are relatively old and the cost of upgrading them is huge, it is basically impossible to upgrade the TCP protocol or adopt new protocols.

Therefore, HTTP/3 chose a new technical solution, which is to make modifications based on UDP. This technology is called QUIC.

So the question is, how does HTTP/3 use UDP? What changes have been made? How to ensure the reliability of the connection? Is there no problem of rigidity in the UDP protocol?

We will analyze these issues in depth in the next article. Stay tuned!

References: https://http3-explained.haxx.se/https://baike.baidu.com/item/Middle Equipment/3688874https://time.geekbang.org/column/article/150159https://juejin.cn/post/6844903853985366023https://time.geekbang.org/column/article/279164