In-depth interpretation of the principles and applications of HTTP/3

HTTP3 is the latest version of the HTTP protocol. Since its inception, HTTP has been the preferred application layer protocol for exchanging hypertext documents. Over the years, HTTP has undergone several major upgrades to keep up with the development of the Internet and the increase in the variety of content exchanged on the WWW.

This article will explore HTTP/3 in depth, introduce the evolution of the HTTP protocol, focus on the characteristics of HTTP/3, and provide a new perspective on the changes that HTTP/3 will bring to the Internet.

background

At the time of the creation of the World Wide Web, it was just a group of computers exchanging hypertext files. Exchanging files between computers was a simple procedure, involving requests and responses. Based on this, a simple text-based protocol was designed. HTTP (Hypertext Transfer Protocol) came into being. Later, it was drafted into a standardized IETF protocol, defined in RFC 1945, also known as HTTP/1.0.

Over the years, HTTP has evolved from HTTP/1.0 to HTTP/1.1 and then to HTTP/2. In each iteration, new features have been added to the protocol to handle a multitude of requirements, such as application layer requirements, security considerations, session handling, and media types. For a deeper look at HTTP/2 and its evolution from HTTP/1.0, see our HTTP/2 concepts article.

Despite several revisions, HTTP's underlying transport mechanism has remained largely unchanged. However, as Internet traffic has exploded, fueled by mobile phones, HTTP's transport mechanism has become problematic in ensuring a smooth web browsing experience.

HTTP/3 was created to handle the transport-related issues of HTTP/2.0, allowing faster access to the Web on a variety of devices. It is based on a new transport layer protocol called QUIC (Quick UDP Internet Protocol), which works on top of UDP. This choice is very different from previous versions of HTTP, which were all based on TCP. TCP is a more reliable protocol than UDP, so why redesign the transport layer of HTTP on top of UDP?

Let’s look at the limitations of running HTTP over TCP and take a closer look at the design ideas behind HTTP/3, which is based on the QUIC protocol.

What is HTTP/3

While the IETF was formally standardizing HTTP/2, Google was independently building a new transport protocol called gQUIC. It later became a new Internet Draft and was named QUIC. Initial experiments with gQUIC proved that gQUIC worked very well in enhancing the web browsing experience under poor network conditions. As a result, gQUIC gained momentum, and the majority of IETF members favored building a new HTTP specification that runs on QUIC. This new initiative was called HTTP/3 to distinguish it from the current HTTP/2 standard.

HTTP/3 is similar to HTTP/2 in terms of syntax and semantics. HTTP/3 follows the same order of request and response message exchange, and its data format includes method, header, status code, and body. However, the significant deviation of HTTP/3 is the order in which the protocol layers are stacked on top of UDP.

How does HTTP/3 work?

The core of HTTP/3 functionality is implemented around the underlying QUIC protocol. Before discussing QUIC and UDP, it is necessary to first list some of the limitations of TCP, which is also the reason for the development of QUIC.

TCP may intermittently hang on data transfer

If a segment with a lower sequence number has not been received, TCP's receiver sliding window will not proceed even if other segments with higher sequence numbers have been received. This will cause the TCP stream to hang momentarily, or in worse cases, close the connection even if one of the segments has not been received. This problem is known as Head-of-Line Blocking (HoL) for TCP streams.

TCP does not support stream-level multiplexing

While TCP does allow multiple logical connections between application layers, it does not allow multiplexing of packets within a single TCP stream. With HTTP/2, the browser can only open one TCP connection with the server and use the same connection to request multiple objects, such as CSS, JavaScript, and other files. While receiving these objects, TCP serializes all objects in the same stream. Therefore, it is not aware of object-level partitioning of TCP segments.

TCP will produce redundant communication

The TCP connection handshake has a redundant sequence of message exchanges, even for connections to known hosts.

The QUIC protocol addresses these issues by introducing some changes to the underlying transport mechanism based on the following design choices.

1. Choose UDP as the underlying transport layer protocol. Building a new transport mechanism on top of TCP will inherit all of the above shortcomings of TCP. Therefore, UDP is a wise choice. In addition, QUIC is built at the user level, so kernel modifications are not required every time the protocol is upgraded.

Stream multiplexing and flow control. QUIC introduces the concept of multiplexing multiple streams on a connection. QUIC implements separate flow control for each stream by design, solving the head-of-line blocking problem of the entire connection.

Flexible congestion control mechanism. TCP's congestion control mechanism is rigid. Every time the protocol detects congestion, it reduces the congestion window size by half. In contrast, QUIC's congestion control is designed to be more flexible and can more efficiently utilize available network bandwidth, resulting in better throughput.

Better error handling. QUIC uses enhanced loss recovery mechanisms and forward error correction to better handle erroneous packets. This feature is a boon for users who can only access the Internet through slow wireless networks, as these network users often experience high error rates during transmission.

Faster handshake. QUIC uses the same TLS module for secure connections. However, unlike TCP, QUIC's handshake mechanism is optimized to avoid redundant protocol exchanges every time communication is established between two known peers.

By building an HTTP/3-based application layer on top of QUIC, you can get all the benefits of the enhanced transport mechanism while retaining the syntax and semantics of HTTP/2. However, you must also note that HTTP/2 cannot be directly integrated with QUIC because the underlying frame mapping from application to transport is incompatible. Therefore, the HTTP working group of IETF proposed HTTP/3 as a new HTTP version and modified the frame mapping according to the frame format requirements of the QUIC protocol.

In addition to this, HTTP/3 uses a new HTTP header compression mechanism called QPACK, which is an enhancement of HPACK used in HTTP/2. Under QPACK, HTTP headers can arrive out of order in different QUIC streams. Unlike TCP in HTTP/2, which ensures the in-order delivery of packets, QUIC streams are delivered out of order and may contain different HTTP headers in different streams. Therefore, QPACK uses a lookup table mechanism to encode and decode headers.

Why is HTTP/3 important?

TCP is over 40 years old. It was standardized in 1981 through RFC 793. It has undergone many updates over the years and is a very robust transport protocol that can support the growth of Internet traffic. However, due to its design, TCP was never well suited to handle data transmission in lossy wireless environments. In the early days of the Internet, wired networks connected every computer in the network.

Now, with the number of smartphones and portable devices outnumbering the number of desktops and laptops, more than 50% of Internet traffic is already transmitted wirelessly. This trend has created problems for the overall web browsing experience, the most important of which is head-of-line blocking in TCP when wireless coverage is insufficient.

Some preliminary experiments by Google have proven that QUIC, as the underlying transport protocol for some of Google's popular services, has greatly improved speed and user experience. Deploying QUIC as the underlying transport protocol for YouTube videos has resulted in a 30% drop in the buffering rate of YouTube video streams, which has a direct impact on the user's video viewing experience. Similar improvements have also been seen when displaying Google search results.

The improvement is very obvious under poor network conditions, which prompted Google to be more active in improving the protocol and eventually propose standardization to IETF.

Thanks to all the improvements that these early experiments have brought, QUIC has become an important factor in taking the World Wide Web into the future. The revamp of HTTP from HTTP/2 to HTTP/3, powered by QUIC, is a logical step in this direction.

Best Use Cases for HTTP/3

HTTP/3 will improve our Internet experience, especially in areas where high-speed wireless networks are still not available. Although HTTP/2 has solved some of the problems, HTTP/3 goes a step further.

Internet of Things (IoT)

HTTP may not be the preferred protocol for the Internet of Things, but in some cases, HTTP-based communication is a good fit for a specific application. HTTP/3 can solve the problem of wireless connection loss for mobile phones that collect data from sensors. The same problem applies to standalone IoT devices installed in vehicles or movable assets. Accessing these devices through HTTP can be more reliable.

Big Data

Enterprises around the world are waking up to the potential of collecting data from multiple departments and combining it into larger information sharing APIs for internal and external audiences. These APIs also pave the way for monetization of data, which can be achieved by hosting this data as streaming API services. Over time, these services will spit out massive amounts of data. Streaming APIs hosted over HTTP/3 will make them more robust and resilient than HTTP/2.

Web VR

The content landscape is changing rapidly as browsers become more powerful. One area where this is happening is web-based VR. While still in its infancy, there are many use cases where VR can play a key role in enhancing collaboration. The web plays a central role in facilitating VR interactions. VR applications require more bandwidth to render the intricate details in virtual scenes, so migrating to HTTP/3 will be a big win.

Adopting HTTP/3: Considerations and Limitations

Transitioning to HTTP/3 involves changes not only in the application layer, but also in the underlying transport layer. As a result, HTTP/3 adoption is more challenging than its predecessor, HTTP/2, which only required changes in the application layer. The transport layer is subject to a lot of scrutiny from the middle layers in the network. These middle layers, such as firewalls, proxies, NAT devices, etc., perform a lot of deep packet inspection to meet their functional requirements. Therefore, the introduction of a new transport mechanism has some impact on IT infrastructure and operations teams.

However, another problem with the widespread adoption of HTTP/3 is that it is based on QUIC, which runs on UDP. Most web traffic, as well as well-known services defined by the IETF, run on top of TCP. This is also why long-running HTTP/3 UDP sessions are affected by the default packet filtering policy of the firewall.

These issues will eventually be resolved with the ongoing standardization work at the IETF. Furthermore, given the positive results shown by Google's early QUIC experiments, support for HTTP/3 is overwhelming, which will eventually force mid-tier vendors to standardize.

HTTP/3 is too cumbersome to be adopted for constrained IoT devices. Many IoT applications are deployed on devices with very small form factors. As a result, they have limited RAM and CPU power. This requirement is necessary for devices to operate efficiently under constraints such as battery power, low bitrates, and lossy connections. HTTP/3 is handled at the transport layer in the form of QUIC on top of the existing UDP, increasing the footprint of HTTP/3 in the entire protocol stack. This makes HTTP/3 bulky and unsuitable for those IoT devices. However, such cases are rare and specialized protocols exist, which avoids the need to support HTTP directly on such devices. In addition, there are IoT-centric protocols such as MQTT.

Getting started with HTTP/3

The HTTP working group of the IETF is working on releasing HTTP/3 in late 2020. Therefore it is not yet officially supported by mainstream web servers such as NGINX and Apache. However, there are several libs that can be used to experiment with this new protocol, and unofficial patches are also provided.

The following is a list of libraries that support HTTP/3 and QUIC transports. Please note that these implementations are based on a version of the Internet standard draft, which is likely to be superseded by a higher version, and the final standard will be published in an RFC.

Quiche (https://github.com/cloudflare/quiche)

Quiche provides a low-level programming interface for sending and receiving packets over the QUIC protocol. It also supports HTTP/3 modules to send HTTP packets over its QUIC protocol implementation. In addition to this, it also provides an unofficial patch for NGINX servers to install and host a web server capable of running HTTP/3. In addition to this, additional programs are provided to support the use of HTTP/3 on Android and iOS mobile applications.

Aioquic (https://github.com/aiortc/aioquic)

Aioquic is a Python implementation of QUIC. It also has a built-in test server and client library for HTTP/3. Aioquic is built on top of the asyncio module, which is Python's standard asynchronous I/O framework.

Neqo (https:/github.com/mozilla/neqo)

Neqo is Mozilla's implementation of QUIC and HTTP/3 in Rust.

If you want to try QUIC, check out this link to the open source implementation of the QUIC protocol maintained by the QUIC Working Group. https://github.com/quicwg/base-drafts/wiki/Implementations

Original link: https://www.ably.io/concepts/http3?utm_medium=referral&utm_source=reddit&utm_campaign=evergreen#limitations

This article is reprinted from the WeChat public account "High Availability Architecture". You can follow it through the following QR code. To reprint this article, please contact the High Availability Architecture public account.