Don't understand the network I/O model? How to get started with Netty

Netty is a network application framework, so from the most fundamental point of view, it is an encapsulation of the network I/O model.

Therefore, to deeply understand Netty's high performance, we must also start with the network I/O model.

After reading this article, you can answer these three questions:

• What are the five I/O models? What are the core differences?
• Synchronous = blocking? Asynchronous = non-blocking?
• What I/O model does Netty use to achieve high performance?

1. Master the key to the five I/O models

You should have heard of the five basic I/O models under Unix systems, which are divided into:

• blocking I/O (synchronous blocking IO, BIO)
• Nonblocking I/O (NIO)
• I/O multiplexing
• signal driven I/O
• asynchronous I/O (asynchronous I/O, AIO)

In fact, many people cannot accurately distinguish the characteristics of each I/O, especially the differences between synchronous/asynchronous and blocking/non-blocking.

Therefore, we will first tell you the most important "key". With this "key", you can easily look at the key issues of the I/O model.

When a network IO occurs, three objects are mainly involved:

• The Process or Application that initiated this IO operation
• System kernel. User processes cannot directly operate I/O devices and must interact with I/O devices through the system kernel.
• I/O devices, including network, disk, etc. This article focuses on the network.

The actual I/O process is mainly divided into two stages :

• Waiting for the data preparation phase.
• Data copy phase. After the data is prepared, it is copied from the kernel to the process.

Take an input operation on a socket as an example.

The first step usually involves waiting for data to arrive from the network. When the packet you are waiting for arrives, it is copied to a buffer in the kernel.

The second step is to copy the data from the kernel buffer to the user-mode buffer.

Here, let us first remember these two stages, as they are the "key" to mastering the differences between all I/O models.

2. Detailed explanation of five I/O models

2.1 Synchronous blocking I/O, BIO

The most basic I/O model that we generally use is synchronous blocking I/O.

Typical applications:

Blocking sockets, Java BIO

Let's interpret the BIO process:

• The application process initiates an I/O request to the kernel, and the thread that initiates the call is blocked , waiting for the kernel to return the result.
• After the data is prepared, it is copied from the kernel to the user-mode memory ( still blocked ), and then the kernel returns the result, the user process ends the blocking and runs again.

"Key" analysis:

The characteristic of BIO is that it is blocked in both stages of IO execution.

Therefore, when we use the BIO model in daily life, the way to improve performance is to use multi-threading .

In general scenarios, BIO in a multi-threaded model is a low-cost, high-return method. However, if too many threads are created in a high-concurrency scenario, it will seriously occupy system resources and reduce the efficiency of the system's response to the outside world.

So can we consider using "thread pool" or "connection pool"?

To a certain extent, yes. The purpose of "pooling" is to reduce the frequency of creating and destroying threads, allowing idle threads to take on new execution tasks, maintaining a reasonable number of threads, and effectively reducing system overhead.

However, the "pooling" technology can only alleviate the resource usage caused by frequent calls to the IO interface to a certain extent. If the "pool" has an upper limit of 100 and we need 1000 IOs, it cannot solve the performance problem. This is due to the limitations of the BIO model itself.

Therefore, non-blocking I/O is needed to try to solve this problem.

2.2 Synchronous Non-Blocking I/O, NIO

To solve the BIO blocking problem, let's consider using the non-blocking NIO model.

Typical applications:

Non-blocking mode of socket

After the application process initiates an I/O request to the kernel, if the data in the kernel is not ready yet, it will no longer "block" waiting for the result, but will return immediately.

From the user process's perspective, after initiating an IO operation, it does not need to wait, but gets a result immediately.

When the user process determines that the result is an error, it knows that the data is not ready, so it starts to initiate a polling operation.

Once the data in the kernel is ready, once the user comes in again, the data will be copied to the user memory immediately and then returned.

Therefore, in non-blocking IO, the user process actually needs to constantly and actively ask the kernel whether the data is ready.

"Key" analysis:

The significant difference between the non-blocking NIO model and BIO is that in the "data waiting" phase, it no longer "blocks" and returns immediately.

But in the "data copy" stage, it is still "blocked".

Although the non-blocking model avoids blocking in the "data waiting" phase, the use of polling will result in a large system context switching overhead and significantly increase the CPU usage.

Therefore, the efficiency of using the non-blocking I/O model alone is not high. Moreover, as the concurrency increases, non-blocking I/O will have serious performance waste.

We can see that the purpose of polling is just to detect whether the data is ready, and the operating system provides a more efficient detection interface.

For example, select() multiplexing mode can detect whether multiple connections are active at once.

2.3 Multiplexed IO

Multiplexing enables one thread to handle multiple I/O handles. In some places, this IO method is also called event-driven IO.

Multipath refers to multiple data channels

Multiplexing refers to using one or more fixed threads to process each Socket.

Typical applications:

Select, poll, epoll three solutions

Java NIO

The IO of multiple processes can be registered to a multiplexer (selector), and then one process calls select, which will monitor all registered IO.

If all the IOs monitored by the selector have no readable data in the kernel buffer, the select calling process will be blocked; at the same time, the kernel will "monitor" all the sockets that the select is responsible for, and if the data in any socket is ready, the select will return;

Then the select calling process can initiate read IO again by itself or notify another process (registered process), and then the process copies the data from the kernel to the user process and reads the data prepared in the kernel.

It can be seen that after multiple processes register IO, only one select calling process is blocked .

Multiplexing solves the problem of synchronous blocking I/O and synchronous non-blocking I/O, and is a very efficient I/O model. We can intuitively see that the advantage of this model is that a single process can handle the IO of multiple network connections at the same time.

"Key" analysis:

In multiplexed I/O, the select phase is "non-blocking" for the "data waiting" phase of multiple sockets.

The "data copy" phase of a single socket is also "blocking".

Special attention needs to be paid here!!!

In fact, if the number of IOs processed is small, a web server using multiplexed IO may not necessarily perform better than a web server using pooling + BIO, and may even have greater latency.

Consider the extreme case where there is only one IO, multiplexing requires 2 system calls (select + recvfrom), while BIO only requires 1 system call (recvfrom).

Therefore, the advantage of multiplexed IO is not that it can process a single connection faster, but that it can handle more connections.

2.4 Signal-driven I/O

When using signal-driven I/O, when the data is ready, the kernel notifies the application process by sending a SIGIO signal, and the application process can start reading the data.

The biggest feature of the signal-driven I/O model is that there is no need for the process to constantly poll the kernel to see if it is ready.

"Key" analysis:

Signal-driven I/O is "non-blocking" in the "data waiting" phase.

When the data is ready, the signal notifies the process and the process starts the "data copy" phase, which is still "blocking".

Signal-driven I/O has several drawbacks:

1) When there are a large number of IO operations, the signal queue may overflow and notification may not be possible.

2) Signal-driven I/O Although useful for dealing with UDP sockets, signal notification means the arrival of a datagram, or the return of an asynchronous error.

However, for TCP, the signal-driven I/O method is not very useful because there are many situations that lead to signal notifications, and it would consume a lot of resources to judge each one.

Therefore, the signal-driven I/O mode is rarely used.

And it is especially important to note that it is still "blocked" during the "data copy" phase.

2.5 Asynchronous I/O, AIO

True asynchronous I/O is AIO.

Typical applications:

JAVA7 AIO, high performance server

Based on the analysis of the previous four models, I believe everyone can clearly understand how this model works.

After the user process initiates an I/O request, it can immediately start doing other things. On the other hand, from the kernel's perspective, when it receives a request, it will first return immediately, so it will not block the user process. Then, the kernel will wait for the data to be prepared, and then copy the data to the user memory. When all this is completed, the kernel will send a signal to the user process to tell it that the I/O operation is completed.

The most important point about AIO is that the process of copying data from the kernel buffer to the user-mode buffer is also completed asynchronously by the system , and the application process only needs to reference the data in the specified array.

The main differences between AIO and signal-driven I/O are:

Signal-driven I/O is notified by the kernel when an I/O operation can begin, while asynchronous I/O is notified by the kernel when the I/O operation has been completed.

"Key" analysis:

"Data waiting" phase, non-blocking

"Data copy" phase, non-blocking

AIO is a truly asynchronous model that does not block the request process.

3. Synchronous = blocking? Asynchronous = non-blocking?

In daily use, we often equate synchronous I/O with blocking I/O and asynchronous I/O with non-blocking I/O.

In fact, strictly speaking, there are still great differences between these two sets of concepts.

3.1 Blocking I/O and Non-Blocking I/O

The difference between blocking and non-blocking is obvious and easy to understand.

In terms of the I/O model, blocking I/O will block the corresponding process until the operation is completed, while non-blocking I/O will return immediately when the kernel is in the "waiting for data to be prepared" phase.

Therefore, we generally believe that the only blocking I/O is BIO, and the other four models are all non-blocking I/O.

3.2 Synchronous I/O and Asynchronous I/O

Let's first look at the definitions of synchronous I/O and asynchronous I/O. According to the POSIX definition:

• Synchronous I/O: A synchronous I/O operation causes the requesting process to be blocked until that I/O operation completes;
• Asynchronous I/O: An asynchronous I/O operation does not cause the requesting process to be blocked;

The difference between the two is that synchronous I/O will block the process when performing "IO operation".

According to this definition, we look at the "key" analysis part of each model above, and we can clearly see that the four models of BIO, NIO, IO multiplexing, and signal-driven IO all belong to synchronous IO.

Because they are "blocked" in the second stage of IO, the stage of actually executing "data copy". Take NIO as an example. When executing the recvfrom system call, if the kernel data is not ready, the process will not be blocked. But when the data in the kernel is ready, recvfrom will copy the data from the kernel to the user memory, and the process will be blocked at this time.

Similarly, signal-driven IO, when the IO data in the kernel is ready, the requesting process is notified by the SIGIO signal, and the requesting process then reads the data from the kernel into the user space, this step is also blocked.

Therefore, there is only one true asynchronous I/O, which is AIO. When a process initiates an IO operation, it returns and does not care about it again until the kernel sends a signal to tell the process that the IO is completed. During this whole process, the process is not blocked at all. As the definition says, it will not be blocked by IO operations.

4. Which I/O model does Netty use?

Netty's I/O model is implemented based on non-blocking I/O, and the underlying layer relies on the multiplexer Selector of the JDK NIO framework.

A multiplexer Selector can poll multiple Channels at the same time. After adopting the epoll mode, only one thread is needed to be responsible for the Selector polling, and thousands of clients can be connected.

We will explain more specific implementation methods and models in the next issue.

By the way, some students must want to ask, why doesn't Netty use AIO?

Because the purpose of AIO is to prevent the I/O thread from blocking the main thread, it is asynchronous I/O, and the kernel notifies when the I/O operation is completed. AIO is suitable for scenarios with a large number of connections and long-term connections.

For AIO, current operating system support is limited and implementation is complex.

Netty also tried AIO, but the results were not ideal and it was eventually abandoned.

bibliography:

"UNIX Network Programming(Volume1,3rd)"