Why don't we use HTTP directly for calls between services but use RPC?

  [[313073]]

What is RPC?

RPC (Remote Procedure Call) is a protocol that requests services from a remote computer program over the network without having to understand the underlying network technology. For example, if two different services A and B are deployed on two different machines, what should service A do if it wants to call a method in service B? Using HTTP requests is certainly possible, but it may be slow and some optimizations are not good. The emergence of RPC is to solve this problem.

What is the RPC principle?

1. The service consumer (client) calls the service in a local call mode;
2. After receiving the call, the client stub is responsible for assembling the method, parameters, etc. into a message body that can be transmitted over the network;
3. The client stub finds the service address and sends the message to the server;
4. The server stub decodes the message after receiving it;
5. The server stub calls the local service based on the decoding result;
6. The local service executes and returns the result to the server stub;
7. The server stub packages the returned results into a message and sends it to the consumer;
8. The client stub receives the message and decodes it;
9. The service consumer gets the final result.

Here is another online timing diagram:

What problem does RPC solve?

From the above introduction to RPC, in general, RPC mainly solves the problem of making calls between different services in a distributed or microservice system as simple as local calls.

Summary of common RPC frameworks?

• RMI (built-in JDK): The RPC built-in to JDK has many limitations and is not recommended.
• Dubbo: Dubbo is an open-source, high-performance service framework developed by Alibaba. It enables applications to implement service output and input functions through high-performance RPC and can be seamlessly integrated with the Spring framework. Currently, Dubbo has become an official component of Spring Cloud Alibaba.
• gRPC: gRPC is a modern, open source, high-performance RPC framework that can run in any environment. It can efficiently connect services within and across data centers with pluggable support for load balancing, tracing, health checks, and authentication. It is also suitable for the last mile of distributed computing to connect devices, mobile applications, and browsers to backend services.
• Hessian: Hessian is a lightweight remoting on http tool that provides RMI functionality using simple methods. Compared with WebService, Hessian is simpler and faster. It uses a binary RPC protocol, and because it uses a binary protocol, it is very suitable for sending binary data.
• Thrift: Apache Thrift is Facebook's open source cross-language RPC communication framework, which has been donated to the Apache Foundation for management. Due to its cross-language features and excellent performance, it is used in many Internet companies. Capable companies will even develop a distributed service framework based on thrift, adding functions such as service registration and service discovery.

Since we have HTTP, why use RPC for service calls?

RPC is just a design

RPC is just a concept and a design to solve the calling problem between different services. It generally includes two protocols: transmission protocol and serialization protocol.

The transport protocol for implementing RPC can be built directly on top of TCP or HTTP. Most RPC frameworks use TCP connections (gRPC uses HTTP2).

HTTP and TCP

Maybe many friends who are not familiar with computer networks have been confused. If you want to really understand it, you need to briefly review the basic knowledge of computer networks:

The five-layer protocol architecture of computer networks that we usually talk about is: application layer, transport layer, network layer, data link layer, and physical layer.

The task of the application layer is to complete specific network applications through the interaction between application processes. HTTP belongs to the application layer protocol, which transmits data (HTML files, image files, query results, etc.) based on the TCP/IP communication protocol. The HTTP protocol works on the client-server architecture. The browser, as an HTTP client, sends all requests to the HTTP server, i.e., the WEB server, through the URL. After receiving the request, the Web server sends a response message to the client. The HTTP protocol is built on the TCP protocol.

The main task of the transport layer is to provide general data transmission services for the communication between two host processes. TCP is a transport layer protocol that mainly solves how data is transmitted in the network. Compared with UDP, TCP provides a connection-oriented, reliable data transmission service.

The key lies in the difference in messages between the TCP protocol used by HTTP and our custom TCP protocol.

The TCP message of http1.1 protocol contains too much information that may be useless during transmission:

 HTTP/1.0 200 OK
 Content-Type: text/plain
 Content-Length: 137582
 Expires: Thu, 05 Dec 1997 16:00:00 GMT
 Last -Modified: Wed, 5 August 1996 15:55:28 GMT
 Server: Apache 0.84 
 
 <html>
 <body>Hello World</body>
 </html>

Using a custom TCP protocol for transmission will avoid the above problem and greatly reduce the overhead of transmitting data. This is the real reason why RPC with a custom TCP protocol is usually used to call services. In addition, mature RPC frameworks also provide "automatic service registration and discovery", "intelligent load balancing", "visual service governance and operation and maintenance", "runtime traffic scheduling" and other functions, which can also be regarded as one of the reasons for choosing RPC for service registration and discovery!

A common misconception

Many articles also mention that the HTTP protocol has an increased overhead in connection establishment and disconnection compared to the custom TCP message protocol, but this view has been denied. The following is an answer from a certain forum:

First of all, we must deny that the HTTP protocol has an increased overhead in connection establishment and disconnection compared to the custom TCP message protocol. The HTTP protocol supports connection pool reuse, which means that a certain number of connections are established without disconnection, and connections are not frequently created and destroyed. The second thing to say is that HTTP can also use Protobuf, a binary encoding protocol, to encode content, so the biggest difference between the two is still in the transmission protocol.

Off topic

In addition, it is also important to note that Spring Cloud Netflix does not use the RPC framework to call between different services, but uses the HTTP protocol for calling. Although the speed is not as fast as RPC, using the HTTP protocol will also bring many other benefits (you can refer to relevant information for yourself).