API Gateway Selection: I use OpenResty!

Today I want to talk to you about the topic of gateway selection. Why did I finally choose OpenResty?

[[436285]]

Image from Baotu.com

Internet companies, regardless of their size, have similar internal technology architectures, as reflected in the following aspects:

• How to customize storage when the amount of data is too large
• How to deploy clusters and balance traffic load when traffic is high?
• The response speed is slow. How to improve the processing speed and introduce multi-level cache
• If there are many machines, how can we ensure that a sudden outage on a server does not affect the service stability of the business cluster?

Although small, it has all the necessary functions. The gateway we are going to talk about today is a key part of it. Regardless of the size of the company, it basically has this system. So what is the gateway used for?

What is a Gateway

The gateway is the intermediate bridge connecting the client and the server. It extracts a lot of common, non-business logic and pre-positions it to the gateway system, reducing a lot of repetitive development work. It is the only traffic entrance for the entire website.

In order to improve the scalability of the system, the gateway usually adopts a component-based architecture with high cohesion and low coupling.

Common component functions:

• Blacklist interception
• log
• Parameter verification
• Authentication
• Current Limitation
• Load Balancing
• Routing and forwarding
• monitor
• Grayscale splitting
• Multi-protocol support
• Circuit breaking, downgrade, retry, data aggregation, etc.

The system design generally adopts the responsibility chain design pattern, defines the abstract interface, and each component implements its own exclusive functions with a single responsibility.

And according to different business request APIs, some nodes are added or deleted, and new node chains are dynamically built to meet diverse business needs.

Gateway Selection

The most common gateways are as follows:

• Tomcat/Jetty+NIO+Servlet3
• Nginx
• Spring WebFlux
• Soul
• Spring cloud Gateway
• Zuul
• OpenResty
• Kong
• Netty self-built

The information of each framework is basically available on the Internet. Considering the space limitation, the advantages and disadvantages of each framework will not be analyzed here.

We plan to choose OpenResty for the following reasons:

• It can realize cross-network gRPC request forwarding, and the underlying layer uses HTTP/2 protocol.
• Supports SSL/TLS certificate encryption, ensuring secure communications.
• In terms of performance, it supports higher concurrent requests.
• Low performance overhead and less latency.

Flip OpenResty

The following figure is the latest statistics of Netcraft's Web server rankings in October 2021:

Netcraft is an Internet service company headquartered in Bath, UK, founded in 1995. The company's official website publishes monthly survey data reports: Web Server Survey series, which has become a way for people to understand the server market share of global websites.

OpenResty is now ranked third in the world and is gaining momentum due to its expansion of Nginx into a dynamic server.

The ticket query function of 12306 and the product details page of JD.com, which we often use, are all provided by OpenResty.

OpenResty is best at being deployed at the traffic entrance and handling various high-concurrency traffic. Next, let's take a closer look at this framework.

The origin of OpenResty

Nginx adopts a master-worker process model with clear division of labor and single responsibility, which is also one of the reasons for its high performance.

①Master process

Management process, processing instructions such as: -s reload, -s stop, through inter-process communication, send management instructions to the worker process, thereby achieving control over the worker process.

②Worker process

Worker processes continuously receive client connection requests and process them. The number is usually set to the same as the number of CPU cores. Nginx also binds each process to each CPU to make full use of its multi-core features.

Multiple worker processes will compete for a shared lock, and only the process that grabs the lock can process the client's request.

If the request is an accept event, it is added to the accept queue; if it is a read or write event, it is added to the read-write queue.

Some people may ask, why is OpenResty based on the Nginx framework?

[[436287]]

The main reason is the high concurrency capability of Nginx. Reverse proxy servers usually have large traffic volumes and do not involve complex calculations themselves. They are I/O intensive services.

Nginx uses event-driven based on the epoll mechanism, which is asynchronous and non-blocking, greatly improving the concurrent processing capability.

However, Nginx is developed in C language, and the threshold for secondary development is relatively high. It is widely used in the market, mostly based on the reserved configuration parameters of nginx.conf, such as reverse proxy, load balancing, static web server, etc.

If you want Nginx to access MySQL and customize the development of some business logic, it is very difficult.

OpenResty combines Nginx and Lua scripts through grafting, which not only retains the high concurrency advantage of Nginx, but also has the development efficiency of scripting language, and greatly reduces the development threshold.

Lua is the fastest, dynamic scripting language, running at a speed close to that of C. LuaJIT precompiles and caches some commonly used Lua functions and tool libraries, and directly uses the cached bytecode the next time it is called, which is very fast.

In addition, Lua supports coroutines, which is very important. Coroutines are user-mode operations, and context switching does not involve kernel mode, so system resource overhead is small; in addition, coroutines occupy very little memory, initially 2KB.

OpenResty Core Architecture

OpenResty is a web platform based on Nginx, which embeds LuaJIT virtual machine to run Lua scripts. Lua programming language is used to script Nginx core and various Nginx C modules.

As shown above:

• Each time a client request is received, a worker process follows up by grabbing the lock.
• A Lua coroutine will be created inside the worker and bound to the request, that is, one request corresponds to one Lua coroutine.
• The lua coroutine sends the request through the network and adds an event to nginx. Then, the current coroutine is in yield, giving up CPU control.
• When the server responds with data, the network process will create a new event, wake up the previous coroutine, and return the result.

Note: Data isolation between different Lua coroutines ensures that different client requests will not affect each other. In addition, only one coroutine can be running at the same time in a worker.

cosocket combines two important features: Lua coroutine + Nginx event notification. cosocket is the most valuable part of OpenResty in terms of technology and practicality.

It allows us to do network programming at a very low cost and in an elegant way, which is several times more efficient than traditional socket programming. It is excellent in terms of resource usage, execution efficiency, and concurrency.

To facilitate development, OpenResty divides an HTTP request into 11 stages, each with its own dedicated responsibilities.

Function description:

• set_by_lua, used to set variables.
• rewrite_by_lua, used for forwarding, redirection, etc.
• access_by_lua, used for access, permissions, etc.
• content_by_lua, used to generate return content.
• balancer_by_lua, load balancing, routing forwarding.
• header_filter_by_lua, used for response header filtering.
• body_filter_by_lua, used for response body filtering.
• log_by_lua, logging.

OpenResty provides a large number of Lua API interfaces for operating Nginx. As long as you are familiar with Lua syntax and have a clear understanding of the operation process of Nginx, you can easily do secondary development on Nginx.

Whether it is used as an application gateway or a high-performance web application, it supports connecting to a variety of rich backend storage, such as MySQL, Redis, Memcache, PostgreSQL, etc., and the surrounding ecology is very rich.

 https://github.com/openresty/lua-nginx-module/#accessbylua

Note: OpenResty's API has a limited usage range. Each API has a corresponding usage stage list. If you use it beyond the range, an error will be reported.

①Deployment and installation

This article takes the CentOS system as an example.

Add the openresty repository so that we can install or update our packages later using the yum update command:

 yum install yum-utils -y 
 
 yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo  

Install the software:

 yum install openresty -y

Install the command line tool resty:

 yum install openresty-resty -y

②Project Practice

Modify the nginx.conf configuration file:

 worker_processes auto;
 worker_rlimit_nofile 1000000;
 events {
 use epoll;
    worker_connections 150000;
 } 
 
 http {
 include mime.types;
    default_type application/octet-stream; 
 
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '  
 '$status $content_length $body_bytes_sent "$http_referer" '  
 '"$http_user_agent" "$http_x_forwarded_for" "$upstream_response_time" "$request_time"' ; 
 
    access_log logs/access.log main; 
 
 server {
 listen 8080;
 location / {
            access_by_lua_block {
 local headers = ngx.req.get_headers(0)
 local trace_id= headers[ "X-Trace-Id" ]
                ngx.log(ngx.ERR, trace_id)
 }
            # ngx.say( "<p>hello !</p>" )
            proxy_pass http://168.12.8.10:8080;
 }
 } 
 
 server {
 listen 8082;
 location / {
            default_type text/html;
            content_by_lua_block {
                ngx.say( "<p>Hello Openresty!</p>" )
 }
 }
 }
 }

nginx.conf is divided into three layers of nesting:

• The outermost layer, http, represents the processing of HTTP protocol.
• The server inside http listens to the port and starts a LuaJIT virtual machine to execute the Lua code.
• Within the same port, different business functions are distinguished, location configuration is adopted, and different business logics are processed through different paths.

Add environment variables:

 echo "export PATH=$PATH:/usr/local/openresty/nginx/sbin" >> /etc/profile 
 
 source /etc/profile

Start openresty. The startup command is the same as nginx:

 nginx -c /usr/ local /openresty/nginx/conf/nginx.conf

To access the Web service:

 curl http://localhost:8082/

If everything is normal, the browser page will output Hello Openresty!

If the nginx.conf configuration items are modified, we can restart:

 nginx -s reload

Author: Tom

Editor: Tao Jialong

Source: Reprinted from the official account Micro Technology (ID: weiguanjishu)