What is an API Gateway?

Hello everyone, I am ApeJava.

What is an API Gateway? What does it do? Why do we need it? Let's talk about it today.

I. Definition

From Wikipedia, a gateway is defined as follows:

In computer networks, a gateway is a server that forwards communication data from other servers. When receiving a request from a client, it processes the request as if it were the source server that owns the resources. Sometimes the client may not even realize that the target of its communication is a gateway.

As can be seen from the definition, the gateway is also a group of servers. It is located between the client and the server and is the only entry point for client requests to enter the server. As shown in the figure below, the API gateway provides several important functions:

• Authentication and security policy enforcement;
• Load balancing and circuit breaking;
• Protocol conversion and service discovery;
• Monitoring, logging, analysis and billing;
• cache;

2. Typical Process Analysis

Next, we will explain the API gateway and its important functions through the classic process of a client initiating an HTTP request to a server.

Step 1: The client sends a request to the API Gateway

The client initiates a request to the server, which is usually based on the HTTP protocol. It can be REST, GraphQL or some other higher-level abstraction. As shown below:

Step 2: API Gateway Verifies HTTP Request

After receiving the client's request, the API gateway will verify the parameters in the HTTP request, as shown below:

Step 3: IP blacklist and whitelist verification

For security reasons, you can set up an IP black and white list on the API gateway to mark the IP addresses that are allowed and not allowed to access the server. The API gateway allows and denies the caller's IP address based on the IP black and white list. See the figure below:

At the same time, the API Gateway can also perform basic rate limit checks on attributes such as IP addresses and HTTP headers. For example, it can reject requests from IP addresses that exceed a certain rate. As shown below:

Step 4: Authentication and Authorization

The API Gateway passes the HTTP request to the identity provider for authentication and authorization. The API Gateway receives an authenticated session from the provider, which contains the scope of operations that the request is allowed to perform. As shown below:

Authentication is the process of verifying the identity of a user or client to confirm that an entity is who it claims to be. In the context of an API, this means ensuring that the requester is a legitimate user or client and has the authority to access the requested resource or service.

Common authentication methods include:

• Basic Authentication: The client uses a Base64-encoded username and password in the request header for authentication. Although it is simple and easy to use, it is not the most secure authentication method because the credentials are transmitted in plain text in the request and can be easily intercepted and decoded.
• Token Authentication: The client uses a specific token in the request header for authentication. The token is usually issued to the client by the authenticated service, has a limited validity period, and is passed in each request. Compared with basic authentication, token authentication is more secure because the token usually does not contain sensitive information and can be transmitted over HTTPS encryption.
• OAuth authentication: OAuth is an open standard for authorization. It allows users to authorize third-party applications to access their resources stored on another service provider without providing their login credentials. OAuth is often used to allow users to access through third-party authentication.

Authorization is the process of determining whether a user or client has the right to access a specific resource or perform a specific operation after successful authentication. It defines the user's permissions and roles in the system and restricts access to resources based on these permissions.

Common authorization methods include:

• Role-Based Authorization: In role-based authorization, users are assigned to different roles, each with different permissions. For example, an administrator role may have access to all resources, while a normal user role may have only restricted permissions.
• Resource-Based Authorization: In resource-based authorization, access permissions are granted directly to specific resources rather than based on roles. Each resource can define its own permission rules that determine which users or roles can access it.
• Access Token: In OAuth authentication, access token is an important component for authorization. Access token contains authorization information about the user or client, as well as the resources and permissions authorized to access.

In general, authentication is used to confirm the identity of the user or client, while authorization is used to determine whether the user or client has the right to access specific resources or perform specific operations. These two steps together ensure that only legitimate and authorized users or clients can use the API and protect the system from unauthorized access. In the API gateway, authentication and authorization are very important functions because they directly affect the security of the entire system and the protection of data.

Step 5: Flow Control and Current Limitation

After the client's request passes authentication, the API gateway can perform higher-level traffic control and current limiting. As shown in the following figure:

Flow control and current limiting are important concepts used in API Gateway to manage and control request traffic. They help maintain the stability of backend services, prevent overloads, and provide better performance and reliability. The following explains these two concepts in detail:

Rate Limiting refers to controlling the request rate to limit the frequency of client requests to the API. This process ensures that the backend service is not overloaded with requests, avoiding excessive consumption of server resources, resulting in system crashes or slow responses. Common rate control methods include:

• Fixed Window Counter: Counts requests for each client or API key within a fixed time window (e.g. every minute). When the preset request limit is exceeded, additional requests are rejected or responses are delayed.
• Sliding Window Counter: Similar to the fixed window counter, but the window is sliding, allowing more flexible control of the request rate.
• Token Bucket Algorithm: Control the request rate by storing tokens in a bucket. Each token represents a request, and the bucket has a fixed capacity. Whenever there is a request, a token will be consumed, and when there are no tokens in the bucket, further requests are restricted.

Rate Limiting refers to limiting the number or rate of requests within a specific time period to prevent requests from exceeding the system's processing capacity. Unlike flow control, rate limiting does not reject additional requests, but temporarily places excess requests in a queue to wait for subsequent processing. Common rate limiting methods include:

• Leaky Bucket Algorithm: The leaky bucket algorithm maintains a bucket of fixed capacity, into which all requests are placed. Then, requests flow out of the bucket at a fixed rate. If there are too many requests, exceeding the capacity of the bucket, the excess requests will be cached or discarded.
• Token Bucket Algorithm: In addition to being a method of flow control, the token bucket algorithm can also be used for current limiting. Similar to flow control, excess requests will be placed in the bucket waiting to be processed.

Traffic control and current limiting are effective means to protect backend services from excessive requests. By setting reasonable request rate limits, you can balance the interaction between the client and the server and ensure the availability and stability of the service. In the API gateway, traffic control and current limiting are usually used together with authentication, authorization and other security measures to form a secure and efficient API management solution.

Step 6: Match the backend server

After HTTP verification is passed, the API Gateway finds the appropriate backend service to process the request through path matching, as shown below:

Step 7: Dynamic Routing

After matching the corresponding server, the request needs to be dynamically routed to any matched server. As shown below:

Step 8: Protocol Conversion

Because some companies are using microservices, and RPC protocol is used between microservices, the HTTP protocol needs to be converted into the corresponding RPC protocol in the API gateway. As shown in the following figure:

When the backend service processes the request, it returns a response to the API gateway, which converts the response back to the HTTP protocol and returns the response to the client. The API gateway also provides other key services. For example, the API gateway should track errors and provide circuit breaking functions to prevent service overload. The API gateway should also provide logging, monitoring, and analysis services to achieve operational observability.

Monitoring and logging are key components for achieving observability and troubleshooting in API gateways and systems. They help developers and system administrators track API usage, performance metrics, and error information in real time, so as to better understand the health of the system and quickly discover and resolve problems. The following explains these two concepts in detail:

Monitoring refers to the process of real-time collection, analysis, and display of various indicators and performance data in the API gateway and system. Through monitoring, we can understand the system's operating status, load, and resource usage, so that we can take timely measures to prevent or resolve potential problems. Common monitoring indicators include:

• Request volume and response time: Track the number of requests received by the API Gateway and how long it takes to process the requests.
• Error rate: records the number of errors that occur when the API Gateway processes requests, including HTTP error codes and custom errors.
• System resource usage: Monitor CPU, memory, disk, and network usage to ensure that system resources are sufficient and there are no resource bottlenecks.
• Request queue length: Tracks the length of the queue of requests waiting to be processed to prevent requests from piling up and causing performance degradation.
• Traffic trends: Understand the request traffic trends of the API to help predict the system load.

Monitoring can be achieved through various monitoring tools and services, such as Prometheus, Grafana, DataDog, etc. The monitoring results can be displayed in the form of charts, dashboards or alerts, allowing developers and operation and maintenance teams to understand the status of the system in real time and make corresponding optimizations and adjustments.

Logging refers to the process of recording key events, status, and error information in the API gateway and system. Logging is an important tool for tracking and debugging, which can provide valuable information when problems occur, helping developers to quickly locate problems and troubleshoot. Common logging content includes:

• Request and response logs: record each request received by the API gateway and the corresponding response, including the request header, request body, response code, response content, etc.
• Error log: records errors that occur when the API gateway processes requests, including exceptions, HTTP error codes, etc.
• Security log: records security-related events, such as authentication failure, authorization rejection, etc.
• Performance log: records performance indicators such as request processing time and resource usage.
• Tracking log: Records tracking information of requests between multiple microservices or backend services to help track the path and processing of requests.

Logs can be stored in local file systems, databases, or centrally managed and analyzed through log aggregation tools (such as ELK Stack: Elasticsearch, Logstash, Kibana). Logging is not only useful for troubleshooting, but also helps analyze user behavior, monitor security risks, etc.

In summary, monitoring and logging are important means to achieve observability and troubleshooting in API gateways and systems. Through monitoring and logging, we can detect potential problems in a timely manner, optimize system performance, and provide a better user experience.

Conclusion

This article explains the gateway and its main functions by analyzing the classic process of a client sending an HTTP request to a server. Because each company's business is different, the above process will also be different. In addition, because the degree of perfection of each company's basic services is different, the implementation of the gateway is different.

For example, some small companies directly use Nginx as a gateway because of their small business traffic, and then purchase some cloud security services. However, some companies with large business volumes need to develop an API gateway separately and then deploy it in a cluster.

In short, the gateway is a protective umbrella for the back-end server, and it acts as a barrier for requests from clients.