Oh my god, you don’t even know how to use API Gateway!

[Original article from 51CTO.com] From the evolution of application architecture, we can find that as business variability and flexibility continue to increase, applications need to respond with more flexible combinations.

[[326027]]

Image via Pexels

At the same time, in order to cope with the challenges of business segmentation and high concurrency, the microservice architecture is widely used, because the application in the microservice architecture will be split into multiple services.

In order to facilitate the client to call these services, the concept of API was introduced. Today we will take a look at the principle of API gateway and how it is applied.

Definition of API Gateway

The term gateway first appeared in network devices. For example, two independent local area networks communicate through a router, and the router in the middle is called a gateway.

In terms of development, it is a gateway between the client and the microservice system. In terms of business, when the client completes a business, it needs to call multiple microservices at the same time.

As shown in Figure 1, when the client initiates an order request, it needs to call services such as product query, inventory deduction, and order update.

Figure 1: Comparison before and after adding API Gateway

If these services need to be called separately by the client to complete, it will increase the complexity of the request and also cause loss of network call performance. Therefore, the call of API gateway is introduced for the application scenario of microservices.

An ordering API gateway is added between the client and the microservices. The client directly issues commands to this API gateway, which completes the calls to the other three microservices and returns the results to the client.

From a system perspective, if any application system needs to be called by other systems, it needs to expose APIs, which represent functional points.

As mentioned in the order example above, if an order function needs to call multiple services, the calls of multiple services need to be aggregated in the order API gateway.

This aggregation method is a bit like the Facade pattern in the design pattern, which provides a unified access point for external calls.

Not only that, as shown in Figure 2, the API gateway can also assist in the communication between two systems by adding a mediator between the systems to assist in the API call.

Figure 2: API gateway connecting two systems

From the perspective of client type, an API gateway can also be added to shield the differences in calls between different clients.

As shown in Figure 3, in the actual development process, the API gateway can also provide different API gateways corresponding to different client types (iOS, Android, PC, mini-programs).

Figure 3: API Gateway connecting client and server

Since the API gateway is located at the junction of the client and the microservice, its functions also include: routing, load balancing, current limiting, caching, logging, publishing, etc.

Spring Cloud Gateway Concepts and Definitions

In the definition of API Gateway, we mentioned why we use API Gateway. It is to solve the problem of client access to multiple microservices.

Since the segmentation of services means that one operation needs to call multiple services at the same time, a unified facade is provided for the aggregation of these services, and this facade is the API gateway.

There are many ways to implement API Gateway, such as Zuul, Kong, etc. Here we will take Spring Cloud Gateway as an example to introduce its specific implementation.

Generally speaking, the API gateway aggregates microservices internally and exposes a unified URL or interface information externally for client calls.

So how does the client connect to and communicate with microservices? The following important concepts need to be introduced.

Figure 4: Routes, assertions, and filters

As shown in Figure 4, Spring Cloud Gateway consists of three parts:

①Route: Any request from the client will be routed and then go to the corresponding microservice.

Each route has a unique ID and a corresponding destination URL, and also contains several predicates and filters.

② Predicate: When the client enters Spring Cloud Gateway through an Http Request, the predicate will match the Http Request according to the configured routing rules.

To put it simply, it is to perform one or more if judgments. If the match is successful, the next step is carried out. Otherwise, the assertion fails and the error message is directly returned.

③Filter: Simply put, it is to filter the requests that pass through, or to obtain and modify them. Note that the function of the filter is bidirectional, that is, both the request and the response will be modified.

Generally speaking, there are two types of filters in Spring Cloud Gateway:

• Gateway Filter
• Global Filter

Gateway Filter is used on single routes and group routes. Global Filter can act on all routes and is a global filter.

How Spring Cloud Gateway works

After talking about the definition and elements of Spring Cloud Gateway, let's take a look at its working principle. In general, it is the process of processing client requests.

Figure 5: Spring Cloud Gateway request processing flow chart

As shown in Figure 5, when the client initiates a request to Spring Cloud Gateway, the request is obtained by HttpWebHandlerAdapter, and the request is extracted to assemble into the gateway context.

Pass the composed context information to the DispatcherHandler component. As a request dispatching processor, DispatcherHandler is mainly responsible for dispatching requests to the corresponding processor for processing.

The processor requested here includes RoutePredicate HandlerMapping (route assertion processing mapper).

The route assertion processing mapper is used to find the route and return the corresponding FilteringWebHandler after finding the route.

It is responsible for assembling the Filter list and performing filtering, and then forwarding the request to the application service. After the application service completes the processing, it finally returns the Response to the client.

When FilteringWebHandler processes the request, it will be handed over to Filter for filtering.

It should be noted here that since the Filter is bidirectional, when the client requests a service, the request will be processed by the Filter in the Pre Filter.

When the service returns to the client after processing the request, it will be processed again through the Post Filter.

Spring Cloud Gateway Best Practices

The above introduces the definition and implementation principle of Spring Cloud Gateway. The following introduces how Spring Cloud Gateway implements gateway functions based on several common scenarios.

We will introduce basic routing, weighted routing, current limiting, and dynamic routing.

Basic Routing

The main function of basic routing is to point to the corresponding URI according to the defined path when the client requests. This process requires the use of the Path routing predicate processor in Predicates.

First, add the corresponding dependencies to the POM file as follows:

 <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-gateway</artifactId>
 </dependency>

Add the following code, where the path "/baidu" defined in the Path is the path address during the request. The corresponding URI, http://www.baidu.com/, is the target address to jump to.

 @Bean
 public RouteLocator routeLocator(RouteLocatorBuilder builder) {
 return builder.routes()
         .route(r ->r.path( "/baidu" )
               .uri( "http://www.baidu.com/" ).id( "baidu_route" )
 ).build();
 }

Similarly, the above functions can also be implemented in the yml file. The configuration file is as follows. To put it simply, it is the setting of Path and URI parameters. The functions implemented are consistent with the above code.

 spring:
 cloud:
 gateway:
 routes:
 - id: baidu_route
        uri: http://baidu.com:80/
 predicates:
 - Path=/baidu

Now start the API gateway. Assuming the gateway's access address is "localhost:8080/baidu", when the user requests this address, it will automatically request the website "www.baidu.com". This configuration is very simple, and friends who have a basic understanding of Nginx should be able to get started quickly.

Weighted Routing

This usage scenario is more common than the simple routing above. When each microservice releases a new version, it usually keeps the old version and the new version coexisting.

Then, the gateway gradually switches the traffic from the old version of the service to the new version of the service. This gradual switching process is often called grayscale release.

At this point, the API gateway plays the role of traffic distribution. Generally speaking, the earliest old version will carry more traffic. For example, 90% of requests will be routed to the old version of the service, and only 10% of requests will be routed to the new service.

This allows us to observe the stability of the new service or get user feedback. Once the new service is stable, we can then import the remaining traffic to it.

Figure 6: Grayscale release, routing to new/old services

As shown in the following code, assuming that the API gateway still uses port 8080, you need to configure routing weights for two different services. Therefore, configure service_old and service_new under routes respectively.

 server.port: 8080
 spring:
 application:
 name : gateway-test
 cloud:
 gateway:
 routes:
 - id: service_old
        uri: http://localhost:8888/v1
 predicates:
 - Path=/gatewaytest
 -Weight=service, 90
 - id: service_new
        uri: http://localhost:8888/v2
 predicates:
 - Path=/gatewaytest
 - Weight=service, 10

The corresponding URIs in the two configurations are the access addresses of the new and old services, respectively, distinguished by "http://localhost:8888/v1" and "http://localhost:8888/v2".

The paths defined in the Predicates are all "/gatewaytest", which means that the access paths are the same for the client. From the path, the client will not know whether they are accessing a new service or an old service.

The main parameter is Weight, which is configured as 90 for the old service and 10 for the new service. That is, 90% of the traffic will request the old service, and 10% of the traffic will request the new service.

To put it simply, if there are 100 requests, 90 of them will request v1 (old service) and the other 10 will request v2 (new service).

Current Limitation

When a service encounters high concurrency in a short period of time and the concurrency exceeds the service's tolerance, current limiting is required. For example: flash sales, rush purchases, and ordering services.

Protect services by limiting the request rate or limiting the request rate within a time window. When the limit rate is reached, the request can be rejected, an error code can be returned, or the request can be directed to a friendly page.

General middleware will have a single-machine current limiting framework, which supports two current limiting modes:

• Control rate
• Controlling concurrency

Here, we use Bucket4j in Guava to implement current limiting operations. As usual, we introduce the dependency of Bucket4j:

 <dependency>
    <groupId>com.github.vladimir-bukhtoyarov</groupId>
    <artifactId>bucket4j-core</artifactId>
 <version>4.0.0</version>
 </dependency>

Since user requests need to be monitored, a custom Filter is created by implementing GatewayFilter, and then the custom Filter is applied through Gateway API Application.

Here we use the token bucket method to limit the flow, so we need to set the bucket capacity (capacity), the number of tokens to be filled each time (refillTokens), and the interval between filling tokens (refillDuration).

After initializing these three parameters, a token bucket is created for the request through the createNewBucket method, and the main logic of current limiting is implemented in the Filter method.

The IP information in the context of the request is obtained through ServerWebExchange, and a corresponding token bucket is established for the IP. The correspondence between the IP and the token bucket is placed in LOCAL_CACHE.

Each time a request passes through, a token is consumed through the tryConsume(1) method. When there is no token left, the HttpStatus.TOO_MANY_REQUESTS status code (429) is returned. At this time, the gateway directly returns that the number of requests is too large, and even if there are more requests coming in, they will not be routed to the corresponding service.

Only when a certain number of tokens are put into the bucket after waiting for the next time interval can the requester obtain the token in the bucket and then request service again.

 public class GatewayRateLimitFilterByIp implements GatewayFilter, Ordered {
    private static final Map<String, Bucket> LOCAL_CACHE = new ConcurrentHashMap<>();
 int capacity;
 int refillTokens;
 Duration refillDuration;
 public GatewayRateLimitFilterByIp() {
 }
 public GatewayRateLimitFilterByIp( int capacity, int refillTokens, Duration refillDuration) {
        this.capacity = capacity;
        this.refillTokens = refillTokens;
        this.refillDuration = refillDuration;
 }
    private Bucket createNewBucket() {
        Refill refill = Refill. of (refillTokens, refillDuration);
        Bandwidth limit = Bandwidth.classic(capacity, refill);
 return Bucket4j.builder().addLimit(limit).build();
 }
 @Override
 public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String ip = exchange.getRequest().getRemoteAddress().getAddress().getHostAddress();
        Bucket bucket = LOCAL_CACHE.computeIfAbsent(ip, k -> createNewBucket());
        if (bucket.tryConsume(1)) {
 return chain.filter(exchange);
 } else {
 exchange.getResponse().setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
 return exchange.getResponse().setComplete();
 }
 }
 }

The above code defines the Filter, which generates a token bucket for the accessed IP, and defines the bucket size, the number of tokens put into the bucket each time, and the interval between putting tokens.

And the filtering logic is rewritten through the Filter method, so the following only needs to apply this Filter to the rules of Spring Cloud Gateway. Define the route assertion and filter of the gateway through the following code.

Create a new filter in Filters with the filter defined in the above code, specify the capacity as 20, and put in tokens every two seconds, one token at a time.

Then when the user accesses the rateLimit path, the flow will be limited according to the customized Filter.

 @Bean
 public RouteLocator testRouteLocator(RouteLocatorBuilder builder) {
 return builder.routes()
            .route(r -> r.path( "/rateLimit" )
                    .filters(f -> f.filter(new GatewayRateLimitFilterByIp(20,1,Duration.ofSeconds(2))))
                    .uri( "http://localhost:8888/rateLimit" )
                    .id( "rateLimit_route" )
 ).build();
 }

The current limiting here is just to provide you with an idea. By implementing GatewayFilter, rewriting the Filter method, adding traffic control code, and then applying it in Spring Cloud Gateway, you can do it.

Dynamic Routing

Since Spring Cloud Gateway itself is also a service, the routing configuration cannot be modified once it is started.

Regardless of the encoding injection method or the configuration method mentioned above, if modifications are required, the service needs to be restarted.

If we go back to the original definition of Spring Cloud Gateway, we will find that each user's request accesses the corresponding microservice through Route, which includes the definitions of Predicates and Filters.

As long as you implement the definition of Route and the Predicates and Filters it contains, and then provide an API interface to update this definition, you can dynamically modify the routing information.

According to this idea, the following steps need to be taken to achieve it:

①Define Routes, Predicates and Filters

Predicates and Filters are included in Route, which is actually the definition of Route entity and the configuration of routing rules for Route.

 public class FilterDefinition {
 //Filter Name
 private String name ;
 //Corresponding routing rules
    private Map<String, String> args = new LinkedHashMap<>();
 }
 public class PredicateDefinition {
 //Predicate Name
 private String name ;
 //Corresponding assertion rules
    private Map<String, String> args = new LinkedHashMap<>();
 }
 public class RouteDefinition {
 //Assertions collection
 private List<PredicateDefinition> predicates = new ArrayList<>();
 //Route collection
 private List< FilterDefinition > filters= new ArrayList<>();
 //uri
 private String uri;
 //Execution order
 private int order = 0;
 }

②Implement routing rule operations, including adding, updating, and deleting

With the definition of the route (Route, Predicates, Filters), we can then write operations for the route definition.

For example: add route, delete route, update route, etc. Write RouteServiceImpl to implement ApplicationEventPublisherAware.

You mainly need to override the setApplicationEventPublisher method, where you will pass in the ApplicationEventPublisher object, through which you can publish the events defined by the route: add, update, and delete.

Some of the code is posted as follows:

 @Service
 public class RouteServiceImpl implements ApplicationEventPublisherAware {
 @Autowired
    private RouteDefinitionWriter routeDefinitionWriter;
    private ApplicationEventPublisher publisher;
 //Add routing rules
 public String add (RouteDefinition definition) {
        routeDefinitionWriter.save(Mono.just(definition)).subscribe();
        this.publisher.publishEvent(new RefreshRoutesEvent(this));
 return "success" ;
 }
 public String update (RouteDefinition definition) {
 try {
          this.routeDefinitionWriter.delete ( Mono.just (definition.getId()));
        } catch (Exception e) {
 }
 try {
            routeDefinitionWriter.save(Mono.just(definition)).subscribe();
            this.publisher.publishEvent(new RefreshRoutesEvent(this));
 return "success" ;
        } catch (Exception e) {
 }
 }
 public String delete (String id) {
 try {
            this.routeDefinitionWriter.delete ( Mono.just (id));
 return "delete success" ;
        } catch (Exception e) {
 }
 }
 @Override
 public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
        this.publisher = applicationEventPublisher;
 }

③Providing an API interface to the outside world allows users or programs to dynamically modify routing rules

From the code point of view, it is a Controller. In this Controller, you only need to call routeServiceImpl, and mainly use the add, update, and delete methods in the customized route implementation class.

To put it simply, it is packaged so that external systems can call it and modify the routing configuration.

The simplified code is as follows. Here, only the add method is packaged. The update and delete methods are not described in detail. The calling method is similar to add.

 public class RouteController {
 @Autowired
    private routeServiceImpl routeService;
 @PostMapping( "/add" )
 public String add (@RequestBody RouteDefinition routeDefinition) {
 try {
            RouteDefinition definition = assembleRouteDefinition(routeDefinition);
 return this.dynamicRouteService.add ( definition );
        } catch (Exception e) {
 }
 return "succss" ;
 }
 }

④Start the program to add and update routes

Assume that the service that updates the API gateway configuration is on port 8888. Then access the current routing information through http://localhost:8888/actuator/gateway/routes. Since no routing is configured now, this information is empty.

Then add a routing rule through http://localhost:8888/route/add, select Post request here, and input type as Json as follows:

 {
 "filter" :[],
 "id" : "baidu_route" ,
 "order" :0,
 "predicates" :[{
 "args" : {
 "pattern" : "/baidu"
 },
 "name" : "Path"
 }],
 "uri" : "https://www.baidu.com"
 }

The content configured in Json is very similar to that of simple route configuration. When Route is set, when Predicates is baidu, the request will be directed to the website www.baidu.com for response.

At this time, when you access the website through the path http://localhost:8888/baidu, you will be routed to the website www.baidu.com.

If you need to modify the route configuration, you can access the API interface http://localhost:8888/route/update and pass in the Json structure through the Post method, for example:

 {
 "filter" :[],
 "id" : "CTO_route" ,
 "order" :0,
 "predicates" :[{
 "args" : {
 "pattern" : "/CTO"
 },
 "name" : "Path"
 }],
 "uri" : "https://www.51CTO.com"
 }

After the update is complete, when you visit http://localhost:8888/CTO, you will be directed to the website www.51CTO.com.

Through the above four steps, you can dynamically change the routing configuration information even without restarting the Spring Cloud Gateway service.

Summarize

Due to the popularity of microservices, API gateways have quietly emerged. The reason for the existence of API gateways is explained. It not only provides a service facade, but also coordinates the communication between different systems and serves different client interfaces.

The best time for API Gateway is to explain the definition and concepts of Spring Cloud Gateway, which implements routing, filters, assertions, and can route different client requests to different microservices, as well as how several components work together to complete the routing work.

In the introduction of best practices, basic routing, weighted routing, current limiting and dynamic routing are explained respectively.

[51CTO original article, please indicate the original author and source as 51CTO.com when reprinting on partner sites]