An article giving you a first experience with Apache APISIX

Apache APISIX is a dynamic, real-time, high-performance API gateway based on OpenResty and Etcd. It has entered Apache for incubation. It provides rich traffic management functions, such as load balancing, dynamic routing, dynamic upstream, A/B testing, canary release, rate limiting, circuit breaking, defense against malicious attacks, authentication, monitoring indicators, service observability, service governance, etc. APISIX can be used to handle traditional north-south traffic and east-west traffic between services.

Compared with traditional API gateways, APISIX has dynamic routing and hot loading plug-in functions, avoiding the reload operation after configuration. At the same time, APISIX supports more protocols such as HTTP(S), HTTP2, Dubbo, QUIC, MQTT, TCP/UDP, etc. It also has a built-in Dashboard, providing a powerful and flexible interface. It also provides rich plug-in support functions and allows users to customize plug-ins.

The above picture is the architecture diagram of APISIX, which is generally divided into two parts: the data plane and the control plane. The control plane is used to manage routing, mainly through etcd to implement the configuration center. The data plane is used to process client requests, which is implemented by APISIX itself and will continuously watch the route, upstream and other data in etcd.

APISIX Ingress

As an API gateway, APISIX also supports being used as an Ingress controller for Kubernetes. The architecture of APISIX Ingress is divided into two parts. One part is APISIX Ingress Controller, which will complete configuration management and distribution as the control plane. The other part, APISIX (agent), is responsible for carrying business traffic.

When the client initiates a request and it reaches Apache APISIX, it will directly transfer the corresponding business traffic to the backend (such as Service Pod) to complete the forwarding process. This process does not need to go through the Ingress Controller, which ensures that if there is a problem, or if changes, expansion or migration are made, it will not affect users and business traffic.

At the same time, on the configuration side, users can create resources through kubectl apply and apply custom CRD configurations to the K8s cluster. The Ingress Controller will continue to watch these resource changes to apply the corresponding configurations to Apache APISIX (through the admin api).

As can be seen from the above figure, APISIX Ingress uses a data plane and control plane separation architecture, so users can choose to deploy the data plane inside or outside the K8s cluster. However, Ingress Nginx puts the control plane and data plane in the same Pod. If there is a slight error in the Pod or the control plane, the entire Pod will crash, which will affect the business traffic. This architectural separation provides users with a more convenient deployment option, and also facilitates the migration and use of related data in the business architecture adjustment scenario.

The core features currently supported by the APISIX Ingress controller include:

• Fully dynamic, supports advanced routing matching rules, and can be extended with more than 50 official Apache APISIX plug-ins & customer-defined plug-ins
• Support CRD, easier to understand declarative configuration
• Compatible with native Ingress resource objects
• Support traffic splitting
• Automatic service registration and discovery, no need to worry about scaling
• More flexible load balancing strategy with built-in health check function
• Support gRPC and TCP layer 4 proxy

Install

We use APISIX in the Kubernetes cluster here. We can install it through Helm Chart. First, add the official Helm Chart repository:

 ➜ helm repo add apisix https://charts.apiseven.com
 ➜ helm repo update  

Since the APISIX Chart package contains the dependencies of the dashboard and ingress controller, we only need to enable it in values ​​to install the ingress controller:

 ➜ helm fetch apisix/apisix
 ➜ tar -xvf apisix-0.7.2.tgz
 ➜ mkdir -p apisix/ci

Create a new values ​​file for installation in the apisix/ci directory, with the following content:

 # ci/prod.yaml
 apisix:
 enabled: true   
 
 nodeSelector: # Fixed on node2
    kubernetes.io/hostname: node2 
 
 gateway:
 type: NodePort
  externalTrafficPolicy: Cluster
 http:
 enabled: true  
 servicePort: 80
 containerPort: 9080 
 
 etcd:
 enabled: true # A 3-node etcd cluster will be automatically created
 replicaCount: 1 # Multiple replicas require modifying the template. Here we temporarily run an etcd pod 
 
 dashboard:
 enabled: true   
 
 ingress-controller:
 enabled: true  
 config:
 apisix:
      serviceName: apisix-admin
 serviceNamespace: apisix #Specify the namespace. If it is not ingress-apisix, you need to re-specify it.

After testing, the official Helm Chart package does not support etcd multi-node clusters very well. I tested running 3 nodes and there was a problem. The template should be modified to be compatible. In addition, the compatibility with external etcd tls clusters is not good. For example, the dashboard Chart needs to modify the template to support tls. So here we test to change it to a 1-copy etcd cluster first.

APISIX needs to rely on etcd. By default, Helm Chart will automatically install a 3-replica etcd cluster and provide a default StorageClass. If you already have a default storage class, you can ignore the following steps. Here we install an nfs provisioner. Use the following command to install a default StorageClass:

 ➜ helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
 ➜ helm upgrade --install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \  
 --set nfs.server=192.168.31.31 \ #Specify nfs address  
 --set nfs.path=/var/lib/k8s/data \ # nfs path  
 --set image.repository=cnych/nfs-subdir-external-provisioner \  
 --set storageClass.defaultClass=true -n kube-system  

A StorageClass will be automatically created after the installation is complete:

 ➜ kubectl get sc
 NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
 nfs-client ( default ) cluster. local /nfs-subdir-external-provisioner Delete Immediate true 35s

Then directly execute the following command for one-click installation:

 ➜ helm upgrade --install apisix ./apisix -f ./apisix/ci/prod.yaml -n apisix  
 Release "apisix" does not exist. Installing it now.
 NAME : apisix
 LAST DEPLOYED: Thu Dec 30 16:28:38 2021
 NAMESPACE: apisix
 STATUS: deployed
 REVISION: 1
 NOTES:
 1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace apisix -o jsonpath="{.spec.ports[0].nodePort}" services apisix-gateway)  
  export NODE_IP=$(kubectl get nodes --namespace apisix -o jsonpath="{.items[0].status.addresses[0].address}")  
  echo http://$NODE_IP:$NODE_PORT

Normally, apisix can be deployed successfully:

 ➜ kubectl get pods -n apisix
 NAME READY STATUS RESTARTS AGE
 apisix-dashboard-b69d5c768-r6tqk 1/1 Running 0 85m
 apisix-etcd-0 1/1 Running 0 90m
 apisix-fb8cdb569-wz9gq 1/1 Running 0 87m
 apisix-ingress-controller-7d5bbf5dd5-r6khq 1/1 Running 0 85m
 ➜ kubectl get svc -n apisix
 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
 apisix-admin ClusterIP 10.97.108.252 <none> 9180/TCP 3h
 apisix-dashboard NodePort 10.108.202.136 <none> 80:31756/TCP 3h
 apisix-etcd ClusterIP 10.107.150.100 <none> 2379/TCP,2380/TCP 3h
 apisix-etcd-headless ClusterIP None <none> 2379/TCP,2380/TCP 3h
 apisix-gateway NodePort 10.97.214.188 <none> 80:32200/TCP 3h
 apisix-ingress-controller ClusterIP 10.103.176.26 <none> 80/TCP 3h

test

Now we can create a routing rule for the Dashboard by creating a new ApisixRoute resource object as shown below:

 apiVersion: apisix.apache.org/v2beta2
 kind: ApisixRoute
 metadata:
 name : dashboard
 namespace: apisix
 spec:
 http:
 - name : root
 match:
 hosts:
          - apisix.qikqiak.com
 paths:
 - "/*"  
 backends:
      - serviceName: apisix-dashboard
 servicePort: 80

After creation, apisix-ingress-controller will map the above resource objects to the configuration in APISIX through the admin api:

 ➜ kubectl get apisixroute -n apisix
 NAME HOSTS URIS AGE
 dashboard [ "apisix.qikqiak.com" ] [ "/*" ] 75m

So in fact, our access point is APISIX, and apisix-ingress-controller is just a tool for listening to crds and then translating crds into APISIX configuration. Now we can access our dashboard through the NodePort port of apisix-gateway:

The default login username and password are both admin. After logging in, you can see the routing information of the dashboard we created above under the routing menu:

Click View below to see the actual routing configuration information under APISIX:

Therefore, if we want to use APISIX, we must also understand the concept of route. Route is the entry point of the request. It defines the matching rules between client requests and services. Routes can be associated with services and upstreams. A service can correspond to a group of routes, and a route can correspond to an upstream object (a group of backend service nodes). Therefore, each request that matches the route will be proxied by the gateway to the upstream service bound to the route.

After understanding the routing, we naturally know that we also need an upstream Upstream to associate. This concept is basically the same as the Upstream in Nginx. Under the Upstream menu, you can see the upstream service corresponding to the dashboard we created above:

In fact, it is to map the Endpoints in Kubernetes to the Upstream in APISIX, and then we can load it ourselves on the APISIX side.

The Dashboard function provided by APISIX is very comprehensive. We can even make all configurations directly on the page, including plug-ins, which is very convenient.

Of course, there are many other advanced features, such as traffic segmentation, request authentication, etc. These advanced features are more convenient to use in crds. Of course, they also support native Ingress resource objects. More advanced uses of APISIX will be explained later.