CryptoMB accelerates TLS handshake performance in service mesh Istio

Author: Yang Ailin, Intel Engineer (Cloud Orchestration Engineer)

The main content of this article is translated from CryptoMB - TLShandshake acceleration for Istio ​​blog​​

In the service mesh Istio, a large amount of mTLS (Mutual Transport Layer Security) authentication is used in the ingress gateway and the communication between microservices. A large amount of computing resources are required to process mTLS, which may become a performance bottleneck. This article describes how to use CryptoMB Private Key Provider configuration in the service mesh ingress gateway and sidecar proxy to accelerate the TLS (Transport Layer Security) handshake.

When it comes to mTLS, the TLS secure connection protocol, encryption and decryption operations are among the most computationally intensive and critical operations. The service mesh Istio uses the proxy program Envoy as a "gateway/sidecar" to handle secure connections and intercept network traffic.

Depending on the use case, when the ingress gateway has to handle a large number of TLS connection requests and handle secure connections between services through the sidecar proxy mode, the Envoy proxy load increases and performance degrades. Of course, the performance degradation depends on many factors, such as the size of the cpuset running the Envoy proxy, the incoming traffic pattern, and the key size. These factors will inevitably affect the response time of the Envoy proxy to new TLS requests. In order to improve and accelerate handshake performance, a new feature is introduced in Envoy 1.20 and Istio 1.14 to achieve TLS acceleration. It can be achieved by leveraging the third generation Intel® Xeon® Scalable Processor instruction AVX512, the Intel® Integrated Performance Primitives (Intel® IPP) cryptographic library, the CryptoMB Private Key Provider in Envoy, and the use of ProxyConfig configuration in Istio.

CryptoMB Introduction

Intel® IPP Crypto Library (https://github.com/intel/ipp-crypto/tree/develop/sources/ippcp/crypto_mb) supports multi-buffer encryption operations. Simply put, multi-buffer encryption refers to the use of SIMD (single instruction multiple data) mechanism, implemented by Intel® Advanced Vector Extensions 512 (Intel® AVX-512) instructions. Up to eight RSA or ECDSA operations are collected into a buffer and processed simultaneously, greatly improving the performance of encryption operations. In recent years, Intel's third-generation Intel® Xeon® Scalable Processors (Ice Lake servers) support Intel® AVX-512 instructions and their extended instructions, such as AVX512 IFMA, AVX512VAES.

The idea behind the CryptoMB Privatekey provider implementation in Envoy is to use Intel® AVX-512 multi-buffer instructions to accelerate RSA operations during the TLS handshake.

Accelerating Envoy with Intel AVX-512 instructions

Envoy uses BoringSSL as the default TLS protocol library. BoringSSL offloads asynchronous private key operations by supporting methods for setting private keys. For this purpose, Envoy implements a private key provider framework Private Key Provider to allow the creation of Envoy extensions that use BoringSSL hooks to handle TLS handshake private key operations (signing and decryption).

The CryptoMB private key provider is a concrete implementation of an Envoy extension Private Key Provider that uses Intel® AVX-512 multi-buffer acceleration technology to handle operations involving TLS RSA in BoringSSL. When a new handshake occurs, BoringSSL calls the private key provider to request encryption operations, and then returns control to Envoy, and the RSA requests are collected in the buffer. When the buffer is full or the timer expires, the private key provider calls Intel AVX-512 to process the buffer. After processing is completed, Envoy will be notified that the encryption and decryption operations are complete and the handshake can continue. The following figure shows the process call to achieve TLS acceleration:

Envoy <-> BoringSSL <-> PrivateKeyProvider

The Envoy worker thread has a buffer size of 8 RSA requests. When the first RSA request is stored in the buffer, a timer is started (the timer duration is set by the poll_delay field in the CryptoMB configuration).

Buffer timer started

When the buffer is full or the timer expires, encryption operations are performed on all RSA requests simultaneously. SIMD (Single Instruction Multiple Data) processing has potential performance benefits compared to the non-accelerated case.

Buffer timer expired

Envoy CryptoMB Private Key Provider configuration

When using Envoy, the regular TLS configuration only uses the private key. When using the Private Key Provider, the private key field will be replaced by the private key provider field. The Private Key Provider field contains two fields: the provider name "providername" and the type configuration "typed config". The type configuration is set to CryptoMbPrivateKeyMethodConfig. This configuration parameter is used to specify the private key and polling delay. The polling delay is to set the "poll_delay" mentioned above. Please see the specific TLS configuration as follows:

TLS configuration using only private key (multi-buffer acceleration is not enabled in this case)

 tls_certificates :
 certificate_chain : { "filename" :
 "/path/cert.pem" }
 private_key : { "filename" : "/path/key.pem" }

TLS configuration using CryptoMB private key provider (with multi-buffer acceleration enabled)

 tls_certificates :
 certificate_chain : { "filename" : "/path/cert.pem" }
 private_key_provider :
 provider_name : cryptomb
 typed_config :
 "@type" :
 type . googleapis . com / envoy . extensions . private_key_providers . cryptomb . v3alpha . CryptoMbPrivateKeyMethodConfig
 private_key : { "filename" : "/path/key.pem" }

 poll_delay : 10 ms

CryptoMB Private Key Provider Configuration in Istio

In Istio, there are several types of sidecar proxy configurations:

Pod level: Pod level configuration is set through resource annotations pod annotations

Mesh level: The mesh level is set using the global Mesh option proxyConfig

EnvoyFilter: Provides a mechanism to customize the Envoy configuration generated by Istio Pilot

CryptoMB Private Key Provider configuration can be applied to the entire mesh, a specific gateway, or a specific pod using pod annotations. Users can also set the "poll_delay" value for PrivateKeyProvider via ProxyConfig, which can also be applied to the entire mesh, i.e. the ingress gateway and all sidecar proxies.

A simple network-wide configuration is as follows:

Sample mesh wide configuration

Istio Mesh full network configuration example:

 apiVersion : install .istio .io / v1alpha1
 kind : IstioOperator
 metadata :
 namespace : istio - system
 name :
 example - istiocontrolplane
 spec :
 profile :
 demo
 components :
 egressGateways : -
 name : istio - egressgateway
 enabled : true
 ingressGateways :
 -
 name : istio - inggressgateway
 enabled : true
 meshConfig :
 defaultConfig :
 privateKeyProvider :
 cryptomb :
 pollDelay : 10 ms

Istio ingress gateway configuration

If the user only wants to configure the ingress gateway as a Private key Provider, the example is as follows:

 apiVersion : install .istio .io / v1alpha1
 kind : IstioOperator
 metadata :
 namespace : istio - system
 name :
 example - istiocontrolplane
 spec :
 profile :
 demo
 components :
 egressGateways :
 -
 name : istio - egressgateway
 enabled : true
 ingressGateways :
 -
 name : istio - inggressgateway
 enabled : true
 k8s :
 podAnnotations :
 proxy . istio . io / config : |
 privateKeyProvider :
 cryptomb :
 pollDelay : 10 ms

Using pod annotations to configure Istio sidecar proxy

If the user only wants to configure specific application Pods as private key provider, the easiest way is to use pod annotations. The example is as follows:

 apiVersion : v1
 kind : ServiceAccount
 metadata :
 name :
 httpbin
 -- -
 apiVersion : v1
 kind : Service
 metadata :
 name : httpbin
 labels :
 app :
 httpbin

 service : httpbin
 spec :
 ports :
 - name :
 http
 port :
 8000

 targetPort : 80

 selector :
 app :
 httpbin
 -- -
 apiVersion : apps / v1
 kind : Deployment
 metadata :
 name :
 httpbin
 spec :
 replicas : 1
 selector :
 matchLabels :
 app :
 httpbin
 version : v1
 template :

 metadata :

 labels :

 app : httpbin

 version : v1

 annotations :

 proxy . istio . io / config : |

 privateKeyProvider :

 cryptomb :

 pollDelay : 10 ms
 spec :

 serviceAccountName : httpbin

 containers :
 -
 image : docker .io / kennethreitz / httpbin

 imagePullPolicy : IfNotPresent

 name : httpbin

 ports :
 -
 containerPort : 80

Application Cases

Since September 2021, CryptoMB code has been integrated into the Envoy community mainline. For more information, please read the official document Envoy 1.23.0

api-v3 extension module ​​CryptoMb private key provider​​. If you are using Istio 1.13, Istio 1.14 or later versions, it already includes the corresponding version of Envoy, such as Envoy 1.22, which already includes CryptoMB by default without having to compile it yourself. This acceleration capability based on CryptoMB encryption/decryption can be applied to the ingress gateway

istio-ingress-gateway can also be used for microservice proxy istio -proxy sidecar containers. Before using, check whether your system is a 3rd generation Intel® Xeon® Scalable processor (codenamed Ice Lake). This processor supports

AVX512 extended instructions, multi-cache instructions. And this acceleration capability includes:

• AVX-512 crypto acceleration for TLS connections
• AVX-512 vector AES for symmetric data encryption

CryptoMB has been adopted by Alibaba Cloud Service Grid product ASM to accelerate TLS. Alibaba Cloud Service Grid ASM combines this Multi-Buffer technology and configures and enables this function to provide TLS acceleration. The usage is as follows:

1) When Multi-Buffer is not enabled, the TLS configuration only needs to include a private key. Once Multi-Buffer is enabled, a private key provider needs to be provided in the TLS configuration, which is the cryptoMB private key provider mentioned above. The message processed by the provider is of the CryptoMbPrivateKeyMethodConfig type, which includes two main fields, one is the private key used, and the other is used to set the waiting time pool_delay that each thread processing queue should be executed, which is used to control the balance between delay and throughput;

2) The control plane configuration can be sent to the data plane Envoy proxy through the xDS protocol. Combined with the Intel® IPP encryption library and CryptoMB private key provider, using the AVX512 instruction set, the service mesh implementation can offload TLS handshakes to handle more connections, reduce latency and save CPU.

3) Alibaba Cloud Service Grid ASM determines the machine model to determine whether the machine supports the AVX512 instruction set, and then decides whether to enable this function.

In Alibaba Cloud Service Mesh ASM products, global configuration is currently provided, and multi-level configuration is being gradually supported. We tested on Alibaba Cloud's G7 machines, enabled multi-Buffer, and performed TLS performance tests on ASM products. We requested
The number of QPS increased by 75 % (this data is public data, from Alibaba Service Grid Product Introduction )