Learn more about load balancers

Every load balancer is a reverse proxy, but not every reverse proxy has to be a load balancer.

Question: What does the OSI model look like?

Question: What are the requirements for a load balancer?

Answer → To create a fault tolerant system so that when a client makes a request to the load balancer, it can communicate with one or more backends.

Question: Explain how a layer 4 load balancer works?

Answer → A layer 4 load balancer works as follows:

(1) Handshake → The Layer 4 load balancer first establishes a TCP connection with the backend server.

(2) Warm-up → The load balancer can establish multiple TCP-based connections with the backend and then keep those connections active. This is to ensure that whenever a client connects to the load balancer, it does not have to go through the connection warm-up process every time.

(3) Client Connection → When a client connects to a Layer 4 load balancer, the connection will have a state at the load balancer level and will be mapped to any one of the connections to the backend servers. Hence, Layer 4 load balancers are stateful.

• Layer 4 load balancers only deal with ports and IP addresses.
• The data is segmented, but it cannot simply touch the data or parse the data.
• All data segments sent by the client to the Layer 4 load balancer will ultimately be sent to a dedicated connection on a server.
• If a client sends some data to a layer 4 load balancer, it can't just send one segment to one server and then the other segment to another server, because the data will be corrupted and everything will go wrong.

Question: Does a layer 4 load balancer also act as a NAT layer?

Answer → Yes, a layer 4 load balancer acts as a NAT layer/gateway/router for the clients.

• Request → Load balancer changes the target IP address while establishing a brand new TCP connection to the backend. The client is completely unaware of this concept.

• Response → The layer 4 load balancer knows that anything received from the application connection must be sent back to this connection with the original client. This is the mapping table maintained by the load balancer.

Question: Can you give an example of HTTP-based calls (Restful API calls) from the client to the backend?

Answer → Imagine a client sends an Http call to a load balancer. Also imagine that this particular request is split into 2 segments.

Step #1) The request arrives at the load balancer, which then chooses to send all the segments of this request to a certain backend server.

Step #2) Now, suppose the client sends another segment (segment #3) and then writes it back to the same connection.

• There is no buffering on this layer 4 load balancer. It just reads and forwards the segments.
• The layer 4 load balancer does not know HTTP. It only knows and understands TCP. All the data that is passed to this layer 4 load balancer (whether it is a gRPC-based request or a socket-based connection), it treats all of this data as data segments.
• However, depending on the MTU on the client side and the MTU on the backend server side, the load balancer may perform some kind of optimization. It may simply take a segment and further break that segment into multiple segments or vice versa. TCP will always try to squeeze out as much performance as possible.

Step #3) Now, suppose the same client initiates another request on the same connection, then that request will also be read first and then written back to the backend server.

Step #4) Now, suppose a different client makes another request, then depending on the load balancing algorithm, the request may be forwarded to another backend server.

Question: What companies provide L4 load balancers?

Answer → Companies like Radware and F5-BigIp provide L4 load balancers.

Question: What are the pros and cons of L4 load balancers?

(1) The following are the advantages of L4 LB:

• This is a simple load balancing strategy. It just doesn't read the layer 7 stuff.
• It is safer because it does not read data from L7.
• It works with any protocol. It is agnostic to the protocol.

(2) The following are the disadvantages of L4 LB:

• It is not smart and does not work with microservices. It has sticky connections. There is no load balancing for each connection. Although we can configure the IP and port combination, it is not recommended in production systems.
• If the client sends a POST request or sends a GET request or sends a PATCH request, the L4 load balancer knows nothing about that. It just sees those requests as incoming segments of data.
• It cannot support caching because it does not know what to cache as it cannot read the data in the L7 layer.
• On L4 LB we cannot do anything like "block certain users", "block certain headers", "block certain authentication methods".

Question: Explain how a layer 7 load balancer works?

Answer → A layer 7 load balancer works as follows:

(1) Handshake → The Layer 7 load balancer also establishes a TCP connection with the backend server.

(2) Warm-up → The load balancer can establish multiple TCP-based connections with the backend and then keep those connections active. This is to ensure that whenever a client connects to the load balancer, it does not have to go through the connection warm-up process every time.

(3) Client Connection → When a client connects to a Layer 7 load balancer, the connection becomes protocol specific.

• A layer 7 load balancer needs to understand anything that is sent to it.
• Any logical request will first be buffered at the L7-LB level, understand it (i.e. decrypt the data), parse it, and then decide to forward the request to one of the backend servers.

NOTE: If you want your L7 LB to face external clients, then:

• Your L7 LB must have a certificate stored on it, because this L7 LB must prove its identity first.
• The L7 LB must also decrypt the data.

Question: How does an L7 load balancer handle HTTP based requests?

Step #1) First, it receives the request and sends it to the backend server 1.

Step #2) Next, suppose the same client sends another request, then L7 LB can actually forward it to other backend servers as well since L7 LB is stateless.

Question: What are the pros and cons of L7 load balancers?

(1) The following are the advantages of L4 LB:

• It provides intelligent load balancing strategies. For example, if the request is for the endpoint "/pictures", the request can be forwarded to a specific host group, perhaps the Pictures microservice.
• As another example, if the request is for the endpoint GET "/comments", the request can be forwarded to a specific host group, perhaps the comments microservice.
• Another example, if the request is to the endpoint POST "/comments", it is a write-intensive workload, so it may go to another host group because it has a specific database designed for this write-intensive workload.
• Therefore, L7 LB is very suitable for microservices.
• We can also perform AuthN and AuthZ on this L7 load balancer. • We can also cache the results.

(2) The following are the disadvantages of L4 LB:

• It's expensive because it does more work like buffering data, reading data, decrypting data, terminating the TLS connection.
• It must share a TLS certificate.
• It might not be efficient and might actually be a bottleneck because it's buffering a lot of data and parsing it. It might slow things down.
• It requires understanding the protocol.