Learn about HTTP proxy in five minutes

[[429696]]

"If you have any questions or dissatisfaction, please talk to my agent." We often hear that a certain star has revealed some explosive news, but the person concerned does not explain, but the agent or studio explains a lot of nonsense.

They are like another mouth for the stars, completing the work that they themselves cannot or are unwilling to do.

The agent here is an entity in real life, a real person.

On the Internet, there is also an entity that replaces the network entity to complete the work that it cannot complete. This network entity is called a server, but it also has a special name: Web proxy.

The network entities above actually refer to the client and the server.

A proxy is usually located between a server and a client, acting as a middleman to pass HTTP messages between various points. Without a proxy, the HTTP client would have to communicate directly with the HTTP server.

So, why doesn't the HTTP client talk directly to the HTTP server, but has to add a proxy in the middle? What role does it play?

First of all, we all know that some foreign websites cannot be accessed in China, but what if we want to visit a gay dating website? Here we need to use a proxy, which can break through its own IP restrictions and visit foreign sites. There are also some confidential companies that prohibit access to the external network, so what should we do if we want to check information? Use a proxy.

Secondly, the proxy can also improve network bandwidth and speed up access. The proxy server will store part of the bandwidth, and there will be a large buffer inside the proxy server. When certain pages are visited, the proxy server will cache these pages. When the same page is visited next time, the proxy server will directly return the buffered page. In this way, the proxy server will save bandwidth while increasing access speed.

The proxy will also hide the real IP of your host, and we can also protect ourselves from network attacks in this way.

In general, the functions of the agent are mainly as follows.

Private and public proxies

A proxy server can provide proxy services for many clients, and a proxy can also be dedicated to a certain client. Just like Tom, the military advisor in the Godfather, only serves the Corleone family, while most lawyers in a law firm serve the public. Therefore, depending on the different responsibilities, a proxy dedicated to a single client is generally called a private proxy, while a proxy that serves most clients is called a public proxy.

Public Agent

Most of the proxy servers we see are public proxies. The biggest feature of public proxies is sharing, but sharing also means risks. It is generally not recommended to use this type of proxy. However, public proxies have their own unique advantages:

• Most public proxies are free, which means you can use them for free (free is good after all).
• Able to support HTTP and SOCKSv5 services.

Here we are familiar with the HTTP protocol, so what is SOCKSv5? We will talk about it later.

• Public proxies are useful for some SEOs if you want to collect data from the Internet.

Private Proxy

Dedicated private proxies are less common, but they do exist, especially when run directly on the client computer. A private proxy is a dedicated IP that can only be used by one client at a time. Private proxies also have some unique advantages over public proxies.

• Fast speed, because only a single client is using it.
• It is safe enough without having to worry about the risk of privacy being leaked.

However, whether it is a public proxy or a private proxy, they all have a common feature, that is, they can act as a client, accept response messages, and return response messages; they can also act as a server, accept client requests, and process client requests.

Therefore, proxy servers play different roles, which is completely based on the reference you choose.

As we all know, a gateway is a network hardware device or network node that is the entry and exit point of a network, as all data must pass through or communicate with a gateway before being routed. A gateway is designed to connect two different networks together, allowing users to communicate across multiple networks.

The gateway in the above description actually plays the role of a proxy. Its function is to help two heterogeneous networks communicate.

However, although a gateway can act as a proxy, a gateway and a proxy server are completely different things.

The biggest difference between a proxy and a gateway is that a gateway does not filter data and cannot block access to certain websites, while a proxy server has many functions.

It can be understood that a gateway is a proxy server without filtering capabilities.

Another network device that has to be mentioned here is the firewall. The firewall can also filter data and perform security checks.

Strictly speaking, a proxy connects two or more applications that use the same protocol. Depending on the different protocols, proxies can be divided into many types, but we generally use the following three types of proxies.

• HTTP proxy is the most common proxy method we see, mainly using proxy browsers to access web pages.
• SOCKS proxy, SOCKS proxy is Socket, it supports multiple protocols, including HTTP, FTP and other types of requests. It is divided into two types: SOCKS 4 and SOCKS 5. SOCKS 4 only supports TCP protocol while SOCKS 5 supports TCP/UDP protocol, and also supports various authentication mechanisms and other protocols.
• SSL proxy, SSL proxy is also called HTTPS proxy. In order to protect the security of sensitive data during Internet transmission, more and more websites are published in SSL encryption form.

At this point, some readers may ask, "cxuan, you have talked about so many things, but what exactly can a gateway do? Can you give some examples?"

Here it comes.

The role of a proxy server

Below I will explain to you the uses and functions of proxy servers through several examples:

Website filtering

This is what we have been talking about above. The proxy server can access some websites, and it also has filtering functions, prohibiting access to some websites.

Document access control

You can use a proxy server to implement unified access control between a large number of web servers and web resources, usually in large enterprises or distributed organizations. For example, the following are three clients with different access control permissions.

• Client A can access specified pages in Server A without restriction.
• Client B has direct and unrestricted access to the Internet.
• Client C needs to enter a password or credentials before accessing Server C's encrypted data.

Security Firewall

A proxy server can also act as a firewall to restrict/filter the inflow and outflow of data, perform security checks, etc.

Web Cache

A proxy cache maintains local copies of frequently used websites to reduce slow and expensive Internet communications.

Reverse Proxy

In addition to being able to pretend to be a client, a proxy can also pretend to be a server, which is called a reverse proxy. However, for the client, the reverse proxy server is equivalent to the target server, which means that the client can directly access the proxy server to directly obtain the resources of the target server.

You can use a reverse proxy to improve performance when accessing public content on a slow web server. In this configuration, these reverse proxies are often referred to as server acceleration.

Transcoder

The proxy server modifies the main format of the content before sending it to the client. This method of modifying the data format is called transcoding.

The transcoding agent can convert GIF images into JPEG images when transmitting them, which is used to reduce the transmission size of the images, and can also compress them.

Anonymous

As the name implies, an anonymous proxy server hides client features. An anonymous proxy removes identity features from HTTP messages, such as the client's IP address, From header, Referer header, cookies, and URI session ID to improve privacy and security.

In fact, proxies also have a hierarchical structure like DNS, except that in the DNS hierarchy, the upper and lower levels are all DNS servers, while in the proxy hierarchy, the upper and lower levels are replaced by proxy servers.

Hierarchy of Agents

In the proxy hierarchy, messages are passed between proxies until they reach the final server, and then the response message is passed back to the client through the proxy. For example, the following figure shows a reverse proxy hierarchy.

The proxy servers in the proxy hierarchy are given a parent and child relationship, the one close to the server is called the parent proxy and the one close to the client is called the child proxy.

This is a static proxy hierarchy. Static means that in this hierarchy, proxy 1 will always forward messages to proxy 2, and proxy 2 will always forward messages to proxy 3.

However, the proxy hierarchy does not have to be static. That is, in the proxy hierarchy, both the parent proxy and the child proxy can be changed. Here are a few ways to dynamically select proxies:

• Load balancing: A child agent can select a parent agent based on the current parent agent's workload level.
• Geographic proximity selection: Of course, you can also select a parent proxy based on geographical proximity.
• Select by protocol and type: The sub-agent will forward the message to different parent proxies or original servers based on the URI.

Client proxy settings

There are four main types of client proxy settings:

• Manual Setup
• Pre-configure the browser proxy. The browser manufacturer will pre-configure the proxy before the client obtains the browser
• Automatically configure the proxy by providing a URI pointing to a proxy automatic configuration file written in JS; the client will retrieve this JS file and run it to decide whether the proxy should be used
• WPAD proxy discovery. Some browsers support the WPAD proxy auto-discovery protocol, which automatically detects which configuration server the browser can download an automatic configuration file from.

Summarize

This article does not contain much content and is not very complicated. It is suitable for reading on the subway or after a meal. You can absorb the content of the article in about five minutes.

This article is reprinted from the WeChat public account "Programmer cxuan", which can be followed through the following QR code. To reprint this article, please contact the programmer cxuan public account.