What happens when you enter a URL in your browser (Part 4): Network packets are transmitted in a LAN - how routers and switches forward packets

In the previous section, we introduced how network packets are encapsulated by the IP module of the protocol stack and then sent out of the local machine through the network card.

What happens when you enter a URL in the browser (Part 3) IP module encapsulation, ARP protocol, IP protocol, ICMP protocol and network card principles

Next we will describe what the data goes through in the LAN after it is transmitted from the client host.

After the network packet is sent from the client host, it must pass through the hub, switch and router to finally enter the Internet. In our real life, the router at home has integrated the functions of the hub and switch. After the network packet reaches the router at home, it can immediately enter the Internet.

In order to understand the functions of hubs, switches and routers, each function is introduced separately for easy understanding.

As shown in the following figure:

1. The electrical signal reaches the hub

From the previous content, we know that the computer network card will convert the data frame encapsulated by the MAC module into an electrical signal (high and low levels composed of 0 and 1), and the electrical signal will reach the hub along the network cable connected to the computer.

A hub contains multiple sockets (interfaces), each of which may be connected to other hubs, switches, routers or multiple computers. Behind each interface is a module with the same function as the PHY (MUA) in the network card for receiving and sending electrical signals.

Each interface on the hub can both send and receive, but each interface cannot send and receive at the same time, or send and receive at the same time, because the hub is based on half-duplex working mode.

When the signal reaches the PHY (MUA) module behind the hub's receiving interface through the network cable, it will enter the relay circuit. The basic function of the relay circuit is to broadcast the input signal to all interfaces of the hub, and the signal will be sent from these interfaces to all devices connected to the hub.

PS: The hub will not parse and process the data sent. That is to say, when the hub receives the electrical signal, it will not restore the electrical signal to digital information (that is, it will not restore the electrical signal to data frames and data packets), but will directly forward the electrical signal.

2. Electrical signals from the hub to the switch

Like hubs, switches also have many interfaces or ports, and each interface is also equipped with a PHY (MUA) module for sending and receiving electrical signals. Unlike hubs, switches store a table of MAC addresses and network cable ports (MAC address table) inside. Moreover, switches are devices that include a data link layer and have a MAC module that can check and parse data frames, while hubs do not. Hubs do not convert electrical signals into data frames (but directly forward electrical signals), let alone parse data frames.

When the electrical signal is received by a port of the switch, the PHY (MUA) module converts the electrical signal into a data frame, and then the MAC module verifies whether the data in the data frame is correct (FCS check). If correct, the Ethernet packet in the frame is placed in the buffer.

PS: We can understand that each port of the switch is equivalent to a network card, with a MAC module and its own buffer. In fact, not every port has an independent PHY (MUA), MAC and memory, but a control chip controls multiple ports at the same time. In fact, there is no network card in the switch, so the switch does not have a MAC address.

The network card itself has a MAC address, so the network card can check whether the MAC address of the recipient of the received packet is its own MAC address. If not, the packet will be discarded. However, since the switch does not have a network card and a MAC address, the switch will receive all packets sent to it, store them in the buffer and forward them, and will not discard them (if the interface corresponding to the sender's MAC address is the same as the interface corresponding to the receiver's MAC, the packet will be discarded, which will be introduced later).

After the packet is placed in the buffer, the MAC module obtains the sender MAC address and the receiver MAC address from the MAC header of the Ethernet packet.

On the one hand, the switch will record the mapping relationship between the sender's MAC address and the port number of the corresponding receiving port of the switch in the MAC address table, so that the switch knows which port the device with the sender's MAC address is connected to. The next time another machine wants to send data to this MAC address, the switch will know which interface to send the data through.

On the other hand, the switch will query whether the receiving MAC address is in the sending port recorded in the MAC address table. For example, if the receiving MAC address is 00-02-B3-1C-9C-F9, it corresponds to the third row in the MAC address table below. The switch will send the packet out from port 8.

PS: If the switch receives a packet whose recipient MAC address is the MAC address of host A, then the port on the switch that receives the packet may not be directly connected to host A, but may be connected to a hub that helps A forward packets or other switches that help A forward packets.

From the above figure, we can also see that the switch ports are also divided into receiving ports and sending ports. The port number corresponding to the MAC address 00-02-B3-1C-9C-F9 is 8, so the sending port number and receiving port number of the switch for this MAC address are both 8, but the sending port and the output port are not the same port, but two different ports, but their port numbers are both 8. This involves the switching circuit of the switch. As shown in the following figure:

The left side is the input port of the switch, and the bottom side is the output port. Port No. 8 on the left is the receiving port of the switch for machine 00-02-B3-1C-9C-F9, and port No. 8 on the bottom is the sending port of the switch for machine 00-02-B3-1C-9C-F9.

MAC address table maintenance

As we just said, when receiving a packet, the switch will record the sender's MAC address and the port number of the receiving port into the MAC address table, and the switch will perform this operation every time it receives a packet. In this way, when receiving a packet sent to this address in the future, the switch can send it to the correct port.

However, the records in the address table are not permanently valid and will be automatically deleted after a period of non-use. This is to avoid situations where the records in the address table are no longer correct due to the movement of the device, such as a laptop being moved from an office to a conference room and connected to a different network cable.

The validity period of the address table record is a few minutes, but if a record is outdated during the validity period (the device is moved), the switch will still forward the packet to the old port, and communication errors will occur. At this time, just restart the switch, the address table will be cleared and updated with the correct information.

Let's look at a special case: If the switch queries the MAC address table and finds that the interface corresponding to the receiving MAC address in the record is the same as the sending interface corresponding to the sending MAC address, how will the switch handle it? The following figure shows how the switch handles it:

A sends a data packet to B. We find that A and B are connected to the same hub, and the hub will broadcast A's packet to all ports of the hub, including B. Therefore, after B receives the packet sent by the hub, it checks the MAC address and finds that it is its own MAC address, so B will respond with a reply packet, which will pass through the hub and then be broadcasted back to A.

However, the hub is also connected to the switch on the left, so the hub will also send this packet to the switch. When the switch checks the MAC address table, it will find that the sending port and the receiving port are the same port number. In order to prevent computer B from receiving duplicate packets, the switch will discard the packet instead of forwarding it back to the hub and then forwarding it to B.

Another special case is that the specified MAC address cannot be found in the MAC address table. This is because the device with the MAC address has never sent a packet to the switch, or the MAC record has been deleted due to expiration.

At this time, the switch will forward the packet to all ports except the source port. Some people may worry whether sending too many packets will cause network congestion. In fact, there is no need to worry, because as long as the target device responds (mainly ARP response here), the switch will record its MAC address in the address table according to the reply packet, and there is no need to broadcast and forward next time.

In addition, if the receiving MAC address is a broadcast address then the switch will also send the packet to other ports besides the source port.

The switch works in full-duplex mode, which is the difference between a switch and a hub. The sending and receiving ports of a switch are independent of each other. The sending port can only send but not receive, and the receiving port is also the same; but when the sending port is sending, the receiving port can also receive, which is the characteristic of full-duplex mode. The single port of the hub can receive and send, but receiving and sending cannot be performed at the same time, otherwise a signal collision will occur. Therefore, when the hub receives a packet, it needs to forward this packet to all other ports, resulting in other ports being unable to receive packets during this period. In terms of forwarding capability, the hub is lower than the switch.

3. Network packets are forwarded through routers

The network packet is forwarded by the hub and switch before reaching the router and being forwarded to the next router.

Of course, network packets do not necessarily have to pass through a hub, then a switch, and then a router. It is also possible that the hub is connected to the router, and the packet can be forwarded to the router after reaching the hub. It is also possible that the computer is directly connected to the router, and the network packet reaches the router directly after being sent from the computer. In a local area network, there are also many hubs and switches between routers. From the current router to the next-hop router, it is necessary to pass through the hub and switch between the two routers. It is also possible that the router itself has integrated the functions of the hub and switch, so that the packet can be directly forwarded to the next-hop router according to the forwarding method mentioned above.

Of course, the focus of this section is not hubs and switches, but the forwarding mechanism of routers. Unlike switches, routers are designed based on IP (with network layer and link layer), while switches are designed based on Ethernet (only with link layer). Therefore, routers can parse MAC packets and IP packets, while switches can only parse MAC packets; switches determine the forwarding target based on the MAC address of the recipient in the packet, while routers determine the forwarding target based on the IP address of the recipient in the packet.

1. Simplified diagram of router structure

The router consists of two parts: forwarding module and port module.

The forwarding module is responsible for determining which next-hop router the packet should be forwarded to based on the destination IP in the packet; the port module is responsible for executing the packet sending and receiving operations.

2. Port module

The port module includes a PHY (MUA) module (which can receive electrical signals and convert frames into digital information), a MAC module (which parses frames into Ethernet packets) and a memory buffer (which temporarily stores network data). The port module is to the router what a network card is to a computer. Each port of the router has an independent MAC address and IP address.

Different types of routers have different port modules installed, such as routers with ADSL components (generally used as routers in the Internet), routers with FTTH components (routers in the Internet), routers supporting dedicated lines (routers in company LANs), and routers supporting wireless LANs or Ethernet (routers in LANs).

Routers with different port modules support different communication functions. For example, a router with ADSL communication function will add a MAC header, PPPOE header and PPP header (for transmission and authentication on the Internet) to the packet before forwarding it and send it to the Internet; while a router with Ethernet or wireless LAN function will only add a MAC header when forwarding and only supports packet transmission and parsing within the LAN.

Generally speaking, the computer's network card supports Ethernet and wireless LAN communication technologies, and the router's port module not only supports Ethernet and wireless LAN but also supports communications such as ADSL, FTTH and various broadband dedicated lines. The port module only needs to be installed with hardware that supports these technologies.

A router can have multiple ports and multiple types of ports (which means that the router supports packet forwarding both in the LAN and on the Internet). Each port may be connected to a network cable (if the port type is wireless, no network cable is required). Different port types require different types of network cables.

3. Forwarding module

There is an important table in the router's forwarding module called the routing table, as shown below:

The contents of the routing table in the router and the matching rules are the same as the routing table in the computer introduced earlier.

Among them, the first column "Destination Address" records the information of the recipient, but it is not the IP of a specific device, but an IP segment with only the network number part of the IP, indicating that the bit values ​​of the host number part are all 0.

The second column indicates the subnet mask of the target address, which is used by the router to determine the number of bits of the network number that need to be compared.

When the router compares the routing table and the recipient IP, it only compares the network number of the recipient IP and ignores the host number.

4. Route aggregation

The records in the first and second columns of the routing table may not be the actual subnet of the target address IP. This is the result of route aggregation. The so-called route aggregation is to merge multiple subnets into one subnet and generate a record in the routing table, which can reduce the number of records in the routing table.

For example, there are three subnets: 10.10.1.0/24, 10.10.2.0/24, and 10.10.3.0/24. Router B needs to forward the packet whose receiving IP is 10.10.2.100, but in fact, Router B's routing table does not have the records of the above three subnets. Instead, it merges the above three subnets into a subnet record of 10.10.0.0/16 (the address recorded here is the aggregated address), and points the next hop to Router A. As shown below:

At this time, router B only needs to match the first 16 bits of the target address 10.10.2.100, that is, 10.10. After the match is correct, B will forward the packet to router A.

Router A's routing table records the above three subnets. According to 10.10.2.100, it matches the subnet 10.10.2.0/24 and the next hop IP address, as shown below:

Route aggregation is to aggregate multiple subnets into one subnet, and the routing table records the aggregated subnet address. Conversely, the router may also subdivide a subnet and register it in the routing table to form multiple records. You can even write the address of a specific computer into the route. In this case, the subnet mask in the second column is 255.255.255.255, which means that all 32 bits need to be matched when matching the routing table.

As for the gateway, interface and hop count, we have already introduced the routing table of the computer above, so we will not repeat them here. The fourth column interface represents a network card in the computer and one of the many ports in the router.

PS: The maintenance method of the routing table is different from that of the switch. The switch maintains the MAC address table by recording the MAC address of the sender's Ethernet packet and the port number of the switch receiving port, and recording the MAC header of the receiver's response Ethernet packet and the receiving port of the switch receiving the packet. The maintenance of the routing table is achieved through manual maintenance by people and the information exchange between routers according to the routing protocol.

So back to the topic, when the packet arrives at the router, how does the router handle it? This will vary depending on the port of the router (such as Ethernet port, wireless LAN port, ADSL, etc.). Here we only introduce how the router Ethernet port receives and forwards packets.

First, the signal reaches the network cable interface, where the PHY (MUA) module and MAC module convert the electrical signal into digital information, check the FCS at the end of the frame, and see if the recipient's MAC address is sent to itself, and put it into the receiving buffer; if it is not a packet sent to itself, it will be discarded (the switch does not have a MAC address and is only responsible for forwarding. Therefore, the switch will not discard Ethernet packets because it verifies that the MAC address is not its own MAC address, but will forward them according to the MAC address table).

After verifying the MAC address, the task of the MAC header is completed and then discarded (the task of the MAC header is to send the packet to the router with the specified MAC address. Therefore, in the previous chapter, we introduced broadcasting to the LAN through the ARP protocol to obtain the MAC address of the recipient. In fact, it is to obtain the MAC address of the router rather than the MAC address of the remote target host). Then the router will match the record of the routing table according to the recipient IP address in the IP packet header to find the IP of the next-hop router (that is, the gateway column in the routing table) and the output port (the interface column of the routing table, that is, which port of this router sends this packet). It should be noted that if the gateway column of the matched record is a specific IP address, then this address is the next forwarding target. If the gateway column is empty, the recipient IP address in the IP header is the next forwarding target.

If the corresponding record cannot be matched in the routing table, the router will discard the packet and inform the sender through an ICMP message.

Before sending this packet, the router needs to re-encapsulate the MAC header because the previous MAC header has been discarded. At this time, the sender MAC address filled in the MAC header is not the MAC address of the source sender, but the MAC address of a sending port of this router (a router has multiple ports, each with a MAC address). The MAC address of the receiver is obtained through ARP query based on the IP of the next forwarding target (which may be the IP of the next-hop router or the IP of the target host). Of course, there is also an ARP cache in the router, so it will be queried in the ARP cache first.

After the network packet is encapsulated, the digital information will be converted into electrical signals and sent out at the port module. This process is the same as the packet sending process on a computer. If the output port is Ethernet type, the network packet will pass through the switch to the next router and be forwarded by the router layer by layer until the network packet reaches the final destination (of course, the introduction of how the packet enters the Internet from the LAN is omitted here, which will be introduced in the next chapter).

It should be noted that if the router sends a packet through an Ethernet port, the header of the Ethernet packet will only have a MAC header. If it is sent through an ADSL port (sending the packet to the network service provider and the Internet), the Ethernet packet will contain MAC, PPPoE and PPP headers.

5. Default Routing

For routers in a local area network, there may not be many routers and computers in a small local area network, so all forwarding targets can be written into the routing table. However, if it is a router in the Internet, due to the large number of devices in the Internet, it is impossible to record the network numbers of all target IPs in the routing table. Therefore, the routing table will configure the default route record in the last row. The target address column and subnet mask column of this row are 0.0.0.0. The subnet mask of 0.0.0.0 means that the number of bits of the network packet receiver IP that needs to be matched is 0, that is, no match is required. The gateway column of this record is the default gateway, which is generally filled in with the address of the router connected to the Internet.

In other words, if the receiving IP in the packet cannot match any other records in the routing table, and there is a default gateway record of 0.0.0.0 in the routing table, the receiving IP will match this record and forward it to the default gateway.

Of course, if the default gateway is not set in the router, then when all records in the routing table fail to match, the router will send an ICMP message to the sender.

6. Validity period of the package

Before a router forwards a packet, its forwarding module will update the TTL field in the IP header. This field indicates the validity period, and the value will decrease by 1 after each router forwards it. When the value becomes 0, it means that the validity period has expired and the packet will be discarded.

This mechanism is to prevent packets from falling into an infinite loop during forwarding. This situation will not occur if the routing table is configured correctly. However, it will occur if there is a problem with the information or a device failure causes temporary routing confusion by switching to a backup router.

TTL is usually set to 64 or 128.

7. Split large network packets through sharding

The port type of the router is not limited to Ethernet, it can also support other LAN or dedicated line communication technologies. The maximum packet length (MTU) that can be transmitted by lines of different port types is also different. For example, in Ethernet, the MTU of the Ethernet packet (including the Ethernet header) is 1500.

When the input port and output port of the router are of different port types (for example, the input port and output port of the router in the LAN may both be Ethernet type ports and lines, but the input port of the router connected to the Internet is an Ethernet type port and the output port is an ADSL type port), and the MTU that the output port can transmit is smaller than the input port, once the size of the incoming packet exceeds the MTU of the output port, the packet will not be sent. As shown below

Sometimes, even if the MTU of the input and output ports are the same, the packet length may exceed the MTU of the output port due to the addition of additional header data after the packet is re-encapsulated. For example, port types such as ADSL and FTTH need to add a PPP header and a PPPoE header before the IP header and then a MAC header before sending the packet.

In order to avoid the situation where the packet exceeds the MTU of the output port and cannot be sent, the IP module in the router provides a fragmentation function. First, it will check whether the packet to be sent can be sent directly, and compare the MTU of the output port with the length of the packet (the packet length of the data frame, excluding the frame header and FCS). If the MTU of the output port is smaller than the length of the packet, the packet will be fragmented. Before fragmentation, it will check whether the flag field in the IP header allows fragmentation; if the flag field indicates that fragmentation is not possible, the router will discard the packet and send an ICMP message to the sender.

The sharding is shown in the following figure:

The blue part in (a) is the split content.

Each fragment will add the same MAC header and IP header (if it is transmitted on Ethernet, these two headers will be added, if it is transmitted via ADSL, PPP and PPPoE headers will also be required).

It should be noted that the fragmentation of IP packets here is different from the fragmentation of HTTP messages by the TCP module introduced in the protocol stack. The TCP module fragments HTTP messages before the data is put into the packet, and each fragment contains a TCP header. The router directly fragments the IP packet, and each fragment has an IP header, but all fragments have only one TCP header. TCP fragmentation is also because the data to be sent may be larger than the MTU.

If a client has a 1M http message to send, this 1M content will be divided into hundreds of fragment packets (IP packets) in the TCP module, but when passing through the router, each IP packet may be further fragmented into two small packets by the router's IP module.

8. The relationship between routers and switches and between routers

In simple terms, the router is responsible for the entire process of delivering the packet to the communication object by continuously passing the packet to the next hop router (this is the responsibility of the network layer, such as the IP module). When the packet reaches the next router from one router, the switch is responsible for broadcasting and obtaining the MAC address (this is the responsibility of the link layer, such as the Ethernet line, etc.).

In addition to Ethernet, there are many other types of networks such as wireless LAN, ADSL and FTTH. If the current router and the next hop router are connected via Ethernet, the IP module of the current router will entrust Ethernet to pass the packet to the next router; if the current router and the next hop router are connected via ADSL, the ADSL line will be entrusted to pass the packet. IP itself is not responsible for packet transmission, but entrusts various communication technologies at the link layer (Ethernet, ADSL, FTTH, wireless LAN, etc.) to pass the packet to the next router.

Of course, now basically many routers have built-in switch functions, so the router itself has the function of broadcasting and performing ARP query MAC address.

4. Additional functions of the router

Above we introduced the basic functions of the router, which is to query the routing table according to the recipient IP in the network packet header, obtain the next hop address, and transmit the packet to the computer at the target address in a relay manner.

In addition to this basic function, routers have some additional functions. Here are two more important functions: address translation and filtering.

1. Address Translation

We know that a device has only one unique IP address. If there are devices with duplicate IP addresses in the network, network packets will not be transmitted normally. However, as the number of Internet devices increases, IP addresses will be allocated quickly. In order to solve this problem, we allow machines in different independent intranets to have duplicate IP addresses while machines in the same intranet do not have duplicate IP addresses, as long as the devices in different intranets do not communicate directly. For this reason, we proposed the concept of private addresses and public addresses.

The unallocated part of the public address that can only be used in the intranet is called a private address, while the fixed address allocated on the Internet is called a public address.

The private address ranges are as follows:

• 10.0.0.0 ~ 10.255.255.255
• 172.16.0.0 ~ 172.31.255.255
• 192.168.0.0 ~ 192.168.255.255

It does not matter if two machines on different intranets have duplicate IP addresses in the above range.

Let's take the company's intranet as an example. Generally speaking, the devices in the company's intranet are divided into two parts: one is the server open to the Internet, and the other is the company's internal devices. The devices open to the Internet are assigned public addresses, and the internal devices are assigned private addresses. The devices assigned private addresses can only communicate directly with the devices in the intranet and cannot directly send and receive network packets with the Internet.

If devices in the intranet want to communicate with devices on the Internet, a special mechanism is required, which is address translation.

The principle of address translation is to rely on the router to rewrite the IP address and port in the IP packet header when forwarding network packets.

When a device on the intranet wants to send a network packet to a remote target machine, when the network packet passes through the router, the router will replace the sender's IP address (private address) in the IP packet header with the address of the router's receiving port (the port address of the router is a public address), and will also rewrite the port number in the packet's TCP header with an idle port number assigned by the router.

The private address and port number before rewriting and the public address and port number after rewriting will be recorded in a table of the router accordingly.

The replaced public address is actually the IP address of the address translation device, that is, the IP address of the router. So the real sender is not the computer in the intranet, but the router.

The receiving IP of the server's reply packet will be written with the public address of the sender, that is, 198.18.8.31, which is the IP of the router. When the packet reaches the company's gateway router, the router will look up the mapping table above to find the corresponding intranet device based on the receiving IP and port of the reply packet and forward it to the device (there is only one public IP in the table above, but in reality there may be multiple public addresses because the router has multiple ports).

When there is only one public address, different port numbers can be used to distinguish different terminals in the intranet, so that the server's reply packet will not be unable to find the correct intranet device.

A record in this mapping table of public addresses and private addresses will be generated when an intranet computer A establishes a connection with an external server B (when the TCP packet of the connection request sent by A arrives at the router). After that, the communication between the intranet computer A and the external device B will use this conversion record until A and B complete the communication, disconnect, and A closes the socket used to communicate with B (at this time the port number of the private address becomes invalid) and this record will be deleted.

We found that the router also rewrites the port number when doing address translation. This is done to make a port number of the public address correspond to a private IP address during address translation, thereby improving the utilization rate of the public address. If the port number is not rewritten, it means that a new public address is required for each conversion of a private address. If the company's LAN is only allocated 10 public addresses, but the company has 1,000 devices (1,000 private addresses), it means that only 10 internal devices can communicate with the Internet at the same time.

Above we introduced the router address translation when the intranet device acts as the client and the remote device on the Internet acts as the server. Now let's talk about the router address translation when the company's intranet device acts as the server and the company's intranet device is accessed from the Internet.

In fact, the basic principle is to map and convert multiple private addresses by reusing multiple port numbers of one (or more) public addresses. The only difference is that when the intranet device is a client request, the record of the address conversion table is generated when the device sends a request and passes through the router (i.e. dynamically generated); and when the intranet device is a server for Internet requests, the address conversion table record in the router needs to be manually added in advance, and as long as the intranet device does not stop providing services, the record in it will not be deleted. As shown in the figure:

In this way, when a user on the Internet accesses 198.18.8.31:8080, he is actually accessing the service provided by port 80 of the machine 10.10.1.1 in the company's intranet.

2. Packet filtering function of router

Packet filtering is to decide whether to forward or discard a packet based on the rules set in the MAC header, IP header and TCP header according to the time limit when forwarding the packet. The firewall devices and software we often talk about use this mechanism.

Although this principle is simple, it is not easy to control flexibly. For example, how to prevent packets from the Internet from entering the intranet through the router, but allow packets sent from the intranet to be forwarded to the Internet through the router.

Many friends would think that if they want the intranet to access the Internet but not the Internet to access the intranet, they only need to allow the router to forward packets from the intranet to the Internet and prohibit forwarding packets from the Internet to the intranet.

However, network packets are transmitted in both directions. For example, in the TCP response mechanism, when the intranet device sends a network packet with a sequence number, the server will send back an ACK packet. At this time, the ACK packet is intercepted, resulting in the inability to respond normally to the request initiated by the intranet to the client (it should be said that the client's request packet cannot be transmitted normally, and the client host will keep waiting for the server's response packet).

This will be covered when exploring the server.