Byte One Follow-up: Do you know the TCP/IP four-layer model? What protocols are there in each layer?

This article is reprinted from the WeChat public account "CS Guide", the author is Dabai. Please contact the CS Guide public account for reprinting this article.

Hello everyone, I am Dabai.

This is the sister series of "Interview Site" - Operating System - Computer Network. In this series, I will explain a specific knowledge point to you through a real interview.

Below is the main text.

1. Content Review

Interviewer: Last time we talked about network layering. Today we will extend this a little bit and talk about what each layer does specifically.

Dabai: Okay, okay!

Interviewer: Before we begin, let’s review the TCP/IP 4-layer model.

• Application Layer
• Transport Layer
• Network Layer
• Network Interface Layer

TCP-IP-4-model

2 Application Layer

Interviewer: We usually recommend learning from the application layer to the network interface layer. Can you first tell me what the application layer does?

Dabai: The application layer is located above the transport layer. It mainly provides information exchange services between applications on two terminal devices. It defines the format of information exchange, and the message will be handed over to the next layer, the transport layer, for transmission. We call the data unit of the application layer exchange a message.

The application layer protocol defines the network communication rules. Different network applications require different application layer protocols. There are many application layer protocols on the Internet, such as the HTTP protocol that supports Web applications, the SMTP protocol that supports email, and so on.

network-layer

HTTP: Hypertext Transfer Protocol

Interviewer: Then let’s talk about the HTTP protocol first!

Dabai: HyperText Transfer Protocol (HTTP) is mainly designed for communication between Web browsers and Web servers.

When we use a browser to browse the web, our web page is loaded through HTTP requests. The whole process is shown in the figure below.

HTTP Request Process

https://www.seobility.net/en/wiki/HTTP_headers

HTTP protocol is based on TCP protocol. Before sending HTTP request, TCP connection must be established first, which means three handshakes are required. Most of the HTTP protocols currently used are 1.1. In the 1.1 protocol, Keep-Alive is enabled by default, so that the established connection can be reused in multiple requests.

In addition, the HTTP protocol is a "stateless" protocol, which cannot record the status of the client user. Generally, we use Session to record the status of the client user. How to determine whether the mailbox really exists?

SMTP: Simple Mail Transfer (Send) Protocol

Interviewer: Do you know the SMTP protocol?

Dabai: SMTP is the abbreviation of Simple Mail Transfer Protocol, which is based on TCP protocol and is used to send emails.

Note: The protocol for receiving emails is not SMTP but POP3.

The SMTP protocol involves a lot of content, and the following two issues are more important:

• Email sending process
• How to determine whether the mailbox really exists?

How to send email?

For example, my email address is "[email protected]", and I want to send an email to "[email protected]". The whole process can be simply divided into the following steps:

• Through the SMTP protocol, I send the email I wrote to the 163 mailbox server (post office).
• 163 The email server found that the email I sent to was from QQ, and then it used the SMTP protocol to forward my email to the QQ email server.
• After receiving the email, the QQ email server notifies the user with the email address "[email protected]" to receive the email, and then the user retrieves the email through the POP3/IMAP protocol.

The process of sending an email

https://www.campaignmonitor.com/resources/knowledge-base/what-is-the-code-that-makes-bcc-or-cc-operate-in-an-email/

How to determine whether the mailbox really exists?

In many scenarios (such as email marketing), we need to determine whether the email address we want to send to really exists. At this time, we can use the SMTP protocol to detect:

• Find the SMTP server address corresponding to the email domain name
• Try to establish a connection with the server
• After the connection is successful, try to send an email to the mailbox that needs verification
• Determine the authenticity of the email address based on the returned result

Recommend several online mailbox validity detection tools:

• https://verify-email.org/
• http://tool.chacuo.net/mailverify
• https://www.emailcamel.com/

POP3/IMAP: email receiving protocol

Interviewer: Do you know the POP3/IMAP protocol?

Dabai: He said a lot of things...

Interviewer: There is no need to elaborate on these two protocols. You just need to understand that both POP3 and IMAP are protocols responsible for receiving emails. In addition, you need to be careful not to confuse these two with the SMTP protocol. The SMTP protocol is only responsible for sending emails, and the protocol that is actually responsible for receiving emails is POP3/IMAP.

IMAP is a newer protocol than POP3 and provides more optional features for users. Almost all modern email clients and servers support IMAP. Most webmail service providers support both POP3 and IMAP.

FTP: File Transfer Protocol

Interviewer: What protocol is used for application layer file transfer?

Dabai: File Transfer Protocol FTP (File Transfer Protocol)

Interviewer: Could you please give us a brief introduction?

Dabai: The FTP protocol mainly provides file transfer services and implements reliable transmission based on TCP. The advantage of using FTP to transfer files is that it can shield the operating system and file storage method.

FTP is designed based on the client-server (C/S) model, establishing two connections between the client and the FTP server. If we want to develop a file transfer software based on the FTP protocol, we first need to understand the principle of FTP. Many books have described the principle of FTP in great detail:

The unique advantage of FTP and the biggest difference from other client-server programs is that it uses two TCP connections between the two communicating hosts (other client-server applications generally have only one TCP connection):

• Control connection: used to transmit control information (commands and responses)
• Data connection: used for data transmission;

This idea of ​​transmitting commands and data separately greatly improves the efficiency of FTP.

FTP working process

DNS: Domain Name System

Interviewer: How is a domain name resolved to an IP address?

Dabai: The Domain Name System (DNS) converts human-readable domain names (e.g., www.baidu.com) into machine-readable IP addresses (e.g., 220.181.38.148). We can think of it as a phone book designed specifically for the Internet.

dns-overview

An IP address is the communication address of a network card in the network world. We can think of it as our home address in the real world.

What is the complete process of DNS resolution? (This will be described in detail in the following article)

The complete process of DNS resolution

https://aws.amazon.com/cn/route53/what-is-dns/

Telnet: Remote Login Protocol

Interviewer: Do you know the remote login protocol?

Dabai: The remote login protocol, logging into other servers through a terminal, is built on the reliable transmission protocol TCP. One of the biggest disadvantages of the Telnet protocol is that all data (including usernames and passwords) are sent in plain text, which is a potential security risk. This is the main reason why Telnet is rarely used today and has been replaced by a very secure protocol called SSH.

SSH: secure network transmission protocol

Dabai: SSH (Secure Shell) is currently the most reliable protocol that provides security for remote login sessions and other network services. The use of the SSH protocol can effectively prevent information leakage during remote management. SSH is built on the reliable transmission protocol TCP.

The main difference between Telnet and SSH is that the SSH protocol encrypts the transmitted data to ensure data security.

3Transport layer

Interviewer: Okay! It seems you have spent a lot of time preparing! Let’s take a look at the transport layer!

Interviewer: Can you first tell me what the transport layer does?

Dabai: The main task of the transport layer is to provide universal data transmission services for the communication between two terminal device processes. The application process uses this service to transmit application layer messages. "Universal" means that it is not targeted at a specific network application, but multiple applications can use the same transport layer service.

The transport layer mainly uses the following two protocols:

• Transmission Control Protocol TCP (Transmisson Control Protocol)--provides connection-oriented, reliable data transmission services.
• User Datagram Protocol UDP (User Datagram Protocol) - provides connectionless, best-effort data transmission service (reliability of data transmission is not guaranteed).

transport-layer

Dabai: We will introduce TCP and UDP in detail in the following articles.

4Network layer

Interviewer: Good job! You still have to keep TCP and UDP in your hands, right?

Dabai: No rush! Let’s finish talking about the network layer first (taking the initiative).

Dabai: The network layer is responsible for providing communication services for different hosts on the packet switching network. When sending data, the network layer encapsulates the segments or user datagrams generated by the transport layer into packets and packages for transmission. In the TCP/IP architecture, since the network layer uses the IP protocol, the packet is also called IP datagram, or datagram for short.

Note: Do not confuse the "user datagram UDP" of the transport layer with the "IP datagram" of the network layer.

Another task of the network layer is to select a suitable route so that the branches passed down by the transport layer of the source host can find the destination host through the router in the network layer.

It is emphasized here that the word "network" in the network layer is no longer the specific network we usually talk about, but refers to the name of the third layer in the computer network architecture model.

The Internet is composed of a large number of heterogeneous networks connected to each other through routers. The network layer protocol used by the Internet is the connectionless Internet Protocol and many routing protocols, so the network layer of the Internet is also called the Internet layer or IP layer.

nerwork-layer

Interviewer: Let me pick out a few important protocols and ask you to introduce them to me! Dabai.

IP: Internet Protocol

Interviewer: Do you know IP protocol?

Dabai: Internet Protocol (IP) is one of the most important protocols in TCP/IP protocol and one of the most important protocols in the network layer. The functions of IP protocol include addressing protocol, defining the format of data packets, etc. It is the main protocol for information transmission in the network layer. Currently, IP protocol is mainly divided into two types, one is the old IPv4 and the other is the newer IPv6. Both protocols are currently in use, but the latter has been proposed to replace the former.

Dabai: We will introduce the detailed introduction of IP protocol in the following articles.

ARP: Address Resolution Protocol

Interviewer: What about ARP protocol?

Dabai: ARP protocol, full name Address Resolution Protocol, solves the conversion problem between network layer address and link layer address. Because an IP datagram always needs to know where the next hop (physical next destination) should go during physical transmission, but IP address belongs to logical address, while MAC address is physical address. ARP protocol solves some problems of IP address conversion to MAC address.

Dabai: We will introduce the detailed introduction of ARP protocol in the following articles.

NAT: Network Address Translation Protocol

Interviewer: Oh! That’s something! Let’s talk more about the NAT protocol!

Dabai: The application scenario of NAT protocol (Network Address Translation) is just like its name - Network Address Translation, which is used in the address translation process from the internal network to the external network. Specifically, in a small subnet (local area network, LAN), each host uses the same IP address under the LAN, but outside the LAN, in the wide area network (WAN), a unified IP address is required to identify the location of the LAN on the entire Internet.

Dabai: We will introduce the detailed introduction of NAT protocol in the following articles.

5Network interface layer

Interviewer: What about the network interface layer?

Dabai: We can regard the network interface layer as a combination of the data link layer and the physical layer.

• The data link layer is usually referred to as the link layer (data transmission between two hosts is always transmitted on a link segment by segment). The function of the data link layer is to assemble the IP datagrams handed down by the network layer into frames and transmit the frames on the link between two adjacent nodes. Each frame includes data and necessary control information (such as synchronization information, address information, error control, etc.).
• The function of the physical layer is to achieve transparent transmission of bit streams between adjacent computer nodes, shielding the differences in specific transmission media and physical devices as much as possible.

network-interface-layer

Finally, I would like to share a very good summary picture of the OSI seven-layer model!

https://wsgzao.github.io/post/osi/

6 Conclusion

Below is a brief summary of the protocols and core technologies included in each layer:

network-protocol-overview

Application layer protocol:

• HTTP protocol (Hypertext Transfer Protocol, a commonly used protocol for web browsing)
• DHCP protocol (Dynamic Host Configuration)
• DNS System Principles (Domain Name System)
• FTP protocol (File Transfer Protocol)
• Telnet protocol (remote login protocol)
• Email protocols, etc. (SMTP, POP3, IMAP)
• ......

Transport layer protocol:

• TCP
  • Segment Structure
  • Reliable data transmission
  • Flow Control
  • Congestion Control
• UDP protocol
• Segment Structure
• RDT (Reliable Data Transfer Protocol)

Network layer protocol:

• IP protocol (the basis of TCP/IP protocol, divided into IPv4 and IPv6)
• ARP protocol (Address Resolution Protocol, used to resolve the mapping between IP addresses and MAC addresses)
• ICMP protocol (control message protocol, used to send control messages)
• NAT Protocol (Network Address Translation Protocol)
• RIP protocol, OSPF protocol, BGP protocol (routing protocol)
• ......

Network interface layer:

• Error Detection Technology
• Multiple access protocol (channel multiplexing technology)
• CSMA/CD Protocol
• MAC Protocol
• Ethernet Technology
• ......