Soul-searching Question 9: What are the unknown secrets of the bizarre online world? (Must-ask questions for online interviews)

You are more powerful than Zhuge Liang. You have a huge intelligence network. No matter where you go, you can receive real-time information about world events and make decisions.

You have an 800-mile express letter that is more powerful than the emperor's. You can chat with your ministers and share the palace life, happiness, and sadness in 1 second.

People all over the world are serving you. You can buy things, book a hotel, eat mantis shrimp from Lake Baikal, and send red envelopes to Xia Yuhe by Daming Lake without leaving home.

......

The Internet has changed our lives. It has also provided us with job opportunities and enabled millions of people to have a full stomach. So, have you ever thought about how such a legendary thing came to our eyes? What unknown secrets are there in it?

Now please join Little Bear in today’s exploration and discovery, and uncover the most mysterious and bizarre technology of the 21st century – the Internet!

1. What makes up a network?

This is a Shenzhen subway map.

Just like this picture, a network is composed of several nodes and links connecting these nodes, just like the Shenzhen subway map is a big network composed of various stations and the links between them.

All nodes can be computers, hubs, switches, routers and other devices, and all connections can be twisted pair cables, coaxial cables, optical fibers, etc.

All devices connected to a computer are called hosts or end systems. In addition to computers and mobile phones, there are also refrigerators, cars, shared bicycles, traffic lights, bracelets, surveillance cameras and other strange devices. Currently in 2020, at least 20 billion devices are connected to the Internet!

The Internet connects many computers together to form small networks, such as home networks, corporate networks, and mobile networks. The connection of many networks is called the Internet (note the capital I).

But it is not enough to connect everything. The information we send and receive will eventually become WeChat messages, games, emails, web pages, etc. In this process, where does the information come from and where does it go? What processing does the sending and receiving process go through? Why can we enjoy network services? Why are there so many colorful presentation forms? How can our data security be guaranteed?

Don't lie! Let's continue with Little Bear's talk about the Internet: Why can we enjoy Internet services?

2. Why can we enjoy network services?

Since 1993, government agencies are no longer responsible for the operation of the Internet. At this time, countless Internet Service Providers (ISPs) have emerged, such as China Telecom, China Unicom, and China Mobile, in order to provide Internet access to the general public.

ISPs apply for many IP addresses from Internet management agencies. Large ISPs build their own communication lines, while small ISPs rent their communication lines and various networking equipment.

As long as we pay the ISP to obtain the right to use the IP, we can access the Internet.

So the so-called Internet access means "accessing the Internet through an IP address obtained from an ISP"

The IP address management agency does not assign a single IP to a single user, but rents a batch of IPs to qualified ISPs for a fee. Therefore, the Internet is jointly owned by countless ISPs. This phenomenon is called "a network within a network".

Different ISPs are divided into different levels according to their size. The one with the widest coverage is the backbone ISP, which is usually at the national level. The second is the region, which is connected in parallel with one or more backbones. Finally, there is the local ISP, which can be directly connected to the backbone or to the region. This is what we use most (such as company intranet private cloud, school, and public cloud vendors).

Any ISP (except the top-level one) can be interconnected with multiple ISPs and become their subordinates, which is called multi-home. At this time, even if one of the providers has a problem, the network will not be disconnected.

Then the question arises: there are so many users but only a few backbone ISPs, and they must cover a huge amount of network data traffic. Even if they have the largest bandwidth and can complete the traffic exchange, it is an eternal truth that the longer the distance, the slower the transmission speed. If the traffic from the adjacent area is forwarded thousands of miles away and then forwarded back, it is a waste of network resources.

In order to avoid this waste, a more efficient and fast forwarding method has emerged, that is, Internet Exchange Point IXP (Internet Exchange Point)

IXP acts as a bridge. As shown in the figure above, the two regional ISPs on the right are connected through IXP, which is equivalent to "bypassing" the highest ISP. This part of the traffic does not need to pay the backbone ISP, and it also reduces network latency.

Since IXP can save money, letting others access IXP is obviously a money-making business. Everyone wants to make money, so who builds the IXP, who operates it, who maintains it, how to charge, and who makes the major ISPs willing to connect to it? All of these require a lot of skill.

For example, if there is a NAP/IXP point in Shenzhen, Shenzhen Mobile users can access the Shenzhen Telecom network without leaving the city. The fees are charged according to the bargaining price of NAP/IXP, which is much cheaper. From the perspective of the topology of the entire network, the traffic is indeed greatly optimized.

What is BAT doing at this time? Strictly speaking, BAT is not an ISP. They provide content, so they are called Internet Content Providers ICP (Internet Content Provider). Content providers have content, and their data centers need to be interconnected with the operator's network to be accessed by their users.

Generally, the dedicated line border gateway protocol BGP (Border Gateway Protocol) is pulled through. Simply put, you send a message to BAT, and they reply the message based on the principle of proximity. According to the source address you visit, they determine where the message comes from. If it is from China Telecom, it will go back through China Telecom's line. If it is from China Unicom, it will go back through China Unicom's line.

[[333739]]

In the past, Internet Data Centers (IDCs) all belonged to operators. In order to make their services run faster, BAT now builds their own data centers, that is, clouds, and provides excess computing resources to the outside world, putting computing on the cloud. This is cloud computing, and the cloud services of Alibaba and Tencent are particularly good.

If an IDC is also an IXP point, the interconnection of the network is the wiring between the corridors, which is not only low-cost but also more stable (no need to worry about the optical cable being dug up). The network quality of this IDC is better than that of an ordinary IDC, and is more valuable. This model is the current hot Internet profit model. IXPs can make no money, but IDCs can make a lot of money.

Combined with something called Software Defined Network (SDN), especially programmable routing, routing can be based on business needs, traffic demand, and how to save money. It can be used to play the role of IXP, which is cheaper than routers and can be scheduled on demand. It can support complex traffic engineering to achieve distributed denial of service (DDOS) and prevent attacks.

3. Why can computers in different countries understand each other?

At this point, you should be very clear about the question of why we can enjoy network services! In order for hosts to communicate with each other, they need people like Qin Shihuang to unify the language so that computers can understand the information transmitted by each other and accurately convert it into a language understandable to humans.

The unified communication method of the language is the protagonist protocol in this section.

For example, the above is a chat between Ma Chunhua's husband and his friend. One person starts the conversation and the other responds. The man in the picture above says "Is it about that?" This is a response, which implies that I am waiting for you to continue. Of course, he can also refuse to answer.

Asking, answering confirmation, or not getting an answer to an inquiry are all interactive processes based on human conversation protocols.

In our human conversation protocol, the query content is the message we send, and there are also responses to the message and other actions taken based on the event (for example, waiting for a period of time without any response)

If people use different protocols, for example, if they don’t understand each other’s language or the other person doesn’t want to talk, the interaction process cannot proceed.

Network protocols are similar to human protocols, except that the people involved in the conversation are replaced by certain hardware and software devices (such as computers, mobile phones, routers). They all use the same language (byte stream, which is eventually parsed into electrical signals) and are strictly restricted by the protocol.

• The hardware is directly connected through the network cable, and the bit flow between the network cards is controlled by the protocol
• The end system controls the packet sending rate during the packet sending and receiving process through the blocking control protocol
• Router protocols that control the path of packets from source to destination

Protocols are everywhere. The above diagram shows a simple description of a computer network communication protocol.

[[333742]]

So far we have revealed how the Internet transmits information and how it is composed. Let’s turn our attention back to our hands, pick up our phones, connect to wifi, and uncover the edge of the network.

4. Why can my computer, mobile phone, and iPad all connect to the Internet at home?

We have mentioned before that all computers and other devices (servers, mobile wearable devices, etc.) connected to the Internet are called end systems. End and edge both refer to the end of the network. Keep this concept in mind.

As for how they are connected, it is thanks to the three major technologies of light, electricity and waves. Didn’t you suddenly realize that what you learned about "Analog Electronics", "Digital Electronics" and "Digital Signal Processing" were not in vain?

An edge router is the first router on the path that connects the local end system to any remote end system (such as the commonly known "cat"), and the network that physically connects the end system to the edge router is called an access network.

The earliest access method was to use telephone lines and cables. The telephone company was the ISP, and the digital model was obtained through digital-to-analog conversion. The home was connected to the ISP through a twisted pair of copper wires (copper wires twisted into a braid).

Because these lines not only have to carry high-frequency network data transmission, but also low-frequency two-way telephone signal transmission, and the material is copper wire, the transmission rate is limited.

The reason why there is no impact between telephone information and network signals can be recalled by recalling the filter principles learned at that time.

In order to achieve faster speed, a hybrid fiber coaxial system HFC (Hybrid Fiber Coax) was developed. The cable is connected to the user's home, still using the local telephone infrastructure, and the optical cable is connected from the cable TV company.

With the development of fiber-to-the-home, the speed is much faster, but it is only available in densely populated communities. If you want to lay a single fiber optic cable without it, it will cost tens of thousands of yuan.

One of the fiber optic connection methods is called passive optical network PON (passive Optic Network), which connects the optical fibers of many households through the optical network terminator ONT (Optical Network Terminator), connects to the nearby distributor, converges into one optical fiber, and finally connects to the local central office (CO) (linking users to the communication operator's switching network) to achieve Internet access.

Of course, the above is all about how to connect out. There are usually many devices in the home, which are assembled into a local area network (LAN) and connected through a modem (DSL) or a switch.

But we can't always plug the phone into an Internet cable when we use it. In order to achieve the purpose of mobile Internet, we need some wireless connection methods, which are the well-known WiFi and cellular networks (2G, 3G, 4G, LET).

Okay, now we know how the end systems are connected, but how do the ends communicate with each other? And who is providing us with application services? It is definitely not an operator hiding in the middle, manually identifying the signal request and then sending the web page to you. If that were the case, 100 operators would not be enough to serve one person.

Some end systems are needed to act as servers. End systems are also called hosts and can be divided into two categories: clients and servers. Clients generally refer to personal computers, mobile phones, iPads and other devices, which generally refer to devices that can be touched and held in the hand and can enjoy services as long as they are connected to the Internet.

A server is a large computer or data center that stores and publishes web sites, videos, relays emails, etc. These data centers may contain hundreds of thousands of servers.

There are two well-known communication modes among them, client-server mode (C/S) and peer-to-peer mode (P2P).

The client-server method is the most commonly used. Our daily use of mobile phones to surf the Internet is the most typical presentation:

The client knows the address of the server and actively initiates communication with the server. The server can handle requests from multiple remote clients at the same time and never stops running.

The peer-to-peer mode means that two hosts may be both service requesters and service providers when communicating:

Image from Baidu Encyclopedia

A very famous network disk uses this technology. In order to achieve acceleration, after you download a file, if people around you want to download the same file, they will treat your computer as a server and transmit it nearby. If many people around you have downloaded this file, there will be many servers, downloading different parts of the file respectively, and finally compiling them into a complete file to achieve the purpose of acceleration.

In the fourth section, we learned how to access and communicate with edge networks. In connection with the previous sections, we know the various ways to connect to the network, and we also know that there are ISPs, IXPs, and ICPs in the middle, but we don’t know how a message sent by a client can reach the server accurately through thousands of mountains and rivers. In the next section, we will take a look at this process.

5. Why can Jerry Guo send a video in the United States, but I can receive it in Nanshan Science Park, Shenzhen, China?

This section discusses the network core, focusing on forwarding tables and routing protocols.

Don't worry about being blocked, I've censored it.

Jerry Guo is currently working as a war correspondent. I have been receiving a lot of his video push notifications on Weibo recently. When I calmed down and thought about it, why would I be able to see a message from him when he is so far away? Is it some special fate? No wonder they say that the Internet connects us, so we should cherish this fate.

Excluding the edge of the network is the core of the network, which is a mesh network consisting of a bunch of packet switches and links.

Of course, there are two other switching methods called circuit switching and message switching, but since packet switching works best, these two methods are rarely used.

In various network applications, end systems exchange messages with each other. The messages contain protocols and can transmit text as well as data, such as video and music.

When transmitting messages from the source system to the destination system, long messages are divided into fragments, called packets. Each packet is a collection of bytes (which can be understood as high and low levels such as 10101100), which are transmitted through communication links and packet switches (routers and link switches).

But! The router can only receive packets from one connected link and then forward it to another adjacent link. What does that mean? It means that there is absolutely no way to send information directly from the United States to Shenzhen.

This is because the storage space in the router is limited. It is impossible to store the locations of so many devices around the world. Even if it can be stored, if a hacker changes the content inside, he can disguise himself as anyone.

So clever programmers designed forwarding tables and routing protocols to map part of the destination address to the output link. If we compare the router to a person and the message to a letter, we can only send the letter by asking for directions!

Everyone only knows the approximate direction and the location of the closer router. Each time they ask for directions, they will be a little closer to the destination. After each time they ask for directions, they will know the location of the next closer person, and they will keep asking like this until the delivery is successful.

Each router has a forwarding table that is responsible for modifying the destination address (or part of the destination address) to the appropriate output link, and you are one step closer to your destination.

Of course, it is not a random modification. A series of algorithms will be used to select the best path, which can usually reach any location within 16 hops (16 times). This is the content of the routing protocol. If you are interested, you can check it out yourself.

You can log in to www.traceroute.org, select a source at random, enter your real IP or domain name, and see how it jumps.

I selected an AS 1221 node in the United States and Baidu's domain name, and it took 13 hops to reach the destination.

6. Why should the network be layered?

From what we have said above, we can see that the network is really too complicated. For complex systems to communicate, there needs to be a clear division of labor and each has its own responsibilities. How is it layered specifically?

Students who have worked in operation and maintenance, implementation, and web development may have heard of four-layer and seven-layer load balancing. In fact, the difference is nothing more than binding an IP or binding a domain name.

Image from Computer Network-Xie Xiren

From the seventh layer to the first layer, each corresponds to a different data packet (message) in a different processing method (protocol). Each message will go through the packaging process when it is transmitted, and will put on seven layers of clothes. When it is received and transmitted, it will go through the unpacking process, and take off seven layers of clothes.

Each layer will carry some key information for the corresponding device to identify. In order to ensure the smooth delivery of the output package, it will go through many devices and many times of unpacking and packing during the transmission process.

Look at the figure c on the right. The computer has a five-layer architecture. On the source device, an application layer message (HTTP message) is transmitted to the transport layer (also called the transmission layer). The transport layer puts a shell (TCP header) and attaches its own information (allowing the receiving end to read the application layer information and error check bit information to prevent tampering during transmission). The TCP and UDP protocols we are familiar with play a role in the transport layer.

Then it is passed to the network layer, which adds information such as the source and destination system addresses (such as IP addresses) (IP header) to let the receiving end know where the packet came from.

Then it is passed to the link layer, which will assemble the data packets into frames (framing), which is the long message divided into short message packets (packets) mentioned in the previous section. Each frame contains data and necessary verification information (synchronization information, address, verification information, etc.) (Ethernet header). The link layer will add a MAC (Media Access Contol) address and transmit a data frame from one node to another node on the same link.

Then it is passed to the physical layer, along with information like how many pins the plug of the connecting cable has and how to connect them.

After sending it out, the next physical device will unpack the packet. If it is a layer 2 switch, it will remove the link layer and the physical layer. If it is a router, it will also remove an additional network layer, and then add its own information and the information of the next hop and continue to pass it on.

The difference between a switch and a router is that a router can process TCP/IP protocol at the network layer and can assign one IP to multiple hosts, while a switch cannot. A switch uses MAC addressing at the link layer and can provide firewall functions, while a router cannot. Routers are mainly used to connect multiple networks, while switches are mainly used to connect more computers to a LAN.

Common home switches refer to Layer 2 switches. Now there are Layer 3 and above switches that can work based on the network layer or even the transport layer.

The entire layered transmission process is shown in the following figure:

Source: Computer Networks - Top-Down Approach

The session layer and presentation layer are not mentioned above. My personal understanding is that these two layers have never been implemented independently, and are always implemented together with the application layer. Most programmers program at the application layer. The application layer mainly adds session persistence and breakpoint download capabilities. The presentation layer mainly does data encryption, decryption, compression, and transcoding (Virtual Terminal Protocol VTP, which defines a unified character set, terminal commands, format control characters, etc.). For example, most hosts use ASCII code, and IBM hosts use EBCDIC encoding.

There is also a security encryption layer that many people have used, but no one has ever wanted to divide it into a hierarchical structure. Its name is SSL/TLS. With the services provided by the security layer, HTTP/SMTP/FTP at the application layer can add an S (Security) after their names, such as HTTPS. In fact, there is no HTTPS protocol in the world, only HTTP protocol. Adding the S suffix just tells everyone that HTTP uses a six-layer structure with the security protection of SSL/TLS.

7. Why can’t I use up the 100M broadband?

OK, I received Jerry Guo's video, but I don't like to play it directly. I want to download it all. But I found that my broadband is 100M, so why can't my Internet speed reach 100M/s? Almost everyone has this question.

100M bandwidth, but the actual speed is only a pitiful 12M/s, this megabyte is not that megabyte~

The normal speed of 100M broadband is 100 Mbit/s, but the unit is different. 8bit=1B. We are concerned about MB/s, so 100 Mbit/s = 100/8 = 12MB/S

8. Why does the game or video playback get stuck?

OK ~, the student standing in the back row was not convinced: "I agreed with you that the speed was 12M/s, but why is it only 1M/S now? How do you explain this, Xiao Xiong?"

I have to sigh, as long as the tricks are sophisticated and no one takes them seriously, as a user I always feel I can't beat the operators, and I always feel like I've spent money but can't enjoy the services. In fact, this is very likely to happen if you watch videos during peak hours.

So why does it get stuck during peak hours? Shouldn't this be the broadband that I enjoy exclusively? In fact, it is shared. Before explaining why it is shared, let me first explain the speed loss that occurs during the transmission process.

Let's go back to the most commonly used packet switching. As we said before, packet switching slices the message into packets for transmission. When transmitting to the packet switch, the switch itself will perform some packet inspection. The time wasted here is called node processing delay.

Most packet switches use a store-and-forward transmission mechanism at the input end of the link. Before the switch outputs the first bit of the packet, the entire packet must be received. This waiting time is called the store-and-forward delay.

After the packet switch receives the entire packet data, it must continue to transmit it to the target host. Each packet switch is connected to many links (hosts), and each link has an output queue (output queue, also called output cache).

If a packet needs to be transmitted to a host on a link, and this link is occupied by another packet transmission, then the packet must wait in the output queue. This waiting time is called queuing delay, which is similar to the time wasted waiting in line at a toll booth.

However, the buffer space of the output queue is limited. If it is overwhelmed, data loss will occur, which is also called packet loss, commonly known as packet loss.

The time wasted in the process of collecting money at the toll booth and then giving it to the card is called transmission delay, which is the time wasted at the moment when the packet switch sends out the message, and is generally very short.

After the data is transmitted, the time it wastes on the road before it reaches the next switch is called propagation delay.

During the entire process, the total time wasted by the end system passing through routers/switches one by one and finally reaching the destination end system is called end-to-end delay. You can also use the tracking tool provided in Section 5 to see the time wasted for each of the three handshakes. Isn’t it cool?

If a 100M bandwidth cable is fully utilized, its peak throughput is 12Mbps. Throughput is the amount of data successfully transmitted per unit time (measured in bits, bytes, packets, etc.).

Our home network is usually accessed through a local area network (LAN), which is actually a type of bandwidth sharing. Generally speaking, the entire community is a local area network, and if the total bandwidth outlet connected to the telecommunications backbone network is 100M, that is, if there are 100 people online at the same time in the community, then 100 people are sharing this 100M bandwidth. Therefore, the broadband access under shared bandwidth may be slower when there are more Internet users.

In communities where broadband is less popular, shared bandwidth can be very cost-effective because it is cheaper and no one is competing with you for bandwidth. However, in communities where broadband is more popular, you can use ADSL (full name Asymmetric Digital Subscriber Line). Each user's network terminal has a separate line connected to the ADSL local end, which can ensure that all bandwidth is exclusively used by each user, and the speed will not slow down even if the number of users increases sharply.

But now it is the 20s of the 21st century. ADSL uses telephone lines to access the Internet. Its maximum theoretical upstream rate can reach 1Mbps and downstream rate can reach 8Mbps. It has long been eliminated. It is the right way to use LAN and get a larger broadband. If you have cable conditions, you can directly use optical fiber. The market price I found online is about one thousand yuan a year for 100M optical fiber, but it depends on whether your community has it.

Here are two simple and direct methods to enhance WiFi signals:

• Buy a wifi amplifier. The signal amplifier receives the signal from the main router, amplifies it and then transmits it out, so the signal at home is still the same, and mobile phones and Pads will connect to the better quality signal.
• When decorating the house, bury an extra network cable, extending it from the main router to the place where the signal is poor, and then connect another router. The main and secondary routers are connected through their respective LAN ports. At this time, the secondary router is equivalent to a wireless switch.

Other methods will be discussed later.

9. Why is your computer attacked?

The current Internet has fully covered our daily work and life, including various government agencies, enterprises and schools. Countless wearable items and home smart devices are all connected to the Internet. In a dark and damp corner of the Internet, there is a group of people who focus on attacking and cracking, stealing user personal information, invading privacy, stealing government and bank information, and secretly withdrawing account funds. Therefore, while enjoying the convenience of interconnection, network security deserves our attention. Fortunately, there is a group of heroes who silently guard our network in all walks of life. They are Bai Ke (not Wang Dachui).

If you are chatting with your beloved girl online, you certainly don't want the chat content to be seen by a third person. This is the confidentiality of network security.

You can't see her face or hear her voice, but you need to make sure that the person you are chatting with is really her, and not someone intercepting the information and pretending to be her. This is endpoint identification technology.

Even if it is really him, you hope that the chat content will not be tampered with by the middleman and her meaning will not be misinterpreted. This is message integrity.

You hope that the chat software will not crash suddenly, and you also hope that the photos and bank card verification information on your phone will not be known to others due to receiving some strange files. When the chat is really being monitored and disguised, you also hope to discover it quickly or avoid these risks at the source. This is the significance of network security.

1. What are the targets of hackers?

• Hackers will attack our hardware: computers, network devices and peripherals. Computer targets are computer components, such as CPU and hard disk. Network devices may be devices such as routers, switches or hubs. Peripheral devices are all devices other than the host (such as cameras, printers, USB flash drives, etc.).
• They also attack our software: operating systems and applications. OS targets are the operating system itself (like Windows), while application targets are the various programs that run on the operating system.
• Attack our network, target the network itself or its protocols, paralyze the network, or deceive the source or destination network to cause identity errors, etc.

2. Let's take a look at their attack methods!

• Virus: A self-replicating program that spreads through some form of infected file.
• Worm: A self-replicating program that spreads without using an infected file; usually worms spread through network services on a computer or through e-mail.
• Trojan horse program: A program that disguises itself as a normal program but has some malicious purpose.
• Buffer overflow: A process that gains control of or crashes another process by overflowing its buffer
• Denial of Service Attack: An attack that prevents legitimate users from accessing or using a host or network.
• Network attack: An attack that attacks a network or users on a network by manipulating network protocols (from the data link layer to the application layer).
• Physical attacks: These attacks are very dangerous and are based on destroying the network or the physical parts of the computer, which may render the computer useless.
• Password attacks: Attacks designed to obtain passwords, such as keyloggers and phishing sites that pretend to be legitimate sites and trick users into entering their passwords
• Information gathering attacks: attacks in which no physical or digital damage is done, and no subversion occurs, but the attacker obtains important information that may be used in further attacks.

The above methods do not include some unknown methods.

3. How to prevent it?

• As long as there is an Internet connection, there may be attacks. As an ordinary person, you should have a good sense of network security. Don't set the same password for each platform, change the password regularly, and set a strong password.
• Be extra careful on websites that begin with http;
• Encrypt sensitive files stored locally;
• Do not open emails from strangers, especially attachments;
• Use trustworthy software, security software, clean up your privacy records in a timely manner, and set up a firewall.

If you have any other ways to prevent Internet safety, you can discuss with me in the comment section!

Summarize

This article takes you into the door of the Internet world through nine questions. After reading it, you should understand

Why can we connect to the Internet, the process of information transmission and reception, who provides us with Internet connection services and content services, network communication protocols and layers, various network devices in the Internet and their functions, common means of network security attack, etc.

Thank you for your reading and attention, see you next time.