How does Baidu Netdisk steal your traffic?

Recently, Baidu has once again been at the center of controversy.

The reason this time is that Baidu Netdisk launched a "user incentive plan" to speed up the transmission speed of the netdisk by sharing users' idle bandwidth and computer storage space.

Of course, Baidu's plan rewards users who share bandwidth and hard drives with points, which can be used to redeem iQiyi VIP monthly cards, fast download coupons and other prizes.

If this activity is carried out openly, it would be fine, as long as the user is willing to take the bait. However, Baidu is cunning enough to not inform users clearly, and the user is allowed to tick the box by default. This is equivalent to making the choice for all users.

After this outrageous operation was discovered, Baidu Netdisk quickly apologized and released a patch, canceling the "user incentive plan", but avoided the question of whether it would continue to occupy users' bandwidth and hard drives.

Netizens commented on this statement: "To be frank, I will continue to use your bandwidth and hard disk, but I will not tell you about it in the future, and I won't even give you points as reward." Why is Baidu Netdisk so obsessed with using users' bandwidth and computer storage space to accelerate its business? What is the use of this P2P technology? Let's start with the operating costs of Baidu Netdisk.

1. How expensive is data storage?

In the central and eastern part of Shanxi Province, there is a city called "Yangquan". Since the 1980s, Yangquan has taken advantage of its coal resources and its economy has developed rapidly, leading all cities in Shanxi and being known as "Little Shanghai".

[[323735]]

Yangquan is the hometown of Baidu founder Robin Li, so it naturally receives more care. Baidu's first large-scale data center (also the largest single data center in Asia) is located here.

[[323736]]

There are 160,000 servers here, which can store 6 EB, or about 6.14 million TB of data. The amount of information these servers can store is equivalent to the total collection of more than 300,000 National Libraries of China.

[[323737]]

The hardware cost of building such a huge data center is staggering, roughly estimated at 1.96 billion yuan, with an average cost of 319.5 yuan per TB of capacity.

Each free user of Baidu Netdisk can have a maximum of 2TB of storage space.

Not only is the hardware expensive, but the electricity bill is also frighteningly expensive.

The entire data center consumes 64,000 kWh of electricity per hour. According to Yangquan City's commercial electricity standard of 0.45 yuan per kWh, the annual electricity bill costs 250 million yuan.

Do you think that's all? There's still internet charges to be calculated.

In order to provide cloud storage services, data centers need to pay broadband operators such as China Unicom and China Telecom about 100,000 yuan/Gb/month for dedicated line access fees.

Based on the average export bandwidth of 332Gb of Chinese data centers, Baidu Yangquan Data Center still has to pay 400 million yuan in Internet fees every year.

Therefore, storing data is really not cheap.

Even without considering the hardware, Baidu Yangquan Data Center spends 106 yuan per year on electricity and bandwidth for every TB of data stored.

Such high operating costs caused many online disk services in China to close down in 2016. However, Baidu Netdisk survived thanks to the support of paying users. Therefore, Baidu Netdisk is very eager for paying users. Once you pay, you can enjoy various privileges and be well served like a master. If you want to get it for free without paying, it will be very unpleasant.

Baidu Netdisk's VIP membership fee is 118 yuan per year, and the SVIP membership fee is 328 yuan per year. As for free users, only the most basic services are provided. No matter how fast your home network speed is, if you are not a member, your speed will be limited. After receiving the download request, how does the Netdisk transmit the video in the data center to you?

2. CDN: A powerful tool for accelerating network disks

If everyone accesses the Yangquan data center to download data at the same time, the large amount of data will not only impact the backbone network, but the bandwidth of the data center will also become a bottleneck. This requires another technology: CDN support. The full name of CDN is Content Delivery Network. As the name suggests, it can better distribute large amounts of content to every user.

This is an overall system that includes four components: distributed storage, load balancing, network request redirection and content management, distributed in multiple nodes across the country.

Through intelligent strategies, CDN can distribute popular resources that most users want to download to cache servers at the edge of the network, which not only reduces access response time but also alleviates the pressure on the backbone network.

In this way, most users can download resources from the local CDN server nearby, without having to squeeze through a large number of people to download from the same central node. This greatly alleviates the pressure on the data center and naturally greatly improves efficiency.

As can be seen from the figure below, Baidu's CDN nodes are spread across the country, covering all provinces, and are particularly densely distributed in economically developed areas.

In this way, basically all users can download data without leaving the province or even the city. The path is short and the bandwidth is sufficient, so the speed will naturally increase.

CDN technology can greatly speed up downloading and realize streaming media download, live broadcast and on-demand, which is very critical to the experience of VIP users of Netdisk.

After all, many users are willing to spend money to buy VIP accounts because they can't stand the slow speed of free accounts. In addition, there are corporate users who use network disks as production tools, so they are naturally more willing to spend money.

Although CDN is good, it also has its shortcomings, that is, it is not flexible enough. If there is a large amount of burst traffic during peak hours, the edge nodes may not be able to cope with the capacity.

So, is there a technology that can make download nodes infinitely expandable, bandwidth, storage almost unlimited, the more users the faster the speed, and the cost is still very low?

3. P2P: User autonomy, mutual assistance acceleration?

It sounds like a pipe dream, but in fact, this technology not only exists, but is very close to us, and most people have used it.

This is P2P technology.

This technology was once very popular in downloading software such as BT and Xunlei. Late at night, with the humming sound of the hard disk, countless lonely souls were comforted.

The full name of P2P is Peer to Peer, which means point-to-point transmission.

As the name suggests, there is no distinction between servers and clients in a P2P network. Each user's device (usually a computer) can download data from other people's computers and share the downloaded data with others for download.

P2P means that users can download resources autonomously and mutually. It can be said that I am for everyone and everyone is for me. The more people download a resource, the more nodes will participate in resource sharing and the faster the download speed will be.

P2P is a sharing network in which each node is a service provider and receiver, and each node has to share part of its own hardware resources (computing power, storage capacity, transmission capacity, etc. of a personal computer).

It is precisely because of these characteristics that P2P networks are difficult to regulate and may cause network traffic storms, widespread piracy, and even the spread of viruses.

When using software such as Xunlei to download via P2P, there are always only a few people willing to share, so there has always been a game between users and Xunlei:

Users want to finish downloading as soon as possible and exit Xunlei to avoid constantly uploading and occupying their own bandwidth; but Xunlei wants users to stay online 24 hours a day to speed up other users' downloads.

There has been no good solution to this problem, unless users are given some incentives to compensate them for their use of bandwidth and storage resources.

4. The combination of CDN and P2P: PCDN

Through the above analysis, we can find that the advantages and disadvantages of CDN and P2P are actually complementary. So, why not combine these two technologies?

Thus, PCDN (P2P+CDN) came into being.

PCDN combines the strengths of both, adopting an upper-layer CDN and lower-layer P2P network architecture, and introducing a manageable P2P autonomous region at the edge server, which can both regulate content and reduce the pressure on the backbone network.

First, the central server synchronizes popular resources to the CDN edge server; then, the edge server sends the resources to the computers of a large number of users, so that users can use P2P to transfer files to each other and speed up downloading.

Taking express delivery companies as an example, CDN is equivalent to local warehouses distributed in various cities, and P2P is equivalent to the local warehouse directly placing a batch of popular goods at the customer's home, and then mobilizing customers in the same community to deliver goods to each other, which is of course faster.

Of course, ordinary users are unaware of this, but they may feel that the computer network speed is slow, the hard drive is spinning non-stop, and the available capacity is inexplicably reduced.

As for the download speed, VIP users must have an improvement. Ordinary users have not paid, so even if they contribute traffic and hard disk, it will be useless and the speed will still be limited.

5. Why did Baidu Netdisk launch the "User Incentive Program"?

Although the PCDN structure is good, there are still uncertainties. Once the user who contributed traffic and hard disk shuts down, the P2P node disappears. How can it be accelerated?

So, we have the "User Incentive Plan" at the beginning of this article. The general meaning is: Don't turn off your computer when you have nothing to do, just leave Baidu Netdisk on, and you will get a reward!

If you can keep the computer online for 7 hours a day with an upload speed of 1MB/s for 30 consecutive days, you can get 1,000 points.

That sounds like a lot! So what can you do with these 1,000 points?

1. One-month membership of iQIYI, 1,000 points;

2. WPS membership experience for 15 days, 10 points;

3.5-minute speed download voucher, 800 points;

4. One online decompression voucher, 500 points.

The above-mentioned 5-minute high-speed download coupon is sold for 1.9 yuan during off-peak hours and 2.9 yuan during peak hours.

In other words, an average user may not even be able to earn back the electricity bill after working hard for a month. This kind of user incentive is indeed a bit shabby.

To be fair, PCDN is a good technology. Tencent, iQiyi, and Youku Tudou are all using it. After all, users' broadband is just sitting there anyway.

If we can use these idle bandwidths in a restrained manner to relieve our own pressure and give some benefits to ordinary users, it can be a win-win situation.

If Baidu Netdisk had informed users of this matter openly, given them the right to choose, and increased user incentives a little bit, the outcome might have been completely different.

Maybe users are scrambling to join the user incentive program to earn points and redeem membership.

Well, that’s all for this issue. I hope it will be helpful to you.