Efficient transfer tips, revealing the pros and cons of Rsync and SCP, helping you make a wise choice!

In daily operation and maintenance work, file transfer tasks frequently occur, and choosing the right tool can significantly improve work efficiency. Rsync and SCP are two common file transfer tools, but they each have their own advantages and disadvantages and are suitable for different scenarios. This article will help you make a wise choice by deeply analyzing the features, usage scenarios, and performance of these two tools, thereby saving time and effort in file transfer.

1. Introduction to Rsync and SCP

1. Rsync: A powerful tool for incremental transfers

Rsync is a tool that supports file synchronization and is widely used to back up and transfer large files. Its most powerful feature is that it supports incremental transfer, which only transfers the difference between the source and the target, thus saving bandwidth and time. In addition, Rsync supports local and remote synchronization, and can retain file metadata information such as permissions, timestamps, etc., which is suitable for complex transmission needs.

2. SCP: A simple and efficient file transfer tool

SCP transfers files based on the SSH protocol, providing a simple and secure way to copy files. SCP is an ideal choice for transferring small files and sensitive data through encrypted transmission via SSH. Compared with Rsync, SCP commands are simpler and can be directly used for single file transfer tasks, but it does not support incremental synchronization and automatic recovery of interruptions.

2. Core Function Comparison

Let's use the following table to understand the difference between Rsync and SCP:

3. Detailed explanation of usage scenarios

1. Rsync: efficient synchronization and backup solution

• Regular data backup: Rsync's incremental transfer function is particularly suitable for regular data backup. Even for large folders or databases, Rsync can efficiently synchronize the updated parts to avoid transferring large amounts of duplicate data each time.
• Large file transfer: When dealing with large files or large amounts of data, Rsync's compression capabilities and incremental transfers can significantly save bandwidth and speed up transfers.
• Strong fault tolerance: When the network environment is unstable, Rsync can resume interrupted transmission without retransmitting the entire file.

2. SCP: A simple solution for fast transmission

• Small file transfer: When you only need to transfer a small number of files, SCP is a faster choice. Its command syntax is simple and direct, and you don't need to configure complex options to complete the secure transfer of files.
• Transmit sensitive data: Since SCP uses SSH encryption, the transmission process is secure and suitable for rapid transmission of sensitive data. Encrypted communication between hosts can be achieved without complex settings.
• Fast deployment: In simple file distribution and temporary file transfer tasks, SCP can complete the task at the fastest speed without the need for complex synchronization processes.

4. Practical Operation Examples

1. Rsync usage examples

Suppose you want to synchronize the local /home/user/docs/ folder to the remote server. The command is as follows:

 rsync -avz /home/user/docs/ user@remote:/backup/docs/

• -a: Archive mode, preserving file permissions, symbolic links, and timestamps.
• -v: Verbose mode, showing the transfer process.
• -z: compress data transmission to improve efficiency.

2. SCP usage examples

Transfer the local file /home/user/file.txt to the remote server:

 scp /home/user/file.txt user@remote:/remote/path/

This command is very intuitive. It directly specifies the local file and remote path. The transmission process is encrypted by SSH by default.

5. Performance comparison and selection suggestions

1. Single file transfer comparison

The following is a script that outputs the CPU utilization during the transfer:

 #!/bin/bash if ! command -v pidstat &>/dev/null || !command -v bc &>/dev/null; then echo "请安装sysstat 和bc 软件包" exit 1 fi #rsync -avz bigfile.img [email protected]:/tmp & scp bigfile.img [email protected]:/tmp & SCP_PID=$! CPU_USAGE_SUM=0 COUNT=0 while kill -0 $SCP_PID 2> /dev/null; do #从pidstat 中正确捕获CPU的值CPU_USAGE=$(pidstat -p $SCP_PID 1 1 | awk '/[0-9]:[0-9][0-9]:[0-9][0-9] PM|AM/ && /(scp|rsync)/ {print $9}') if [[ $CPU_USAGE =~^[0-9]+([.][0-9]+)?$ ]]; then CPU_USAGE_SUM=$(echo "$CPU_USAGE_SUM + $CPU_USAGE" | bc) COUNT=$((COUNT + 1)) fi sleep 1 done if [ $COUNT -ne 0 ]; then AVG_CPU=$(echo "scale=2; $CPU_USAGE_SUM / $COUNT" | bc) echo "平均CPU : $AVG_CPU%" else echo "没有采集到CPU 数据." fi

First, we construct a large file through dd and execute the following command:

 dd if=/dev/zero of=bigfile.img bs=1M count=1024

To test the transfer using SCP, execute the following command:

 time ./scp-test.sh

After waiting for a while, the result is printed out as follows:

 root@didiplus:~# time ./scp-test.sh bigfile.img 100% 1024MB 22.2MB/s 00:46平均CPU :4.77% real 0m46.837s user 0m9.321s sys 0m39.456s

Then, test the transmission using the rsync command and execute the following command:

 time ./rsync-test.sh

After waiting for a while, the result is printed out as follows:

 root@didiplus:~# time ./rsync-test.sh sending incremental file list bigfile.img sent 32,892 bytes received 35 bytes 7,317.11 bytes/sec total size is 1,073,741,824 speedup is 32,609.77平均CPU :23.50% real 0m6.087s user 0m1.591s sys 0m0.403s

After comparison, it is found that when SCP transfers a single file, the CPU utilization is significantly lower than that of rsync, and the transmission speed is faster.

 root@didiplus:~# time ./rsync-test.sh sending incremental file list sent 63 bytes received 12 bytes 50.00 bytes/sec total size is 1,073,741,824 speedup is 14,316,557.65没有采集到CPU 数据. real 0m2.030s user 0m0.387s sys 0m0.050s root@didiplus:~# time ./scp-test.sh bigfile.img 100% 1024MB 22.8MB/s 00:44平均CPU :5.08% real 0m46.800s user 0m9.426s sys 0m38.125s

However, when you perform the rsync transfer again, you will notice that the second transfer speed is significantly faster and almost no CPU resources are used. This is because rsync will intelligently determine whether the file already exists, and the existing file will not be transferred repeatedly. SCP will retransmit the entire file every time regardless of whether the file already exists.

2. Directory transfer comparison

Modify the script, change the file to a directory, and keep the rest unchanged, as shown below

 #rsync -avz /root/test_dir/ [email protected]:/tmp & scp -rq /root/test_dir/ [email protected]:/tmp &

Then, test the SCP transmission effect, run the script, and the output results are shown as shown below. Since each SCP transmission is a complete transmission, the time and CPU usage of the two transmissions are almost the same.

 root@didiplus:~# time ./scp-test.sh平均CPU :3.98% real 0m16.368s user 0m2.974s sys 0m11.934s root@didiplus:~# time ./scp-test.sh平均CPU :4.66% real 0m14.366s user 0m2.979s sys 0m11.137s

Next, we test the rsync transmission efficiency. The same script is executed twice, and the output results are as follows:

 root@didiplus:~# time ./rsync-test.sh sending incremental file list ./ bigfile.img sent 131 bytes received 229,426 bytes 41,737.64 bytes/sec total size is 314,572,800 speedup is 1,370.35平均CPU :5.99% real 0m6.086s user 0m0.778s sys 0m0.213s root@didiplus:~# time ./rsync-test.sh sending incremental file list sent 81 bytes received 12 bytes 186.00 bytes/sec total size is 314,572,800 speedup is 3,382,503.23没有采集到CPU 数据. real 0m2.029s user 0m0.386s sys 0m0.058s

Since rsync can detect whether a file already exists, the second transfer takes much less time than the first. If we only compare the time and CPU utilization of the first transfer, SCP is faster and consumes less resources, whether it is transferring files or directories.

3. Incremental scenario comparison

Finally, test the directory increment by copying some new files into the directory to be transferred.

 cp hostResourceUtil.py install.sh xp-install.log test_dir/

Then, test rsync and execute the test rsync script. The output is as follows:

 root@didiplus:~# time ./rsync-test.sh sending incremental file list ./ hostResourceUtil.py install.sh xp-install.log sent 10,654 bytes received 76 bytes 7,153.33 bytes/sec total size is 314,604,934 speedup is 29,320.12没有采集到CPU 数据. real 0m2.030s user 0m0.396s sys 0m0.052s root@didiplus:~# time ./scp-test.sh平均CPU :5.33% real 0m14.460s user 0m2.915s sys 0m10.500s

Rsync calculates the list of files that need to be transferred and only transfers the new files, while scp is not as good as rsync in the incremental scenario.

Summarize

How to choose between Rsync and SCP?

• Scenarios for choosing Rsync: When you need to perform large-scale file transfers, regular backups, or have high requirements for transfer efficiency and stability, Rsync is undoubtedly the first choice.
• Choose SCP when: When you only need to transfer a small amount of files occasionally and need to complete the task quickly, SCP is simple and direct, with concise commands, making it an ideal tool for transferring a small amount of files.

Rsync and SCP both have their own advantages and disadvantages. Only by understanding their advantages and disadvantages can you make the best choice in actual work. Whether it is efficient data backup or easy small file transfer, choosing the right tool can not only improve work efficiency, but also bring more convenience to your workflow.