rsync同步和備份檔案到本地
轉載請註明文章出處：https://tlanyan.me/use-rsync-backup-files/
rsync
是主機間同步和備份的神器。相對於ftp
、scp
等工具，rsync
功能更強大，同步/傳輸效率更高，實屬伺服器的必備工具。
最近使用rsync
時發現一個問題：PC和行動硬碟之間用rsync
同步，修改過的二進位制大檔案會整個檔案重傳，效率十分低下。說好的rsync
只傳輸差異部分呢？還是二進位制檔案的問題？但rsync
的man手冊明明這樣寫的：
Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied.It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.
帶著這個疑問上網查詢，找到一個和我有同樣困惑的人：Smarter filetransfers than rsync? 。
幸運的是有人完美的回答了這個問題：
Rsync will not use deltas but will transmit the full file in its entirety if it – as a single process – is responsible for the source and destination files. It can transmit deltas when there is a separate client and server process running on the source and destination machines. The reason that rsync will not send deltas when it is the only process is that in order to determine whether it needs to send a delta it needs to read the source and destination files. By the time it’s done that it might as well have just copied the file directly.
翻譯過來是：主機間通過網路同步檔案，每個主機各執行一個rsync
程序分別本機內的檔案hash，然後通過網路傳輸差異部分；主機內的同步只有一個程序，rsync
認為與其先對比檔案再複製差異部分，不如直接進行復制來得快，故而選擇傳送整個檔案。
仔細想一下，rsync
的行為是合理的：主機間通訊的瓶頸在網路頻寬，先計算差異部分再傳效率高；同主機內是硬碟對拷，速度是網路速度的十來倍，直接拷貝比一般比先對比再傳輸更快，直接複製整個檔案是很好的選擇。
寫了個指令碼測試rsync
的行為：
#!/bin/bash echo "make test file" dd if=/dev/zero of=testfile bs=1024k count=512 echo "cp test file" cp testfile syncfile echo "make changes to test file" echo '1234567890' >> testfile echo "rsync file in local..." rsync -avh -P testfile syncfile echo "" echo "restore sync file" dd if=/dev/zero of=syncfile bs=1024k count=512 echo "rsync file via network" rsync -avh -P testfile localhost:~/syncfile
測試指令碼輸出結果如下：
結果和預期的一致：本機內同步時，直接全量複製；走SSH協議後，僅傳送差異部分，顯著提高效率。
rsync
的做法沒毛病，但僅做過小部分修改的大檔案，同主機內全量拷貝也很傷人。解決辦法是用測試指令碼內的模擬網路傳輸。Linux系統的主機基本都內建SSHD，寫命令時加上localhost
和代表網路的冒號即可；Windows 10的1809版本上，OpenSSH已經成為系統的內建組建，安裝和使用也省心。此外有Cygwin
、Bitvise SSH Server
等可供選擇，安裝好後也同步大檔案也不再是問題。