• 是一款開源的,快速的，多功能的m,可實現全量及增量的本地或者遠程數據同步備份的優秀工具,rsync適用於window/unix/linux等多種操作系統平臺;
• 在同步備份數據時,默認情況下,rsync通過其獨特的quick check 算法,
  • 它僅同步大小或者最後修改時間發生變化的文件或者目錄,
  • 當然也可根據權限,屬主等屬性的變化同步,但需要指定相應的參數
  • 甚至可以實現只同步一個文件裏有變化的內容部分,所以,可以實現快速的同步備份數據.

rsync與ssh的scp以及cp命令之間的對比

rsync

• 可以是遠程的全量及增量的復制數據
• 可以是本地不同分區或者目錄之間全量及增量的復制數據
• 傳輸數據不是加密的
• rsync還可以實現刪除文件和目錄的功能,相當於rm命令

ssh的scp

• 每次都是遠程的全量的復制數據
• 傳輸數據是加密傳輸的

cp

• 每次都是本地的全量的復制數據

rsync的特性

• 可以更新整個目錄樹和文件系統
• 支持拷貝特殊文件如鏈接文件,設備等
• 可以選擇保留符號鏈接、硬鏈接、文件所有權、權限、設備和時間(-p)
• 可實現增量同步,即只同步發生變化的數據,因此數據傳輸效率很高,tar
• 不需要特殊的特權來安裝。
• 內部管道可以減少多個文件的延遲。
• 可以使用rsh、ssh、rcp等方式配合傳輸文件(rsync本身不對數據進行加密)
• 可以通過socket(進程方式)傳輸文件和數據(服務端和客戶端)
• 支持匿名rsync或認證(無需系統用戶)的進程模式傳輸，可實現方便安全的進行數據備份及鏡像,這是鏡像的理想選擇。

rsync使用的場景

兩臺服務器之間數據同步

[老男孩linux實戰培訓初級第10次課後作業實戰考試集群架構圖-3]()

把所有客戶服務器數據同步到備份服務器

生產場景集群架構服務器備份方案

rsync的工作方式

• 單個主機本地之間的數據傳輸(此時類似於cp命令)
• 借助rcp,ssh等通道來傳輸數據(此時類似於scp命令)
• 以守護進程(socket)的方式傳輸數據(這是rsunc自身的重要功能)

rsync(1)                                                              rsync(1)

NAME
       rsync — a fast, versatile, remote (and local) file-copying tool

SYNOPSIS
       Local:  rsync [OPTION...] SRC... [DEST]

       Access via remote shell:
         Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
         Push: rsync [OPTION...] SRC... [USER@]HOST:DEST

       Access via rsync daemon:
         Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]
               rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
         Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
               rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST

       Usages with just one SRC arg and no DEST arg will list the source files instead of copying.
 

rsync命令的使用

rsync選項

 -v, --verbose               顯示詳細信息
 -q, --quiet                 suppress non-error messages
     --no-motd               suppress daemon-mode MOTD (see manpage caveat)
 -c, --checksum              skip based on checksum, not mod-time & size
 -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
     --no-OPTION             turn off an implied OPTION (e.g. --no-D)
 -r, --recursive             遞歸的目錄
 -R, --relative              use relative path names
     --no-implied-dirs       don‘t send implied dirs with --relative
 -b, --backup                make backups (see --suffix & --backup-dir)
     --backup-dir=DIR        make backups into hierarchy based in DIR
     --suffix=SUFFIX         set backup suffix (default ~ w/o --backup-dir)
 -u, --update                skip files that are newer on the receiver
     --inplace               update destination files in-place (SEE MAN PAGE)
     --append                append data onto shorter files
     --append-verify         like --append, but with old data in file checksum
 -d, --dirs                  transfer directories without recursing
 -l, --links                 copy symlinks as symlinks
 -L, --copy-links            transform symlink into referent file/dir
     --copy-unsafe-links     only "unsafe" symlinks are transformed
     --safe-links            ignore symlinks that point outside the source tree
 -k, --copy-dirlinks         transform symlink to a dir into referent dir
 -K, --keep-dirlinks         treat symlinked dir on receiver as dir
 -H, --hard-links            preserve hard links
 -p, --perms                 保存權限
 -E, --executability         preserve the file‘s executability
     --chmod=CHMOD           affect file and/or directory permissions
 -A, --acls                  preserve ACLs (implies --perms)
 -X, --xattrs                preserve extended attributes
 -o, --owner                 僅保留所有者(超級用戶)
 -g, --group                 保留組
     --devices               preserve device files (super-user only)
     --copy-devices          copy device contents as regular file
     --specials              preserve special files
 -D                          same as --devices --specials
 -t, --times                 保存修改時間
 -O, --omit-dir-times        omit directories from --times
     --super                 receiver attempts super-user activities
     --fake-super            store/recover privileged attrs using xattrs
 -S, --sparse                handle sparse files efficiently
 -n, --dry-run               perform a trial run with no changes made
 -W, --whole-file            copy files whole (without delta-xfer algorithm)
 -x, --one-file-system       don‘t cross filesystem boundaries
 -B, --block-size=SIZE       force a fixed checksum block-size
 -e, --rsh=COMMAND           specify the remote shell to use
     --rsync-path=PROGRAM    specify the rsync to run on the remote machine
     --existing              skip creating new files on receiver
     --ignore-existing       skip updating files that already exist on receiver
     --remove-source-files   sender removes synchronized files (non-dirs)
     --del                   an alias for --delete-during
     --delete                delete extraneous files from destination dirs
     --delete-before         receiver deletes before transfer, not during
     --delete-during         receiver deletes during transfer (default)
     --delete-delay          find deletions during, delete after
     --delete-after          receiver deletes after transfer, not during
     --delete-excluded       also delete excluded files from destination dirs
     --ignore-errors         delete even if there are I/O errors
     --force                 force deletion of directories even if not empty
     --max-delete=NUM        don‘t delete more than NUM files
     --max-size=SIZE         don‘t transfer any file larger than SIZE
     --min-size=SIZE         don‘t transfer any file smaller than SIZE
     --partial               keep partially transferred files
     --partial-dir=DIR       put a partially transferred file into DIR
     --delay-updates         put all updated files into place at transfer‘s end
 -m, --prune-empty-dirs      prune empty directory chains from the file-list
     --numeric-ids           don‘t map uid/gid values by user/group name
     --timeout=SECONDS       set I/O timeout in seconds
     --contimeout=SECONDS    set daemon connection timeout in seconds
 -I, --ignore-times          don‘t skip files that match in size and mod-time
     --size-only             skip files that match in size
     --modify-window=NUM     compare mod-times with reduced accuracy
 -T, --temp-dir=DIR          create temporary files in directory DIR
 -y, --fuzzy                 find similar file for basis if no dest file
     --compare-dest=DIR      also compare destination files relative to DIR
     --copy-dest=DIR         ... and include copies of unchanged files
     --link-dest=DIR         hardlink to files in DIR when unchanged
 -z, --compress              在傳輸過程中壓縮文件數據
     --compress-level=NUM    explicitly set compression level
     --skip-compress=LIST    skip compressing files with a suffix in LIST
 -C, --cvs-exclude           auto-ignore files the same way CVS does
 -f, --filter=RULE           add a file-filtering RULE
 -F                          same as --filter=‘dir-merge /.rsync-filter‘
                             repeated: --filter=‘- .rsync-filter‘
     --exclude=PATTERN       exclude files matching PATTERN
     --exclude-from=FILE     read exclude patterns from FILE
     --include=PATTERN       don‘t exclude files matching PATTERN
     --include-from=FILE     read include patterns from FILE
     --files-from=FILE       read list of source-file names from FILE
 -0, --from0                 all *-from/filter files are delimited by 0s
 -s, --protect-args          no space-splitting; only wildcard special-chars
     --address=ADDRESS       bind address for outgoing socket to daemon
     --port=PORT             specify double-colon alternate port number
     --sockopts=OPTIONS      specify custom TCP options
     --blocking-io           use blocking I/O for the remote shell
     --stats                 give some file-transfer stats
 -8, --8-bit-output          leave high-bit chars unescaped in output
 -h, --human-readable        output numbers in a human-readable format
     --progress              show progress during transfer
 -P                          same as --partial --progress
 -i, --itemize-changes       output a change-summary for all updates
     --out-format=FORMAT     output updates using the specified FORMAT
     --log-file=FILE         log what we‘re doing to the specified FILE
     --log-file-format=FMT   log updates using the specified FMT
     --password-file=FILE    read daemon-access password from FILE
     --list-only             list the files instead of copying them
     --bwlimit=KBPS          limit I/O bandwidth; KBytes per second
     --write-batch=FILE      write a batched update to FILE
     --only-write-batch=FILE like --write-batch but w/o updating destination
     --read-batch=FILE       read a batched update from FILE
     --protocol=NUM          force an older protocol version to be used
     --iconv=CONVERT_SPEC    request charset conversion of filenames
 -4, --ipv4                  prefer IPv4
 -6, --ipv6                  prefer IPv6
     --version               print version number
(-h) --help                  show this help (-h works with no other options)

rsync的使用

• 本地同步

  rsync avz /etc/hosts /tmp/

• 刪除文件(相當於rm)

rsync -r --delete /null/ /data/ # 比較null目錄,與null目錄保持一致,null目錄什麽都沒有,data目錄就會刪除所有

[root@zy ~]# mkdir null
[root@zy ~]# mkdir data
[root@zy ~]# ls
data  null  release  semaphore  test.sh  testsource.sh
[root@zy ~]# touch data/index
[root@zy ~]# ls
data  null  release  semaphore  test.sh  testsource.sh
[root@zy ~]# #rsync -r --delete ./null/ ./data/
[root@zy ~]# ls ./null/
[root@zy ~]# ls ./data/
index
[root@zy ~]# rsync -r --delete ./null/ ./data/ 
[root@zy ~]# ls ./null/
[root@zy ~]# ls ./data/ 
#=======================================================
[root@zy ~]# mkdir null/ddd
[root@zy ~]# rsync -r --delete ./null/ ./data/
[root@zy ~]# ls ./null/
ddd
[root@zy ~]# ls ./data/
ddd

• 指定通道進行同步數據(註意權限問題)

  rsync -avz /etc/hosts -e ‘ssh -p 22‘ 192.168.0.101:/tmp/ # push
  rsync -avz -e ‘ssh -p 22‘ 192.168.0.101:/tmp/hosts $HOME # pull
  rsync -vzrtopg --progress -e ssh --delete [email protected]:/www/* /databack/experiment/rsync

• 使用一個遠程shell程序(如rsh、ssh)來實現將遠程機器的內容拷貝到本地機器

  rsync -avz 192.168.1.100:/tmp/hosts /data

• 使用一個遠程shell程序(如rsh、ssh)來實現將本地機器的內容拷貝到遠程機器

  rsync -avz *.c [email protected]:/tmp

• ==從遠程rsync服務器中拷貝文件到本地機==

  rsync -av [email protected]::www /databack

• ==從本地機器拷貝文件到遠程rsync服務器中==

rsync -av /databack [email protected]::www

• 列遠程rsync服務器文件列表

rsync -av /databack [email protected]::www

rsync後臺服務的方式同步

配置服務

vi /etc/xinetd.d/rsync
# default: off
# description: The rsync server is a good addition to an ftp server, as it #       allows crc checksumming etc.
service rsync
{
        disable = no
        flags           = IPv6
        socket_type     = stream
        wait            = no
        user            = root
        server          = /usr/bin/rsync
        server_args     = --daemon
        log_on_failure  += USERID
}

服務啟動

/etc/init.d/xinetd restart

配置rsync的配置文件

/etc/rsyncd.conf ==需要手動創建==

vi /etc/rsyncd.conf

uid=root
gid=root
max connections=4
log file=/var/log/rsyncd.log
pid file=/var/run/rsyncd.pid
lock file=/var/run/rsyncd.lock
secrets file=/etc/rsyncd.passwd # 同步時需要認證
hosts deny=172.16.78.0/22

[dockerimages]
comment= backup web
path=/home/xue.long/dockerimages
read only = no
exclude=test
auth users=marion

配置認證文件

認證文件中的用戶建議不要使用系統上的用戶,保證系統的安全

echo "work:abc123" > /etc/rsyncd.passwd
chmod 600 /etc/rsyncd.passwd

開始備份

rsync -avz --progress --delete [email protected]::www /databack/experiment/rsync

恢復

rsync -avz --progress /databack/experiment/rsync/ [email protected]::www

Rsync本地及遠程同步工具