1. 程式人生 > >Redis學習——Redis持久化之AOF備份方式儲存資料

Redis學習——Redis持久化之AOF備份方式儲存資料

新技術的出現一定是在老技術的基礎之上,並且完善了老技術的某一些不足的地方,新技術和老技術就如同JAVA中的繼承關係。
子類(新技術)比父類(老技術)更加的強大!

在前面介紹了 Redis學習——Redis持久化之RDB備份方式儲存資料之後,下面在整理和學習一下Redis的AOF方式儲存資料。
首先丟擲幾個問題,然後回答這些問題,最後逐步的對AOF進行介紹。

1.RDB可以搞定備份恢復的事情,為什麼還會出現AOF?

    答案:這個問題的答案可以參看,上面說的Redis持久化之RDB備份方式儲存資料中的RDB的缺點。
    也就是使用RDB進行儲存時候,如果Redis伺服器傳送故障,那麼會丟失最後一次備份的資料!
    AOF出現試著來解決這個問題!

2.同時出現RDB和AOF是衝突呢?還是協作?

答案:是協作,不會衝突!那麼是如何協作,首先載入哪一個檔案呢?
進行測試,生成dump.rdb和appendonly.aof檔案,然後在appendonly.aof使檔案最後隨便加入一些東西,使檔案出錯,然後重新啟動redis服務,發現服務沒有啟動成功!那麼就可以知道首先載入的是aof檔案,使用redis-check-aof 工具修復aof檔案,重新啟動,發現啟動成功!
總結:兩者可以共存,但是首先啟動找的是aof。
當redis伺服器掛掉時,重啟時將按照以下優先順序恢復資料到記憶體:
如果只配置AOF,重啟時載入AOF檔案恢復資料;
如果同時 配置了RBD和AOF,啟動是隻載入AOF檔案恢復資料;
如果只配置RBD,啟動是講載入dump
檔案恢復資料。 恢復時需要注意,要是主庫掛了不能直接重啟主庫,否則會直接覆蓋掉從庫的AOF檔案,一定要確保要恢復的檔案都正確才能啟動,否則會沖掉原來的檔案。

如何修復:redis-check-aof –fix appendonly.aof

3.AOF有什麼優缺點?

這個看文章下面的介紹吧!

日誌的形式來記錄每個寫操作(讀操作不記錄),將Redis執行過的所有寫指令記錄下來(讀操作不記錄),只許追加檔案但不可以改寫檔案,redis啟動之初會讀取該檔案重新構建資料,換言之,redis重啟的話就根據日誌檔案的內容將寫指令從前到後執行一次以完成資料的恢復工作。

二:Redis配置檔案redis.conf中關於AOF相關配置

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

1.預設AOF沒有開啟

    appendonly no #如果要開啟,改為yes
  • 啟動:修改為yes,啟動,這裡要注意的是啟動的目錄和儲存aof檔案目錄是否一致!檢視目錄命令(config get dir),這一點在上一篇RDB中講過,不懂的請看上一篇RDB中 3:配置位置,以及快照恢復
  • 修復:使用redis-check-aof –fix 進行修復
  • 恢復:重啟redis然後重新載入

2.預設名稱

    appendonlyfilename   appendonly.aof

3.三種appendfsysnc:同步策略

    #always:同步持久化,每次發生資料變更會立即記錄到磁碟,效能較差到資料完整性比較好
    everysec:出廠的預設推薦,非同步同步,每秒記錄一次
    #no:不同步

三:重寫(rewrite)(檢視Linux伺服器命令-free記憶體, df磁碟),時間換空間的思想在裡面!
1.重寫是什麼?

 AOF採用檔案追加方式,檔案會越來越大為避免出現此種情況,新增了重寫機制,當AOF檔案的大小超過所設定的閾值時,Redis就會啟動AOF檔案的內容壓縮,只保留可以恢復資料的最小指令集.可以使用命令bgrewriteaof!

2.重寫原理

    AOF檔案持續增長而過大時,會fork出一條新程序來將檔案重寫(也是先寫臨時檔案最後再rename),
遍歷新程序的記憶體中資料,每條記錄有一條的Set語句。重寫aof檔案的操作,並沒有讀取舊的aof檔案,
而是將整個記憶體中的資料庫內容用命令的方式重寫了一個新的aof檔案,這點和快照有點類似

3.觸發機制

Redis會記錄上次重寫時的AOF大小,預設配置是當AOF檔案大小是上次rewrite後大小的一倍且檔案大於64M時觸發。預設64M。
--看觸發機制配置,知道公司實力!

四:優點

  • 每修改同步:appendfsync always 同步持久化 每次發生資料變更會被立即記錄到磁碟 效能較差但資料完整性比較好
  • 每秒同步:appendfsync everysec 非同步操作,每秒記錄 如果一秒內宕機,有資料丟失
  • 不同步:appendfsync no 從不同步

五:缺點

  • 相同資料集的資料而言aof檔案要遠大於rdb檔案,恢復速度慢於rdb
  • aof執行效率要慢於rdb,每秒同步策略效率較好,不同步效率和rdb相同

六:總結

客戶端--->命令請求--->伺服器 ------->網路協議格式的命令內容-->AOF檔案
  • AOF 檔案是一個只進行追加的日誌檔案
  • Redis可以在AOF檔案體積變得過大時,自動地在後臺對AOF進行重寫
  • AOF檔案有序地儲存了對資料庫執行所有寫入操作,這些寫入操作作為redis協議的格式儲存,因此AOF檔案的內容非常容易被人讀懂,對檔案進行分析也很輕鬆
  • 對於相同的資料集來說,AOF檔案的體積通常大於RDB檔案的體積,根據所使用的fsync策略,AOF的速度可能會慢於RDB

下面是彩蛋

最後進行一個總的總結那麼在使用redis持久化的時候到底該怎麼選擇?

  • RDB持久化方式能夠在指定的時間間隔能對你的資料進行快照儲存
  • AOF持久化方式記錄每次對伺服器寫的操作,當伺服器重啟的時候會重新執行這些命令來恢復原始的資料,AOF命令以redis協議追加儲存每次寫的操作到檔案末尾.Redis還能對AOF檔案進行後臺重寫,使得AOF檔案的體積不至於過大。

只做快取:如果你只希望你的資料在伺服器執行的時候存在,你也可以不使用任何持久化方式.

同時開啟兩種持久化方式
–在這種情況下,當redis重啟的時候會優先載入AOF檔案來恢復原始的資料,
因為在通常情況下AOF檔案儲存的資料集要比RDB檔案儲存的資料集要完整.
–RDB的資料不實時,同時使用兩者時伺服器重啟也只會找AOF檔案。那要不要只使用AOF呢?
作者建議不要,因為RDB更適合用於備份資料庫(AOF在不斷變化不好備份),
快速重啟,而且不會有AOF可能潛在的bug,留著作為一個萬一的手段。

2.效能建議

  • 因為RDB檔案只用作後備用途,建議只在Slave上持久化RDB檔案,而且只要15分鐘備份一次就夠了,只保留save 900 1這條規則。
  • 如果Enalbe AOF,好處是在最惡劣情況下也只會丟失不超過兩秒資料,啟動指令碼較簡單隻load自己的AOF檔案就可以了。代價一是帶來了持續的IO,二是AOF rewrite的最後將rewrite過程中產生的新資料寫到新檔案造成的阻塞幾乎是不可避免的。只要硬碟許可,應該儘量減少AOF rewrite的頻率,AOF重寫的基礎大小預設值64M太小了,可以設到5G以上。預設超過原大小100%大小時重寫可以改到適當的數值。
  • 如果不Enable AOF ,僅靠Master-Slave Replication 實現高可用性也可以。能省掉一大筆IO也減少了rewrite時帶來的系統波動。代價是如果Master/Slave同時倒掉,會丟失十幾分鐘的資料,啟動指令碼也要比較兩個Master/Slave中的RDB檔案,載入較新的那個。新浪微博就選用了這種架構

歡迎訪問我的csdn部落格,我們一同成長!

不管做什麼,只要堅持下去就會看到不一樣!在路上,不卑不亢!