1. 程式人生 > >Redis核心解讀–AOF與REWRITE機制

Redis核心解讀–AOF與REWRITE機制

  • 自動的bgrewriteaof

    為了避免aof檔案過大,我們會週期性的做bgrewriteaof來重整aof檔案。以前我們會額外的配置crontab在業務低峰期執行這個命令,這額外的增加一個workaroud的指令碼任務在大叢集裡是很糟糕的,不易檢查,出錯無法即時發現。

    於是這個自動bgrewriteaof功能被直接加到redis的內部。首先對於aof檔案,server物件新增一個欄位來記錄aof檔案的大小server.appendonly_current_size,每次aof發生變化都會維護這個欄位。

    aof.c
    =================
    116     nwritten = write(server.appendfd,server.aofbuf,sdslen(server.aofbuf));
    .....
    128     server.appendonly_current_size += nwritten;

    bgrewriteaof完畢或者例項啟動載入aof資料後也會呼叫aofUpdateCurrentSize這個函式維護這個欄位,同時會記錄下此時的aof檔案的大小server.auto_aofrewrite_base_size作為基準值,用於接下來判斷aof增長率。

    aof.c
    =================
    385     aofUpdateCurrentSize();
    386     server.auto_aofrewrite_base_size = server.appendonly_current_size;

    有了當前值和基準值我們就可以判斷aof檔案的增長情況。另外還需要配置兩個引數來判斷是否需要自動觸發bgrewriteaof。

    redis.h
    ===============
    int auto_aofrewrite_perc; /* Rewrite AOF if % growth is > M and... */
    off_t auto_aofrewrite_min_size; /* the AOF file is at least N bytes. */

    auto_aofrewrite_perc: aof檔案的大小超過基準百分之多少後觸發bgrewriteaof。預設這個值設定為100,意味著當前aof是基準大小的兩倍的時候觸發bgrewriteaof。把它設定為0可以禁用自動觸發的功能。
    auto_aofrewrite_min_size: 當前aof檔案大於多少位元組後才觸發。避免在aof較小的時候無謂行為。預設大小為64mb。
    兩個引數都是可以在conf裡靜態配置,或者通過config set來動態修改的。

    redis 127.0.0.1:6379> config get auto-aof-rewrite-percentage
    1) "auto-aof-rewrite-percentage"
    2) "100"
    redis 127.0.0.1:6379> config get auto-aof-rewrite-min-size
    1) "auto-aof-rewrite-min-size"
    2) "1048576"
    redis 127.0.0.1:6379> config get auto-aof-rewrite-min-size
    1) "auto-aof-rewrite-min-size"
    2) "1048576"
    redis 127.0.0.1:6379> config set auto-aof-rewrite-percentage 200
    OK
    redis 127.0.0.1:6379> config set auto-aof-rewrite-min-size 10485760
    OK

    然後就是觸發檢查的主邏輯,serverCron時間事件中每次都會檢查現有狀態和引數來判斷是否需要啟動bgrewriteaof。

    redis.c
    ===============
    635          if (server.bgsavechildpid == -1 &&
     636              server.bgrewritechildpid == -1 &&
     637              server.auto_aofrewrite_perc &&
     638              server.appendonly_current_size > server.auto_aofrewrite_min_size)
     639          {
     640             long long base = server.auto_aofrewrite_base_size ?
     641                             server.auto_aofrewrite_base_size : 1;
     642             long long growth = (server.appendonly_current_size*100/base) - 100;
     643             if (growth >= server.auto_aofrewrite_perc) {
     644                 redisLog(REDIS_NOTICE,"Starting automatic rewriting of AOF on %lld%% growth",growth);
     645                 rewriteAppendOnlyFileBackground();
     646             }
     647         }

    以上程式碼顯示,如果aof檔案增長百分率growth大於auto_aofrewrite_perc,則自動的觸發後一個bgrewriteaof。

    延遲bgrewriteaof

    這是個小的改進,手動觸發的bgrewriteaof的時候如果同時存在bgsave在備份,會推遲這次操走的事件,設定server.aofrewrite_scheduled=1,待到bgsave結束後的下一次serverCron裡才會觸發。

    設定aofrewrite_scheduled=1

    aof.c
    706 void bgrewriteaofCommand(redisClient *c) {
    707     if (server.bgrewritechildpid != -1) {
    708         addReplyError(c,"Background append only file rewriting already in progress");
    709     } else if (server.bgsavechildpid != -1) {
    710         server.aofrewrite_scheduled = 1;
    711         addReplyStatus(c,"Background append only file rewriting scheduled");
    712     } else if (rewriteAppendOnlyFileBackground() == REDIS_OK) {
    713         addReplyStatus(c,"Background append only file rewriting started");
    714     } else {
    715         addReply(c,shared.err);
    716     }
    717 }

    觸發bgrewriteaof

    redis.c
     598     if (server.bgsavechildpid == -1 && server.bgrewritechildpid == -1 &&
     599         server.aofrewrite_scheduled)
     600     {
     601         rewriteAppendOnlyFileBackground();
     602    }