1. 程式人生 > >redis 大批量資料插入導致MISCONF Redis is configured to save RDB snapshots的解決

redis 大批量資料插入導致MISCONF Redis is configured to save RDB snapshots的解決

開發十年,就只剩下這套架構體系了! >>>   

PS:之前寫過一遍,那個方法沒有徹底解決,現找到真正的解決方法

環境:redis 3.2.100 windows版(注意!!!這是關鍵),win10,redis客戶端spring boot 2.0.7,以及配對的spring data redis

某功能會頻繁地大批量地往redis寫入資料,資料量大概10秒內超過10000條

插入時可能會報錯,報錯時間點不定,大概都是執行一段時間後報錯,錯誤資訊如下:

nested exception is io.lettuce.core.RedisCommandExecutionException: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

同時在redis的log找到如下報錯:

=== REDIS BUG REPORT START: Cut & paste starting from here ===
Redis version: 3.2.100
[18808] 15 Mar 11:12:49.805 # --- EXCEPTION_ACCESS_VIOLATION
[18808] 15 Mar 11:12:49.805 # --- STACK TRACE
redis-server.exe!LogStackTrace(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:95)(0x0014E250, 0x0014FF90, 0x00000000, 0x4013A7F8)
redis-server.exe!UnhandledExceptiontHandler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000000, 0x38D470E0)
KERNELBASE.dll!UnhandledExceptionFilter(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x401555B4, 0x38E3C15C, 0x00000000, 0x00000000)
ntdll.dll!memset(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0014EEB0, 0x00000000, 0x40140E48, 0x0014EEB0)
ntdll.dll!_C_specific_handler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x0014E3C0, 0x0014E9C0, 0x00000000)
ntdll.dll!_chkstk(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0014E3C0, 0x0014E9C0, 0x3623AB10, 0x00000000)
ntdll.dll!RtlWalkFrameChain(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x511B7D55, 0x0014F220, 0x6506FC19)
ntdll.dll!KiUserExceptionDispatcher(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x40094F01, 0x0014F1F0, 0x400BFA57, 0x0014F2E0)
redis-server.exe!dictSdsHash(c:\release\redis\src\server.c:496)(0x0014F1F0, 0x400BFA57, 0x0014F2E0, 0x00000001)
redis-server.exe!dictFind(c:\release\redis\src\dict.c:517)(0x0014F220, 0x00000001, 0x0000002B, 0x00000001)
redis-server.exe!getExpire(c:\release\redis\src\db.c:871)(0x00000001, 0x0014F2E0, 0x0014F2E0, 0x03C07040)
redis-server.exe!rdbSaveRio(c:\release\redis\src\rdb.c:814)(0x40167210, 0x00DD0000, 0x00000005, 0x011D2754)
redis-server.exe!rdbSave(c:\release\redis\src\rdb.c:884)(0x00DD0000, 0x00DD0000, 0x5C8B2352, 0x00000005)
redis-server.exe!QForkChildInit(c:\release\redis\src\win32_interop\win32_qfork.cpp:337)(0x00000005, 0x00000000, 0x004BEF40, 0x00000005)
redis-server.exe!QForkStartup(c:\release\redis\src\win32_interop\win32_qfork.cpp:515)(0x00000006, 0x00000000, 0x00000000, 0x004A5550)
redis-server.exe!main(c:\release\redis\src\win32_interop\win32_qfork.cpp:1240)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
redis-server.exe!__tmainCRTStartup(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
KERNEL32.DLL!BaseThreadInitThunk(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
[18808] 15 Mar 11:12:50.398 # --- INFO OUTPUT
[5660] 15 Mar 11:12:55.080 # fork operation failed
[5660] 15 Mar 11:12:55.226 # Background saving terminated by signal 1
[5660] 15 Mar 11:12:55.353 * 10000 changes in 60 seconds. Saving...

網上能找到很多解決方法(具體執行baidu),一種主流的方法是redis配置stop-writes-on-bgsave-error設為false,但這種方法只是讓出錯不停止,錯誤還是存在的,所以不想用這種方法

 

另外原因大概也清楚:就是redis寫入資料會先寫到記憶體,然後每隔一段時間會把記憶體資料寫入rdb(一個檔案資料庫),以實現不會因為關機記憶體資料丟失就沒了資料。問題是在寫入rdb時出錯

但不知道寫入rdb出錯原因在哪,網上很多說是記憶體不夠,我看了記憶體佔用也很低,只用了200+m

 

此問題直接原因應該是寫入rdb時出錯,但根本原因在於redis本身有bug

redis只有linux版,而windows版是微軟自己做的,而且也停止更新,大概因此有bug也是正常吧

解決方法是redis(服務端)使用3.0.504,此版本