前言

Redis叢集模式主要有2種:

• 主從叢集
• 分散式叢集。

前者主要是為了高可用或是讀寫分離，後者為了更好的儲存資料，負載均衡。
本文主要講解主從叢集。本章主要講解後一半部分，Redis叢集。

原理

Redis為了實現負載均衡，提供叢集模式。以三個節點為例，叢集模式相當於將1-15000片分片，分為1-5000、5000-10000、10000-15000。每個節點分一段資料片。這樣的話，當一個節點宕機後，這個節點沒有備份的話，此段分片將不再可以使用。所以，官方推薦，叢集內的每個節點都應該配備一個從節點，作為冷備。部署原理圖如下所示(暫略)。

部署

由於沒有那麼多的機器，所以我們一般單機部署6個節點(3主3從)，也就是官網推薦的模式。

主要步驟如下:

• 安裝ruby，因為分配叢集的程式碼時ruby指令碼；
• 分配叢集
• 檢視叢集結果

部署程式碼

# mkdir cluster-test
# cd cluster-test
# mkdir 7000 7001 7002 7003 7004 7005
# 在資料夾內 分別放置redis.conf檔案 檔案內容見下
# 啟動6個節點
# nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
＃ 使用ruby指令碼分配叢集資源
./redis-trib.rb create --replicas 1 127.0
 
.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

# redis.conf
port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

### 啟動redis程序7000 7001 7002 7003 7004 7005

touch start-all.sh
###  內容如下
cd 7000
nohup ../../src/redis-server  redis.conf > start.log 2>&1
 
 &
cd ../7001
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7002
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7003
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7004
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7005
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

chmod +x start-all.sh
./start-all.sh

ps -ef|grep redis 檢視程序
[[email protected]77 src]$ ps -ef|grep redis
cpic     19876     1  0 3月21 ?       05:14:19 ./redis-server *:6379
cpic     19886     1  0 3月21 ?       06:17:36 ./redis-sentinel *:26379 [sentinel]
cpic     25971     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7000 [cluster]
cpic     25972     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7001 [cluster]
cpic     25973     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7002 [cluster]
cpic     25974     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7003 [cluster]
cpic     25975     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7004 [cluster]
cpic     25976     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7005 [cluster]
cpic     26273 24815  0 15:15 pts/0    00:00:00 grep --color=auto redis

成功結果

＃ create 指令碼初始化後腳本
[[email protected]76 src]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
Adding replica 127.0.0.1:7003 to 127.0.0.1:7000
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
M: 562665aca5f7db25bbd81e7b971c2f4c9aa65f96 127.0.0.1:7000
   slots:0-5460 (5461 slots) master
M: 6f93b84f7da7d6ea1b77bce1709432de472bbbe3 127.0.0.1:7001
   slots:5461-10922 (5462 slots) master
M: 6522a9ddec5b40cade141793351df44fe40a529f 127.0.0.1:7002
   slots:10923-16383 (5461 slots) master
S: f122aea6aa02be27f77454c0fcde9c65b891cd47 127.0.0.1:7003
   replicates 562665aca5f7db25bbd81e7b971c2f4c9aa65f96
S: e5b5bbd378d8b80250dca6fb5db80e755f88c53c 127.0.0.1:7004
   replicates 6f93b84f7da7d6ea1b77bce1709432de472bbbe3
S: 93ac017741ad4e65a4de3cb17ae295e6e5505b9a 127.0.0.1:7005
   replicates 6522a9ddec5b40cade141793351df44fe40a529f
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 562665aca5f7db25bbd81e7b971c2f4c9aa65f96 127.0.0.1:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 6f93b84f7da7d6ea1b77bce1709432de472bbbe3 127.0.0.1:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: e5b5bbd378d8b80250dca6fb5db80e755f88c53c 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 6f93b84f7da7d6ea1b77bce1709432de472bbbe3
M: 6522a9ddec5b40cade141793351df44fe40a529f 127.0.0.1:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 93ac017741ad4e65a4de3cb17ae295e6e5505b9a 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 6522a9ddec5b40cade141793351df44fe40a529f
S: f122aea6aa02be27f77454c0fcde9c65b891cd47 127.0.0.1:7003
   slots: (0 slots) slave
   replicates 562665aca5f7db25bbd81e7b971c2f4c9aa65f96
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

# 自動生成的node.conf檔案
[root@cpic-redis-76 7005]# cat nodes.conf
1c7cbff1069612078d63329e5841c7430757a2cd 127.0.0.1:7003@17003 slave f5cc2de688640de572d16d72c2f174de027609a6 0 1516672833000 4 connected
dd09a06827e9bfeab3876f4464cd467f6d1279f0 127.0.0.1:7002@17002 master - 0 1516672833086 3 connected 10923-16383
5863f9ee11d3ee3beef466d76f48f221b48aa6b5 127.0.0.1:7004@17004 slave 3ed6531ec985e50e87ab8436d49f4c32ce82bdef 0 1516672832000 5 connected
3ed6531ec985e50e87ab8436d49f4c32ce82bdef 127.0.0.1:7001@17001 master - 0 1516672832000 2 connected 5461-10922
9013209020c2fe2322df8c420b1c251fe9c84955 127.0.0.1:7005@17005 myself,slave dd09a06827e9bfeab3876f4464cd467f6d1279f0 0 1516672831000 6 connected
f5cc2de688640de572d16d72c2f174de027609a6 127.0.0.1:7000@17000 master - 0 1516672832885 1 connected 0-5460
vars currentEpoch 6 lastVoteEpoch 0

# 啟動的log日誌
[[email protected]76 7005]# cat start.log
nohup: 忽略輸入
5627:C 22 Jan 21:02:58.212 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
5627:C 22 Jan 21:02:58.212 # Redis version=4.0.6, bits=64, commit=00000000, modified=0, pid=5627, just started
5627:C 22 Jan 21:02:58.212 # Configuration loaded
5627:M 22 Jan 21:02:58.214 * Increased maximum number of open files to 10032 (it was originally set to 1024).
5627:M 22 Jan 21:02:58.215 * Node configuration loaded, I'm 9013209020c2fe2322df8c420b1c251fe9c84955
5627:M 22 Jan 21:02:58.216 * Running mode=cluster, port=7005.
5627:M 22 Jan 21:02:58.216 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
5627:M 22 Jan 21:02:58.216 # Server initialized
5627:M 22 Jan 21:02:58.216 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
5627:M 22 Jan 21:02:58.216 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
5627:M 22 Jan 21:02:58.216 * Ready to accept connections
5627:M 23 Jan 10:00:27.212 # configEpoch set to 6 via CLUSTER SET-CONFIG-EPOCH
5627:M 23 Jan 10:00:27.286 # IP address for this node updated to 127.0.0.1
5627:S 23 Jan 10:00:31.238 * Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
5627:S 23 Jan 10:00:31.239 # Cluster state changed: ok
5627:S 23 Jan 10:00:31.981 * Connecting to MASTER 127.0.0.1:7002
5627:S 23 Jan 10:00:31.981 * MASTER <-> SLAVE sync started
5627:S 23 Jan 10:00:31.981 * Non blocking connect for SYNC fired the event.
5627:S 23 Jan 10:00:31.981 * Master replied to PING, replication can continue...
5627:S 23 Jan 10:00:31.982 * Trying a partial resynchronization (request 033127a1144abc3683a83a6b7257c2b56d2bd0b8:1).
5627:S 23 Jan 10:00:31.983 * Full resync from master: e5e51569d1f49762952e6cc05e1f13ddff68719e:0
5627:S 23 Jan 10:00:31.983 * Discarding previously cached master state.
5627:S 23 Jan 10:00:32.084 * MASTER <-> SLAVE sync: receiving 175 bytes from master
5627:S 23 Jan 10:00:32.084 * MASTER <-> SLAVE sync: Flushing old data
5627:S 23 Jan 10:00:32.084 * MASTER <-> SLAVE sync: Loading DB in memory
5627:S 23 Jan 10:00:32.084 * MASTER <-> SLAVE sync: Finished with success
5627:S 23 Jan 10:00:32.085 * Background append only file rewriting started by pid 10164
5627:S 23 Jan 10:00:32.108 * AOF rewrite child asks to stop sending diffs.
10164:C 23 Jan 10:00:32.108 * Parent agreed to stop sending diffs. Finalizing AOF...
10164:C 23 Jan 10:00:32.108 * Concatenating 0.00 MB of AOF diff received from parent.
10164:C 23 Jan 10:00:32.108 * SYNC append only file rewrite performed
10164:C 23 Jan 10:00:32.109 * AOF rewrite: 0 MB of memory used by copy-on-write
5627:S 23 Jan 10:00:32.181 * Background AOF rewrite terminated with success
5627:S 23 Jan 10:00:32.181 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
5627:S 23 Jan 10:00:32.181 * Background AOF rewrite finished successfully

BUG

• BUG1

[[email protected] src]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
>>> Creating cluster
[ERR] Node 127.0.0.1:7005 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
解決辦法: 刪掉原有資料dump.rdb/appendonly.aof/nodes.conf檔案。

• BUG2

ruby安裝一些列困難。

Reference