1. 程式人生 > >redis叢集搭建之二~使用redis-trib.rb方法

redis叢集搭建之二~使用redis-trib.rb方法

關於叢集的架構請參考redis叢集架構
一、準備階段
準備階段略長,若已經熟悉redis cluster,可以跳過
1.需要安裝ruby
在開始搭建redis之前,你需要確保你本地有ruby(2.0以上),且該ruby 中有redis 擴充套件包。
ruby 官網地址
ruby 官網安裝教程
一般我都是直接下載安裝(即官網教程裡的“Building from Source”)安裝完畢之後記得確認是否安裝成功
//檢視本機是否安裝ruby,以及ruby版本

//注意這裡的2.3.1 以及安裝時間2016-05-01
$ ruby -v
ruby 2.3.1p481 (2016-05-01 revision 45883
) [universal.x86_64-darwin13]

2.需要有ruby redis包

//驗證是否安裝好ruby redis擴充套件包:[這裡檢視ruby 擴充套件包有更好的辦法請告訴我,我對ruby完全不熟悉]
//如果沒有安裝好ruby redis,請看"2.1 安裝ruby redis 擴充套件包",
//如果已經安裝好,直接看"二 實施階段"
# find / -name "redis"
/Library/Ruby/Gems/2.3.1/gems/redis-3.0.6/lib/redis

2.1 安裝ruby redis 擴充套件包

//檢視你本地是否有gem,如果沒有安裝,請看"2.2 安裝gem",否則請看下面內容
$ gem -v 2.0.14 #gem install redis -v 3.0.6 //因為gems官網的映象在國外,所以國內網路經常斷連。你需要連結一個國內映象,我用的是https://ruby.taobao.org //檢視現在映象的來源地址 $ gem sources -l *** CURRENT SOURCES *** https://rubygems.org/ /** * 增加https://ruby.taobao.org為映象,並且移除官網的https://rubygems.org/映象 */ $ gem sources --add https://ruby.taobao.org/ --remove https://rubygems.org/
//檢視現在映象的來源地址 $ gem sources -l *** CURRENT SOURCES *** https://ruby.taobao.org //安裝ruby redis 3.0.6包[這個包需要與你本地的redis包的版本一致哦] #gem install redis -v 3.0.6 //驗證是否安裝好redis擴充套件包:[這裡檢視ruby 擴充套件包有更好的辦法請告訴我,我對ruby完全不熟悉] # find / -name "redis" /Library/Ruby/Gems/2.3.1/gems/redis-3.0.6/lib/redis

2.2 安裝gem
rubygems官網下載地址
下載後就按照正常的安裝步驟安裝即可

//"./configure" 該步驟可以配置安裝路徑,以及其他引數,請檢視幫助
$ ./configure
$ make
$ sudo make install
$ gem -v
2.0.14

4.認識redis.conf檔案關於redis叢集(redis cluster)的配置
要使redis以叢集方式啟動,而不是普通單例方式啟動,需要更改redis.conf檔案以下欄位.儲存為demo_redis.conf

//這裡的port 隨意你定,你只要保證該埠7000以及(7000+10000)埠是空閒的即可。前者是用於服務client的埠。後者是用於各個例項相互通訊的bus port,預設為服務client埠加10000
port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

5.認識redis-trib.rb指令碼
該指令碼的位置在./../redis-3.0.6/src/redis-trib.rb。你可以檢視有的各個命令

$ ./../redis-3.0.6/src/redis-trib.rb 
Usage: redis-trib <command> <options> <arguments ...>

  create          host1:port1 ... hostN:portN
                  --replicas <arg>
  check           host:port
  info            host:port
  fix             host:port
                  --timeout <arg>
  reshard         host:port
                  --from <arg>
                  --to <arg>
                  --slots <arg>
                  --yes
                  --timeout <arg>
                  --pipeline <arg>
  rebalance       host:port
                  --weight <arg>
                  --auto-weights
                  --threshold <arg>
                  --use-empty-masters
                  --timeout <arg>
                  --simulate
                  --pipeline <arg>
  add-node        new_host:new_port existing_host:existing_port
                  --slave
                  --master-id <arg>
  del-node        host:port node_id
  set-timeout     host:port milliseconds
  call            host:port command arg arg .. arg
  import          host:port
                  --from <arg>
                  --copy
                  --replace
  help            (show this help)

流程為
這裡寫圖片描述

二 實施階段(使用redis-trib.rb方法)

//再次確認
//ruby 已經安裝
$ ruby -v
ruby 2.3.1p481 (2016-05-01 revision 45883) [universal.x86_64-darwin13]
//ruby redis 包已經安裝
# find / -name "redis"
/Library/Ruby/Gems/2.3.1/gems/redis-3.0.6/lib/redis
//大致路徑為
1.建立各個例項的目錄
2.啟動各個目錄內的例項
3.連線各個例項
4.給各個例項分配slot
5.增加例項為master
    5.1啟動例項
    5.2增加例項為master
    5.3為新增加的master 再分割槽(resharding) ,即從其他master移動一些slot
6.增加一個例項為salve
    6.1啟動例項
    6.2增加例項為master
7.通過master reshard刪除一個master例項
7.1 移除改master 例項的slot
7.2 del-node
8.通過master failover刪除一個master例項
8.1 人工備份一個master
8.2 del-node 
9.刪除一個salve 例項

1.建立各個例項的目錄

//一般以port為目錄名
$ ls -al
total 0
drwxr-xr-x  10   admin   340  5  8 16:38 .
drwxrwxr-x+ 81 root           admin  2754  5  1 23:40 ..
drwxr-xr-x   3   admin   102  5  8 16:37 9001
drwxr-xr-x   3   admin   102  5  8 16:38 9002
drwxr-xr-x   3   admin   102  5  8 16:38 9003
drwxr-xr-x   3   admin   102  5  8 16:39 9004
drwxr-xr-x   3   admin   102  5  8 16:39 9005
drwxr-xr-x   3   admin   102  5  8 16:39 9006
drwxr-xr-x@ 25   admin   850  4 26 21:53 redis-3.0.6
//將demo_redis.conf(見上面“4.認識redis.conf檔案關於redis叢集(redis cluster)的配置”) 複製到各個目錄中,例如9001 目錄中
$ cp  ./demo_redis.conf ./9001/
$ ls -al  ./9001/
drwxr-xr-x   6   admin    204  5  5 17:45 .
drwxr-xr-x  12   admin    408  5  6 12:23 ..
-rwxr-xr-x   1   admin  41611  5  1 23:50 redis.conf
//將該redis.conf中的port埠號改為9001
//其他各個目錄也同樣

2.啟動各個目錄內的例項

/**
 * [一定要再該埠的目錄下面哦,再外面目錄都是不行的,
 * 即便你將redis-server後面地址也重新寫正確,都是會報錯的,
 * 原因是redis啟動後要在該目錄下面生成相關與該埠的 
 * log,aof,rdb,node.*.conf等檔案]
 */
$ cd 9001
$ pwd
/Applications/redis-cluster/9001
//redis-server這裡的只要是你的redis-server地址就可以,我這裡是正好是redis-3.0.6包下面
$ ./../redis-3.0.6/src/redis-server  ./redis.conf
//[驗證-是否啟動了redis 9001例項]
$ ps -e|grep 9001
15962 ??         0:00.01 ./../redis-3.0.6/src/redis-server *:9001 [cluster] 
15965 ttys005    0:00.00 grep 9001
deMacBook-Pro:9001 $ ./../redis-3.0.6/src/redis-cli -c -p 9001
127.0.0.1:9001> cluster nodes
c33b0b3eea734d962022be568344ba9ec64356a9 :9001 myself,master - 0 0 0 connected

//再示例9002
$ cd ./../9002
$ ./../redis-3.0.6/src/redis-server redis.conf 
$ ps -e|grep 900
15962 ??         0:00.10 ./../redis-3.0.6/src/redis-server *:9001 [cluster] 
15974 ??         0:00.01 ./../redis-3.0.6/src/redis-server *:9002 [cluster] 
15976 ttys005    0:00.00 grep 900         
$ ./../redis-3.0.6/src/redis-cli -c -p 9002
127.0.0.1:9002> cluster nodes
296cb822111108b5d67a4753339a936aee1be59a :9002 myself,master - 0 0 0 connected
//其他目錄也同理

3.連線各個例項

$ ps -e|grep 900
15962 ??         0:00.68 ./../redis-3.0.6/src/redis-server *:9001 [cluster] 
15974 ??         0:00.62 ./../redis-3.0.6/src/redis-server *:9002 [cluster] 
15989 ??         0:00.42 ./../redis-3.0.6/src/redis-server *:9003 [cluster] 
15997 ??         0:00.40 ./../redis-3.0.6/src/redis-server *:9004 [cluster] 
16030 ??         0:00.06 ./../redis-3.0.6/src/redis-server *:9005 [cluster] 
16039 ??         0:00.01 ./../redis-3.0.6/src/redis-server *:9006 [cluster] 
16034 ttys004    0:00.00 src/redis-cli -c -p 9005
16041 ttys005    0:00.00 grep 900
/*連線*/
$ ./../redis-3.0.6/src/redis-trib.rb create --replicas 1 127.0.0.1:9001 127.0.0.1:9002 127.0.0.1:9003  127.0.0.1:9004  127.0.0.1:9005  127.0.0.1:9006
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:9001
127.0.0.1:9002
127.0.0.1:9003
Adding replica 127.0.0.1:9004 to 127.0.0.1:9001
Adding replica 127.0.0.1:9005 to 127.0.0.1:9002
Adding replica 127.0.0.1:9006 to 127.0.0.1:9003
......
   replicates 7bda0805a4172de778f1a7d30a5c9d851c563812
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
//[驗證] 叢集是正常可寫的
127.0.0.1:9001> set test test
-> Redirected to slot [6918] located at 127.0.0.1:9002
OK

4.給各個例項分配slot

/**
 1. 在client 檢視例項狀態,可以看到有3個master,3個slave
 2. master還有各自的slot分佈情況,例如9003 slots:10923-16383 說明已經分配好slots,就不需要分配了
 */
$ src/redis-cli -c -p 9001
127.0.0.1:9001> cluster nodes
b1953bf7307e0d844ecf4cc34256aea791e00370 127.0.0.1:9005 slave 296cb822111108b5d67a4753339a936aee1be59a 0 1462698245682 5 connected
16e79f7ceb0bfd6fb128200ad8c12b4f72a36176 127.0.0.1:9004 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462698244170 4 connected
296cb822111108b5d67a4753339a936aee1be59a 127.0.0.1:9002 master - 0 1462698247198 2 connected 5461-10922
7bda0805a4172de778f1a7d30a5c9d851c563812 127.0.0.1:9003 master - 0 1462698246185 3 connected 10923-16383
c33b0b3eea734d962022be568344ba9ec64356a9 127.0.0.1:9001 myself,master - 0 0 1 connected 0-5460
aa74ab2bb7a31f74a3b9b30efb4090d39472742d 127.0.0.1:9006 slave 7bda0805a4172de778f1a7d30a5c9d851c563812 0 1462698245178 6 connected

//測試9001例項,從上面可以看到9001管理的slots為0-5460
//a9001是在卡槽2641上的,即是在9001node 管理的卡槽
127.0.0.1:9001> cluster keyslot a9001
(integer) 2641
127.0.0.1:9001> set a9001 9001
OK
127.0.0.1:9001> keys *
1) "a9001"
127.0.0.1:9001> cluster keyslot test
(integer) 6918
//測試9002例項,從上面看到9002 管理的slots為5461-10922
127.0.0.1:9001> cluster keyslot test2
(integer) 8899
//顯然命中9002管理的卡槽slot 8899  然後客戶端也跳轉到了9002
127.0.0.1:9001> set test2 9002
-> Redirected to slot [8899] located at 127.0.0.1:9002
OK
127.0.0.1:9002> keys *
1) "test"
2) "test2"
//同理測試9003

5.增加例項為master
5.1 啟動例項

//一定要在該例項的目錄下啟動
$ cd ../9007
$ pwd
/Applications/redis-cluster/9007
//啟動例項
$ ./../redis-3.0.6/src/redis-server redis.conf
//*[驗證例項已經啟動]*/
$ ps -e|grep 9007
16071 ??         0:00.01 ./../redis-3.0.6/src/redis-server *:9007 [cluster] 
$ ./../redis-3.0.6/src/redis-cli -c -p 9007
127.0.0.1:9007> cluster nodes
74b02117a30268842e0ce617b235ee48e3152d04 :9007 myself,master - 0 0 0 connected

5.2增加例項為master

$ ./../redis-3.0.6/src/redis-trib.rb  add-node 127.0.0.1:9007 127.0.0.1:9001
>>> Adding node 127.0.0.1:9007 to cluster 127.0.0.1:9001
>>> Performing Cluster Check (using node 127.0.0.1:9001)
M: c33b0b3eea734d962022be568344ba9ec64356a9 127.0.0.1:9001
......
[OK] New node added correctly.

5.3為新增加的master 再分割槽(resharding) ,即從其他master移動一些slot

//你可以在client 檢視現在9007的例項是沒有slots的
127.0.0.1:9007> cluster nodes
b1953bf7307e0d844ecf4cc34256aea791e00370 127.0.0.1:9005 slave 296cb822111108b5d67a4753339a936aee1be59a 0 1462699050020 2 connected
aa74ab2bb7a31f74a3b9b30efb4090d39472742d 127.0.0.1:9006 slave 7bda0805a4172de778f1a7d30a5c9d851c563812 0 1462699049010 3 connected
74b02117a30268842e0ce617b235ee48e3152d04 127.0.0.1:9007 myself,master - 0 0 0 connected
c33b0b3eea734d962022be568344ba9ec64356a9 127.0.0.1:9001 master - 0 1462699045986 1 connected 0-5460
16e79f7ceb0bfd6fb128200ad8c12b4f72a36176 127.0.0.1:9004 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462699046994 1 connected
296cb822111108b5d67a4753339a936aee1be59a 127.0.0.1:9002 master - 0 1462699048003 2 connected 5461-10922
7bda0805a4172de778f1a7d30a5c9d851c563812 127.0.0.1:9003 master - 0 1462699044980 3 connected 10923-16383
//我們從9001 移動200個slots 給9007
$ ./../redis-3.0.6/src/redis-trib.rb reshard --from c33b0b3eea734d962022be568344ba9ec64356a9 \
--to 74b02117a30268842e0ce617b235ee48e3152d04 \
--slots 200 --yes --timeout 5000 127.0.0.1:9001 
>>> Performing Cluster Check (using node 127.0.0.1:9001)
......
Moving slot 199 from 127.0.0.1:9001 to 127.0.0.1:9007: 

6.增加一個例項為salve
6.1啟動例項

$ cd ../9008
$ pwd
/Applications/redis-cluster/9008
$ ./../redis-3.0.6/src/redis-server redis.conf
//
//[驗證]
$ ps -e|grep 9008
16092 ??         0:00.01 ./../redis-3.0.6/src/redis-server *:9008 [cluster] 
16094 ttys005    0:00.00 grep 9008
$ ./../redis-3.0.6/src/redis-cli -c -p 9008
127.0.0.1:9008> cluster nodes
b88cea86d04a4a3d2fb60d6c97284b5bd06e6a5d :9008 myself,master - 0 0 0 connected

6.2增加例項為slave

//增加9008例項為9001(即該c33b...6a9node點)例項的slave
$ ./../redis-3.0.6/src/redis-trib.rb  add-node \
--slave \
--master-id c33b0b3eea734d962022be568344ba9ec64356a9 127.0.0.1:9008 \
127.0.0.1:9001
>>> Adding node 127.0.0.1:9008 to cluster 127.0.0.1:9001
...
[OK] New node added correctly.

7.通過master reshard刪除一個master例項
我們試著刪除9007節點
7.1 移除該master 例項的slot

//這裡"74b02..."9007 node,而"c33b0b..."9001 node
$ ./../redis-3.0.6/src/redis-trib.rb reshard \
--from 74b02117a30268842e0ce617b235ee48e3152d04 \
--to c33b0b3eea734d962022be568344ba9ec64356a9 \
--slots 200  \
--yes \
127.0.0.1:9001
>>> Performing Cluster Check (using node 127.0.0.1:9001)
......
Moving slot 199 from 127.0.0.1:9007 to 127.0.0.1:9001: 

7.2 del-node

//檢視cluster nodes 發現 9007已經沒有slots
127.0.0.1:9002> cluster nodes
74b02117a30268842e0ce617b235ee48e3152d04 127.0.0.1:9007 master - 0 1462700615149 7 connected
//刪除9007 無slots的master 節點
$ ./../redis-3.0.6/src/redis-trib.rb del-node 127.0.0.1:9001 74b02117a30268842e0ce617b235ee48e3152d04
>>> Removing node 74b02117a30268842e0ce617b235ee48e3152d04 from cluster 127.0.0.1:9001
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
//[驗證]
$ ps -e|grep 9007
16147 ttys005    0:00.00 grep 9007
//檢視nodes表 已經無9007
127.0.0.1:9002> cluster nodes
b1953bf7307e0d844ecf4cc34256aea791e00370 127.0.0.1:9005 slave 296cb822111108b5d67a4753339a936aee1be59a 0 1462700880271 5 connected
b88cea86d04a4a3d2fb60d6c97284b5bd06e6a5d 127.0.0.1:9008 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462700881277 8 connected
7bda0805a4172de778f1a7d30a5c9d851c563812 127.0.0.1:9003 master - 0 1462700879261 3 connected 10923-16383
16e79f7ceb0bfd6fb128200ad8c12b4f72a36176 127.0.0.1:9004 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462700876236 8 connected
aa74ab2bb7a31f74a3b9b30efb4090d39472742d 127.0.0.1:9006 slave 7bda0805a4172de778f1a7d30a5c9d851c563812 0 1462700876742 6 connected
296cb822111108b5d67a4753339a936aee1be59a 127.0.0.1:9002 myself,master - 0 0 2 connected 5461-10922
c33b0b3eea734d962022be568344ba9ec64356a9 127.0.0.1:9001 master - 0 1462700877749 8 connected 0-5460

8.通過master failover刪除一個master例項
我們這裡刪除9003,使9003 的從庫slave 9006 failover 它。
8.1 人工備份一個master

//使9006 備份9003,成為新的master
/**
 * This command, that can only be send to a Redis Cluster slave node, forces the slave to start a manual 
 * failover of its master instance.
 * 這個命令必須在slave node上面執行,使slave去failover它的master(小三去趕下去正室,從而成為正室)
 * */
 $ ./../redis-3.0.6/src/redis-cli -c -p 9006
127.0.0.1:9006> cluster failover
OK
//[驗證]
127.0.0.1:9006> cluster nodes
b1953bf7307e0d844ecf4cc34256aea791e00370 127.0.0.1:9005 slave 296cb822111108b5d67a4753339a936aee1be59a 0 1462701334986 5 connected
16e79f7ceb0bfd6fb128200ad8c12b4f72a36176 127.0.0.1:9004 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462701332974 8 connected
b88cea86d04a4a3d2fb60d6c97284b5bd06e6a5d 127.0.0.1:9008 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462701335994 8 connected
c33b0b3eea734d962022be568344ba9ec64356a9 127.0.0.1:9001 master - 0 1462701331964 8 connected 0-5460
aa74ab2bb7a31f74a3b9b30efb4090d39472742d 127.0.0.1:9006 myself,master - 0 0 9 connected 10923-16383
7bda0805a4172de778f1a7d30a5c9d851c563812 127.0.0.1:9003 slave aa74ab2bb7a31f74a3b9b30efb4090d39472742d 0 1462701337004 9 connected
296cb822111108b5d67a4753339a936aee1be59a 127.0.0.1:9002 master - 0 1462701333978 2 connected 5461-10922

8.2 del-node

//
//刪除9003
$ ./../redis-3.0.6/src/redis-trib.rb del-node 127.0.0.1:9001 7bda0805a4172de778f1a7d30a5c9d851c563812
>>> Removing node 7bda0805a4172de778f1a7d30a5c9d851c563812 from cluster 127.0.0.1:9001
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
//[驗證] 已經刪除9003 node
127.0.0.1:9006> cluster nodes
b1953bf7307e0d844ecf4cc34256aea791e00370 127.0.0.1:9005 slave 296cb822111108b5d67a4753339a936aee1be59a 0 1462701613370 5 connected
16e79f7ceb0bfd6fb128200ad8c12b4f72a36176 127.0.0.1:9004 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462701609338 8 connected
b88cea86d04a4a3d2fb60d6c97284b5bd06e6a5d 127.0.0.1:9008 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462701610349 8 connected
c33b0b3eea734d962022be568344ba9ec64356a9 127.0.0.1:9001 master - 0 1462701614375 8 connected 0-5460
aa74ab2bb7a31f74a3b9b30efb4090d39472742d 127.0.0.1:9006 myself,master - 0 0 9 connected 10923-16383
296cb822111108b5d67a4753339a936aee1be59a 127.0.0.1:9002 master - 0 1462701612363 2 connected 5461-10922

9.刪除一個salve 例項

//
//刪除9008 slave node,node-id:7bda0805a...
$ ./../redis-3.0.6/src/redis-trib.rb del-node 127.0.0.1:9001 7bda0805a4172de778f1a7d30a5c9d851c563812
>>> Removing node 7bda0805a4172de778f1a7d30a5c9d851c563812 from cluster 127.0.0.1:9001
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
zhengshanshandeMacBook-Pro:9008 zhengshanshan$ ./../redis-3.0.6/src/redis-trib.rb del-node 127.0.0.1:9001 b88cea86d04a4a3d2fb60d6c97284b5bd06e6a5d
>>> Removing node b88cea86d04a4a3d2fb60d6c97284b5bd06e6a5d from cluster 127.0.0.1:9001
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
//[驗證] 已經刪除9008 node
$ ps -e|grep 9008
16183 ttys005    0:00.00 grep 9008
127.0.0.1:9006> cluster nodes
b1953bf7307e0d844ecf4cc34256aea791e00370 127.0.0.1:9005 slave 296cb822111108b5d67a4753339a936aee1be59a 0 1462701683953 5 connected
16e79f7ceb0bfd6fb128200ad8c12b4f72a36176 127.0.0.1:9004 slave c33b0b3eea734d962022be568344ba9ec64356a9 0 1462701686980 8 connected
c33b0b3eea734d962022be568344ba9ec64356a9 127.0.0.1:9001 master - 0 1462701684962 8 connected 0-5460
aa74ab2bb7a31f74a3b9b30efb4090d39472742d 127.0.0.1:9006 myself,master - 0 0 9 connected 10923-16383
296cb822111108b5d67a4753339a936aee1be59a 127.0.0.1:9002 master - 0 1462701685972 2 connected 5461-10922