1. 程式人生 > >部署redis集群

部署redis集群

修改 etc slave 訪問 技術 disco x86 rwx pos

1 案例1:部署redis集群

1.1 問題

  • 具體要求如下:
  • 準備集群環境
  • 安裝redis並創建集群
  • 查看集群信息

1.2 方案

搭建redis集群,拓撲規劃如圖-1所示:

技術分享圖片

圖-1

IP,端口規劃如表-1所示:

表-1

技術分享圖片

1.3 步驟

實現此案例需要按照如下步驟進行。

步驟一:準備集群環境

1)按照表-1配置主機名,ip地址,配置yum源(系統源)這裏不再操作

2)把redis的軟件包傳到6臺數據庫服務器上面,安裝redis服務器,六臺服務器同樣操作(以51為例)

  1. [[email protected] ~]# yum -y install gcc gcc-c++ make
  2. [[email protected] ~]# cd redis
  3. redis/ redis-cluster/
  4. [[email protected] ~]# cd redis/
  5. [[email protected] redis]# ls
  6. lnmp redis-4.0.8.tar.gz
  7. [[email protected] redis]# tar -xf redis-4.0.8.tar.gz
  8. [[email protected] redis]# cd redis-4.0.8/
  9. [[email protected] redis-4.0.8]# make && make install
  10. [[email protected] redis-4.0.8]# ./utils/install_server.sh
  11. Welcome to the redis service installer
  12. This script will help you easily set up a running redis server
  13. Please select the redis port for this instance: [6379]
  14. Selecting default: 6379
  15. Please select the redis config file name [/etc/redis/6379.conf]
  16. Selected default - /etc/redis/6379.conf
  17. Please select the redis log file name [/var/log/redis_6379.log]
  18. Selected default - /var/log/redis_6379.log
  19. Please select the data directory for this instance [/var/lib/redis/6379]
  20. Selected default - /var/lib/redis/6379
  21. Please select the redis executable path [/usr/local/bin/redis-server]
  22. Selected config:
  23. Port : 6379
  24. Config file : /etc/redis/6379.conf
  25. Log file : /var/log/redis_6379.log
  26. Data dir : /var/lib/redis/6379
  27. Executable : /usr/local/bin/redis-server
  28. Cli Executable : /usr/local/bin/redis-cli
  29. Is this ok? Then press ENTER to go on or Ctrl-C to abort.
  30. Copied /tmp/6379.conf => /etc/init.d/redis_6379
  31. Installing service...
  32. Successfully added to chkconfig!
  33. Successfully added to runlevels 345!
  34. Starting Redis server...
  35. Installation successful! //安裝成功
  36. [[email protected] redis-4.0.8]# ss -antlp | grep 6379 //查看時有端口
  37. LISTEN 0 128 127.0.0.1:6379 *:* users:(("redis-server",pid=10788,fd=6))

2)修改配置文件,6臺redis服務器都要修改(以51為例子)

  1. [[email protected] redis-4.0.8]# /etc/init.d/redis_6379 stop
  2. //停止已經開啟的redis服務
  3. Stopping ...
  4. Waiting for Redis to shutdown ...
  5. Redis stopped
  6. [[email protected] redis-4.0.8]# vim /etc/redis/6379.conf
  7. ...
  8. bind 192.168.4.51 //修改ip
  9. port 6351 //不允許相同,只指定物理接口的ip
  10. daemonize yes //以守護進程方式運行
  11. pidfile /var/run/redis_6351.pid
  12. cluster-enabled yes //是否啟用集群,前提是以守護進程方式運行
  13. cluster-config-file nodes-6351.conf
  14. //存儲集群信息的配置文件,自動生成,不允許相同
  15. cluster-node-timeout 5000 //集群節點通信超時時間
  16. ...
  17. [[email protected] redis-4.0.8]# /etc/init.d/redis_6379 start //啟動服務
  18. Starting Redis server...
  19. [[email protected] redis-4.0.8]# ss -antlp | grep 6351 //查看有端口
  20. LISTEN 0 128 192.168.4.51:6351 *:* users:(("redis-server",pid=11092,fd=6))
  21. LISTEN 0 128 192.168.4.51:16351 *:* users:(("redis-server",pid=11092,fd=8)) //16051:集群中的主機通信時用的端口
  22. [[email protected] redis-4.0.8]# ps -C redis
  23. PID TTY TIME CMD

註意:其他幾臺主機在修改時請註意ip,端口等的修改,不要和51主機的一樣

3)關閉防火墻51-56主機(以51為例子)

  1. [[email protected] redis-4.0.8]# getenforce
  2. Permissive
  3. [[email protected] redis-4.0.8]# systemctl disable firewalld
  4. //關閉防火墻不自啟

4)查看集群信息

  1. [[email protected] redis-4.0.8]# redis-cli -h 192.168.4.51 -p 6351
  2. 192.168.4.51:6351> ping
  3. PONG
  4. 192.168.4.51:6351> cluster info
  5. cluster_state:fail
  6. cluster_slots_assigned:0
  7. cluster_slots_ok:0
  8. cluster_slots_pfail:0
  9. cluster_slots_fail:0
  10. cluster_known_nodes:1
  11. cluster_size:0
  12. ...
  13. 192.168.4.51:6351> cluster nodes
  14. f81f997d5ed988ec1587558e78d5f7dbc96abcbf :[email protected]16351 myself,master - 0 0 0 connected

步驟二:創建集群(在任意一臺上執行創建集群的腳本都可以)這裏在51上面執行

1)部署ruby腳本運行環境(在51上面執行)

  1. [[email protected] redis-4.0.8]# cd /root/redis-cluster/
  2. [[email protected] redis-cluster]# ls
  3. redis-3.2.1.gem ruby-devel-2.0.0.648-30.el7.x86_64.rpm
  4. [[email protected] redis-cluster]# yum -y install ruby rubygems
  5. [[email protected] redis-cluster]# rpm -ivh –nodeps \
  6. ruby-devel-2.0.0.648-30.el7.x86_64.rpm
  7. warning: ruby-devel-2.0.0.648-30.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
  8. Preparing... ################################# [100%]
  9. Updating / installing...
  10. 1:ruby-devel-2.0.0.648-30.el7 ################################# [100%]
  11. [[email protected] redis-cluster]# which gem
  12. /usr/bin/gem
  13. [[email protected] redis-cluster]# gem install redis
  14. Successfully installed redis-3.2.1
  15. Parsing documentation for redis-3.2.1
  16. Installing ri documentation for redis-3.2.1
  17. 1 gem installed

2)生成創建集群的腳本

  1. [[email protected] redis-cluster]# cd /root/redis/redis-4.0.8/src/
  2. [[email protected] src]# cp redis-trib.rb /usr/local/bin/
  3. [[email protected] src]# ll /usr/local/bin/redis-trib.rb
  4. -rwxr-xr-x. 1 root root 65991 Sep 27 16:12 /usr/local/bin/redis-trib.rb

3)創建集群

  1. [[email protected] src]# redis-trib.rb create --replicas 1 \
  2. 192.168.4.51:6351 192.168.4.52:6352 \
  3. 192.168.4.53:6353 192.168.4.54:6354 \
  4. 192.168.4.55:6355 192.168.4.56:6356
  5. //--replicas 1 給每一個主配置一個從庫
  6. [[email protected] log]# redis-trib.rb create --replicas 1 192.168.4.51:6351 192.168.4.52:6352 192.168.4.53:6353 192.168.4.54:6354 192.168.4.55:6355 192.168.4.56:6356
  7. >>> Creating cluster
  8. >>> Performing hash slots allocation on 6 nodes...
  9. Using 3 masters:
  10. 192.168.4.51:6351
  11. 192.168.4.52:6352
  12. 192.168.4.53:6353
  13. Adding replica 192.168.4.55:6355 to 192.168.4.51:6351
  14. Adding replica 192.168.4.56:6356 to 192.168.4.52:6352
  15. Adding replica 192.168.4.54:6354 to 192.168.4.53:6353
  16. ...
  17. ...
  18. ...
  19. [OK] All nodes agree about slots configuration.
  20. >>> Check for open slots...
  21. >>> Check slots coverage...
  22. [OK] All 16384 slots covered.

4)查看集群信息,任意一臺主機訪問本機的redis服務查看即可

cluster info 查看集群信息

cluster nodes 查看集群節點信息

  1. [[email protected] log]# redis-cli -h 192.168.4.52 -p 6352
  2. 192.168.4.52:6352> CLUSTER INFO
  3. cluster_state:ok //狀態
  4. cluster_slots_assigned:16384
  5. cluster_slots_ok:16384
  6. cluster_slots_pfail:0
  7. cluster_slots_fail:0
  8. cluster_known_nodes:6
  9. cluster_size:3
  10. cluster_current_epoch:6
  11. cluster_my_epoch:2
  12. cluster_stats_messages_ping_sent:367
  13. cluster_stats_messages_pong_sent:327
  14. cluster_stats_messages_meet_sent:5
  15. cluster_stats_messages_sent:699
  16. cluster_stats_messages_ping_received:327
  17. cluster_stats_messages_pong_received:372
  18. cluster_stats_messages_received:699
  19. 192.168.4.52:6352> CLUSTER NODES //查看集群節點信息
  20. 63afbb5e7d63b1f142358634578a3488e3c9e634 192.168.4.54:[email protected]16354 slave bc5c4e082a5a3391b634cf433a6486c867cfc44b 0 1538039278871 4 connected
  21. bc5c4e082a5a3391b634cf433a6486c867cfc44b 192.168.4.53:[email protected]16353 master - 0 1538039278571 3 connected 10923-16383
  22. 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:[email protected]16356 slave 8568fbd73cb296cad6915d524e34761b2114af47 0 1538039278069 6 connected
  23. 7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55:[email protected]16355 slave a3083123bc5c87a76aab2655171634d4ee84f418 0 1538039278000 5 connected
  24. 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:[email protected]16352 myself,master - 0 1538039277000 2 connected 5461-10922
  25. a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51:[email protected]16351 master - 0 1538039277869 1 connected 0-5460
  26. 192.168.4.52:6352>

5)測試集群

命令:

redis-cli -c -h ip地址 -p 端口

  1. [[email protected] log]# redis-cli -c -h 192.168.4.51 -p 6351
  2. 192.168.4.51:6351> set name jim
  3. -> Redirected to slot [5798] located at 192.168.4.52:6352
  4. OK
  5. 192.168.4.52:6352> get name
  6. "jim"
  7. 192.168.4.52:6352> set class linux
  8. OK
  9. 192.168.4.52:6352> get class
  10. "linux"
  11. 192.168.4.52:6352> set pay 26800
  12. -> Redirected to slot [4013] located at 192.168.4.51:6351
  13. OK
  14. 192.168.4.51:6351> get pay
  15. "26800"

集群不能用的情況:

有半數或者半數以上的主庫機器掛掉,集群就不能用了

把一個從庫升級成主,沒有從庫,集群不能用(前提是:有半數或者半數以上的主庫機器掛掉)

一個主庫掛掉,它的從庫自動頂替為主庫,正常使用(前提是:有半數或者半數以上的主庫機器能用),掛掉的主庫修復好後,會成為從庫,不會搶占為主

6)集群節點選舉策略(三主,三從)

停止某個主庫的redis服務,對應的從庫會自動升級為主庫

先查看節點信息的主從情況

  1. [[email protected] log]# redis-cli -c -h 192.168.4.51 -p 6351
  2. 192.168.4.51:6351> CLUSTER nodes
  3. ...
  4. 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:[email protected]16352 master - 0 1538040400840 2 connected 5461-10922
  5. 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:[email protected]16356 slave 8568fbd73cb296cad6915d524e34761b2114af47 0 1538040400000 6 connected
  6. ...
  7. 192.168.4.51:6351>

看誰是誰的從庫,如:

看節點前後的編號id是否有相同的

如:8568fbd73cb296cad6915d524e34761b2114af47

發現52的從庫為56

停止主庫52

  1. [[email protected] log]# redis-cli -h 192.168.4.52 -p 6352 shutdown
  2. [[email protected] log]# redis-cli -c -h 192.168.4.51 -p 6351
  3. 192.168.4.51:6351> CLUSTER NODES
  4. ...
  5. 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:[email protected]16352 master,fail - 1538041052349 1538041051000 2 disconnected //52的主庫壞掉
  6. 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:[email protected]16356 master - 0 1538041066000 7 connected 5461-10922 //56成為主庫
  7. ...

開啟52,發現52成為從庫

  1. [[email protected] redis-4.0.8]# /etc/init.d/redis_6352 start
  2. Starting Redis server...
  3. [[email protected] log]# redis-cli -c -h 192.168.4.51 -p 6351
  4. 192.168.4.51:6351> CLUSTER NODES
  5. 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:[email protected]16352 slave 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 0 1538041254000 7 connected

2 案例2:管理redis集群

2.1 問題

  • 具體要求如下:
  • 練習添加主機
  • 練習刪除主機

2.2 步驟

實現此案例需要按照如下步驟進行。

步驟一:添加主機

1)部署一臺新redis服務器,ip為192.168.4.58,裝包,初始化,啟用集群配置,重啟服務(這裏之前已經操作過,不會的可以參考案例1)

2) 添加集群4.58(添加master節點)

格式:redis-trib.rb 選項 參數

選項: add-nade 添加主機(不指定角色為主)

由於之前是在51上面創建ruby腳本,所以只有51上面有redis-trib.rb命令,在51上面執行

  1. [[email protected] ~]# redis-trib.rb add-node 192.168.4.58:6358 192.168.4.51:6351
  2. >>> Adding node 192.168.4.58:6358 to cluster 192.168.4.51:6351
  3. >>> Performing Cluster Check (using node 192.168.4.51:6351)
  4. S: a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51:6351
  5. slots: (0 slots) slave
  6. replicates 7e8d9121f44d8331ff55b45c218b87df9bda1b70
  7. M: 7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55:6355
  8. slots:0-5460 (5461 slots) master
  9. 1 additional replica(s)
  10. S: 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:6352
  11. slots: (0 slots) slave
  12. replicates 28e06c5f24a2b6c6412f81369e09bc9653cc51ff
  13. M: bc5c4e082a5a3391b634cf433a6486c867cfc44b 192.168.4.53:6353
  14. slots:10923-16383 (5461 slots) master
  15. 1 additional replica(s)
  16. S: 63afbb5e7d63b1f142358634578a3488e3c9e634 192.168.4.54:6354
  17. slots: (0 slots) slave
  18. replicates bc5c4e082a5a3391b634cf433a6486c867cfc44b
  19. M: 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:6356
  20. slots:5461-10922 (5462 slots) master
  21. 1 additional replica(s)
  22. [OK] All nodes agree about slots configuration.
  23. >>> Check for open slots...
  24. >>> Check slots coverage...
  25. [OK] All 16384 slots covered.
  26. >>> Send CLUSTER MEET to node 192.168.4.58:6358 to make it join the cluster.
  27. [OK] New node added correctly.

3)檢查集群主機的狀態信息

選項:check 檢查權限

  1. [[email protected] ~]# redis-trib.rb check 192.168.4.58:6358 //查看狀態
  2. >>> Performing Cluster Check (using node 192.168.4.58:6358)
  3. M: c5e0da48f335c46a2ec199faa99b830f537dd8a0 192.168.4.58:6358
  4. slots: (0 slots) master //發現沒有hash槽
  5. 0 additional replica(s)
  6. M: 7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55:6355
  7. slots:0-5460 (5461 slots) master
  8. 1 additional replica(s)
  9. ...
  10. S: a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51:6351
  11. slots: (0 slots) slave
  12. replicates 7e8d9121f44d8331ff55b45c218b87df9bda1b70
  13. [OK] All nodes agree about slots configuration.
  14. >>> Check for open slots...
  15. >>> Check slots coverage...
  16. [OK] All 16384 slots covered.

4)手動對集群進行分片遷移

選項:reshard 重新分配hash槽

  1. [[email protected] ~]# redis-trib.rb reshard 192.168.4.58:6358
  2. How many slots do you want to move (from 1 to 16384)?4096
  3. //拿出多少個hash 槽給主機192.168.4.58
  4. What is the receiving node ID? c5e0da48f335c46a2ec199faa99b830f537dd8a0
  5. //主機192.168.4.58的id值
  6. Source node #1:all //從當前所有的主裏面獲取hash槽
  7. Do you want to proceed with the proposed reshard plan (yes/no)?yes
  8. ...
  9. Moving slot 12283 from 192.168.4.53:6353 to 192.168.4.58:6358:
  10. Moving slot 12284 from 192.168.4.53:6353 to 192.168.4.58:6358:
  11. Moving slot 12285 from 192.168.4.53:6353 to 192.168.4.58:6358:
  12. Moving slot 12286 from 192.168.4.53:6353 to 192.168.4.58:6358:
  13. Moving slot 12287 from 192.168.4.53:6353 to 192.168.4.58:6358:

再次查看發現4.58有4096個hash slot

  1. [[email protected] ~]# redis-trib.rb check 192.168.4.58:6358
  2. >>> Performing Cluster Check (using node 192.168.4.58:6358)
  3. M: c5e0da48f335c46a2ec199faa99b830f537dd8a0 192.168.4.58:6358
  4. slots:0-1364,5461-6826,10923-12287 (4096 slots) master
  5. 0 additional replica(s)

5)刪除master角色的主機

先刪除主機占用的hash槽

  1. [[email protected] ~]# redis-trib.rb reshard 192.168.4.58:6358
  2. How many slots do you want to move (from 1 to 16384)?4096
  3. //移除hash 槽的個數
  4. What is the receiving node ID? bc5c4e082a5a3391b634cf433a6486c867cfc44b
  5. //要移動給誰的id即目標主機(這裏可以隨機寫一個master的ID)
  6. Source node #1: c5e0da48f335c46a2ec199faa99b830f537dd8a0
  7. //從誰那移動即源主機(這裏寫4.58的ID)
  8. Source node #2:done //設置完畢
  9. ...
  10. Moving slot 12282 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
  11. Moving slot 12283 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
  12. Moving slot 12284 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
  13. Moving slot 12285 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
  14. Moving slot 12286 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
  15. Moving slot 12287 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
  16. Do you want to proceed with the proposed reshard plan (yes/no)?yes //提交
  17. ...
  18. Moving slot 12282 from 192.168.4.58:6358 to 192.168.4.53:6353:
  19. Moving slot 12283 from 192.168.4.58:6358 to 192.168.4.53:6353:
  20. Moving slot 12284 from 192.168.4.58:6358 to 192.168.4.53:6353:
  21. Moving slot 12285 from 192.168.4.58:6358 to 192.168.4.53:6353:
  22. Moving slot 12286 from 192.168.4.58:6358 to 192.168.4.53:6353:
  23. Moving slot 12287 from 192.168.4.58:6358 to 192.168.4.53:6353:

刪除集群主機4.58(刪除之後redis服務自動關閉)

  1. [[email protected] ~]# redis-trib.rb del-node 192.168.4.58:6358 \
  2. c5e0da48f335c46a2ec199faa99b830f537dd8a0 //刪除誰+刪除的id
  3. >>> Removing node c5e0da48f335c46a2ec199faa99b830f537dd8a0 from cluster 192.168.4.58:6358
  4. >>> Sending CLUSTER FORGET messages to the cluster...
  5. >>> SHUTDOWN the node.

6)添加從節點主機,隨機添加

  1. [[email protected] ~]# redis-trib.rb add-node --slave \
  2. 192.168.4.57:6357 192.168.4.51:6351
  3. >>> Adding node 192.168.4.57:6357 to cluster 192.168.4.51:6351
  4. >>> Performing Cluster Check (using node 192.168.4.51:6351)
  5. ……
  6. ……
  7. [OK] All 16384 slots covered.
  8. Automatically selected master 192.168.4.51:6351
  9. >>> Send CLUSTER MEET to node 192.168.4.57:6357 to make it join the cluster.
  10. Waiting for the cluster to join.
  11. >>> Configure node as replica of 192.168.4.51:6351.
  12. [OK] New node added correctly.

7)移除從節點,從節點主機沒有槽位範圍,直接移除即可

命令格式:

redis-trib.rb del-node 192.168.4.57:6357 主機id值

  1. [[email protected] ~]# redis-trib.rb del-node 192.168.4.57:6357 \
  2. f6649ea99b2f01faca26217691222c17a3854381
  3. >>> Removing node f6649ea99b2f01faca26217691222c17a3854381
  4. from cluster 192.168.4.57:6351
  5. >>> Sending CLUSTER FORGET messages to the cluster...
  6. >>> SHUTDOWN the node.

部署redis集群