redis叢集之使用浮動vip實現故障切換全過程
目錄
生產環境：
使用三臺裝有redis的伺服器實現一臺做主資料庫伺服器兩臺做從資料庫伺服器，在使用哨兵監控三臺伺服器。使得當主資料庫伺服器發生故障的時候，從資料庫伺服器可以在一定的時間內切換成主資料庫伺服器，在使用浮動vip的技術實現讀寫分離。
搭建步驟：
1.三臺伺服器配置ip地址分別是192.168.4.21 192.168.4.22 192.168.4.23
2.在三臺伺服器上配置redis伺服器 redis-3.2.11
3.更改redis的配置檔案，指定主從。
4.編寫哨兵檔案
5.編寫浮動vip指令碼
搭建的整體過程：
1.配置IP （只舉例一臺配置過程）
[[email protected] ~]# nmcli connection modify eth0 ipv4.method manual ipv4.addresses 192.168.4.21/24 connection.autoconnect yes
[[email protected] ~]# nmcli connection up eth0
2.配置redis伺服器（只舉例一臺配置過程）
[[email protected] ~]# tar -xaf redis-3.2.11.tar.gz
[[email protected] ~]# yum install -y gcc gcc-c++
[[email protected] ~]# cd redis-3.2.11/
[[email protected] redis-3.2.11]# make
[[email protected] redis-3.2.11]# make install
[[email protected] redis-3.2.11]# cd utils/
[[email protected] utils]# ls
build-static-symbols.tcl corrupt_rdb.c generate-command-help.rb hyperloglog lru redis_init_script redis-sha1.rb speed-regression.tcl
cluster_fail_time.tcl create-cluster hashtable install_server.sh redis-copy.rb redis_init_script.tpl releasetools whatisdoing.sh
[[email protected] utils]# ./install_server.sh
Welcome to the redis service installer
This script will help you easily set up a running redis server
Please select the redis port for this instance: [6379]
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]
Selected default - /var/lib/redis/6379
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server...
Installation successful!
[[email protected] utils]# ss -untlp | grep redis
tcp LISTEN 0 128 127.0.0.1:6379 *:* users:(("redis-server",pid=5016,fd=4))
3.更改redis的配置檔案，指定主從
主庫的配置 （使用192.168.4.21做主機）
[[email protected] ~]# vim /etc/redis/6379.conf
62 bind 0.0.0.0
85 port 6379
129 daemonize yes
[[email protected] ~]# vim /etc/init.d/redis_6379 +43
$CLIEXEC -h 192.168.4.21 -p $REDISPORT shutdown
從庫的配置（192.168.4.22，192.168.4.23） 在這兩臺機器上做以下相同的配置
[[email protected] ~]# vim /etc/redis/6379.conf
62 bind 192.168.4.22
85 port 6379
129 daemonize yes
266 slaveof 192.168.4.21 6379
[[email protected] ~]# vim /etc/init.d/redis_6379 +43
$CLIEXEC -h 192.168.4.22 -p $REDISPORT shutdown
[[email protected] ~]# vim /etc/init.d/redis_6379 +43
$CLIEXEC -h 192.168.4.23 -p $REDISPORT shutdown
4.編寫哨兵檔案 （每臺redis伺服器上都要進行以下的配置）
[[email protected] ~]# vim /etc/sentinel.conf
bind 0.0.0.0
protected-mode no
daemonize yes
port 26379
dir "/tmp"
sentinel monitor mymaster 192.168.4.21 6379 1
sentinel down-after-milliseconds mymaster 3000
sentinel failover-timeout mymaster 5000
sentinel client-reconfig-script mymaster /usr/local/bin/reconfig.sh
5.編寫浮動vip指令碼 （每臺redis伺服器上都要進行以下的配置）
[[email protected] ~]#ifconfig eth0:1 192.168.4.100/24
[[email protected] ~]# /usr/local/bin/reconfig.sh
#!/bin/bash
# mymaster leader start 192.168.1.13 6379 192.168.1.12 6379
VIP="192.168.4.100/24"
local_ip=$( /sbin/ifconfig -a|grep inet|grep -v 127.0.0.1|grep -v inet6|awk '{print $2}'|tr -d "addr:"| head -1 )
if [[ "${local_ip}" == "$4" ]];then
/usr/sbin/ifconfig eth0:1 down
elif [[ "${local_ip}" == "$6" ]];then
/usr/sbin/ifconfig eth0:1 "${VIP}"
fi
[[email protected] ~]# ls -ld /etc/sentinel.conf
-rwxrwxrwx. 1 root root 726 8月 14 14:28 /etc/sentinel.conf
[[email protected] ~]# ls -ld /usr/local/bin/reconfig.sh
-rwxrwxrwx 1 root root 358 8月 10 15:36 /usr/local/bin/reconfig.sh
6.配置完成後需要重啟動服務 (每臺伺服器都需要配置)
[[email protected] ~]# /etc/init.d/redis_6379 restart ------- 開啟redis伺服器的命令
Starting Redis server...
[[email protected] ~]# /etc/init.d/redis_6379 status ------- 關閉redis伺服器的命令
Redis is running (6531)
如果重新啟動服務的過程中遇到以下問題：
[[email protected] ～]# /etc/init.d/redis_6379 restart
Stopping ...
Could not connect to Redis at 192.168.4.21:6379: Connection refused
Waiting for Redis to shutdown ...
Waiting for Redis to shutdown ...
則需要用原有的賬號登入關掉資料庫伺服器才可以關掉
[[email protected] ～]# redis-cli shutdown
效果驗證
1.在後臺啟動哨兵（每臺redis伺服器上都要進行以下的配置）
[[email protected] redis-3.2.11]# redis-sentinel /etc/sentinel.conf
注意：如果該命令中間出現了兩個空格，則會報如下錯誤，此時只需要調整命令 redis-sentinel 一個空格/etc/sentinel.conf
[[email protected] redis-3.2.11]# redis-sentinel /etc/sentinel.conf
5153:X 10 Aug 17:08:15.748 # Fatal error, can't open config file ' /etc/sentinel.conf'
[[email protected] redis-3.2.11]# ss -utnlpn | grep redis --------檢視哨兵
tcp LISTEN 0 128 *:26379 *:* users:(("redis-sentinel",pid=5160,fd=4))
tcp LISTEN 0 128 127.0.0.1:6379 *:* uses:(("redis-server",pid=4975,fd=4))
2.主資料庫服務的效果：
[[email protected] redis]# ./bin/redis-cli -h 192.168.4.21 -p 6379
192.168.4.21:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.4.22,port=6379,state=online,offset=1733865,lag=1
slave1:ip=192.168.4.24,port=6379,state=online,offset=1734004,lag=1
master_repl_offset:1734143
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:685568
repl_backlog_histlen:1048576
[[email protected] ~]# ifconfig
eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.4.100 netmask 255.255.255.0 broadcast 192.168.4.255
ether 52:54:00:c7:66:0a txqueuelen 1000 (Ethernet)
3.從資料庫服務的效果 (只展示一臺redis資料庫伺服器的效果)
[[email protected] redis]# ./bin/redis-cli -h 192.168.4.22 -p 6379
192.168.4.22:6379> info replication
# Replication
role:slave
master_host:192.168.4.21
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:1731210
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
4.使用vip可以看到的主庫的資訊
[[email protected] redis]# ./bin/redis-cli -h 192.168.4.100 -p 6379
192.168.4.100:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.4.22,port=6379,state=online,offset=15866,lag=0
slave1:ip=192.168.4.24,port=6379,state=online,offset=15866,lag=1
master_repl_offset:15866
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:15865
測試過程
1.把主資料庫伺服器關機後可以看到的效果
[[email protected] redis]# ./bin/redis-cli -h 192.168.4.21 -p 6379
192.168.4.21:6379> shutdown
2.用vip可以看到此時主資料庫以及變化，相應的從庫資訊也變化了
[[email protected] redis]# ./bin/redis-cli -h 192.168.4.100 -p 6379
192.168.4.100:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.4.22,port=6379,state=online,offset=25921,lag=1
master_repl_offset:26060
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:26059
3.可以看到此時主資料庫伺服器是192.168.4.23上了
[[email protected] redis]# redis-cli -h 192.168.4.23 -p 6379
192.168.4.23:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.4.22,port=6379,state=online,offset=32774,lag=1
master_repl_offset:33052
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:33051
[[email protected] redis]# ifconfig
eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.4.100 netmask 255.255.255.0 broadcast 192.168.4.255
ether 52:54:00:38:6a:9c txqueuelen 1000 (Ethernet)
4.原來的主資料庫192.168.4.21修復好後重新加入集群后,將自動程式設計當前叢集的從庫
[[email protected] redis-3.2.11]# /etc/init.d/redis_6379 start
[[email protected] redis-3.2.11]# ss -untlp | grep redis
tcp LISTEN 0 128 192.168.4.21:6379 *:* users:(("redis-server",pid=1763,fd=4))
tcp LISTEN 0 128 *:26379 *:* users:(("redis-sentinel",pid=1598,fd=4))
[[email protected] redis-3.2.11]# redis-cli -h 192.168.4.21
192.168.4.21:6379> info replication
# Replication
role:slave
master_host:192.168.4.23
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:295467
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0