1. 程式人生 > >redis 3.2 cluster搭建

redis 3.2 cluster搭建

版本:redis 3.2.10 cluster
部署架構:
   192.168.65.31  M1(6379)  S2(6380)
   192.168.65.32  M2(6379)  S3(6380)
   192.168.65.33  M2(6379)  S1(6380)
部署目錄:
/apps/svr/redis3.0/bin/ #redis程式目錄
/apps/dbdat/redis_data_6379 #redis 6379資料檔案目錄
/apps/dbdat/redis_data_6380 #redis 6380資料檔案目錄
/apps/conf/redis #redis配置檔案目錄
/apps/logs/redis #redis日誌檔案目錄
1.安裝python
yum install gcc-c++
yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel
更改yum的python版本:
vi /usr/bin/yum
#!/usr/bin/python改為#!/usr/bin/python-2.6.6
將python改名:
cp /usr/bin/python /usr/bin/python-2.6.6
下載Python-2.7.13.tgz:https://www.python.org/downloads/
tar -zxvf Python-2.7.13.tgz
cd Python-2.7.13
./configure
make && make install
2.7.13預設會安裝在/usr/local/bin下,而系統自帶的2.6.6是在/usr/bin下
再刪除系統預設的 python-config 軟連結:
rm -f /usr/bin/python-config
最後建立新版本的 Python 軟連結:
cd /usr/bin
mv python python_old
ln -s /usr/local/bin/python /usr/bin/python
ln -s /usr/local/bin/python2.7-config /usr/bin/python-config
ln -s /usr/local/include/python2.7 /usr/include/python2.7
vi /etc/profile
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/apps/svr/mariadb10/lib:/usr/local/lib
source /etc/profile
檢查版本並執行python:
python -V
python
新版 Python 安裝 setuptools:setuptools-0.6c11.tar.gz
tar -zxvf setuptools-0.6c11.tar.gz
python setup.py install
easy_install-2.7放在了/usr/local/bin下
安裝pip:
easy_install pip(若能上外網)
下載安裝pip-9.0.1.tar.gz:https://pypi.python.org/pypi/pip
tar -zxvf pip-9.0.1.tar.gz
python setup.py install
pip放在了/usr/local/bin下
每臺機器: root執行   
/usr/local/bin/pip install click
/usr/local/bin/pip install Werkzeug 
/usr/local/bin/pip install retrying 
/usr/local/bin/pip install hiredis 
/usr/local/bin/pip install six 

2.安裝其它包
zlib-1.2.6.tar.gz
./configure  
make  
make install  
# ruby1.9.2  (http://www.ruby-lang.org/en/downloads/releases/)
tar -zxvf ruby-1.9.2-p0.tar.gz  
./configure -prefix=/usr/local/ruby  
make  
make install  
cp ruby /usr/local/bin
make時出現報錯:
ossl_pkey_ec.c:819: error: 'EC_GROUP_new_curve_GF2m' undeclared (first use i...)
cd /apps/software/ruby-1.9.2-p0/ext/openssl
解決方法:
vi ext/openssl/ossl_pkey_ec.c  #ext目錄放置在ruby安裝檔案的解壓目錄ruby-1.9.3-p0裡.
#在method = EC_GFp_nist_method();下一行新增 #if !defined(OPENSSL_NO_EC2M)  
#在method = EC_GF2m_simple_method();下一行新增 #endif  
--------------------------如下-----------------------------------
           if (id == s_GFp_simple) {  
               method = EC_GFp_simple_method();  
           } else if (id == s_GFp_mont) {  
               method = EC_GFp_mont_method();  
           } else if (id == s_GFp_nist) {  
               method = EC_GFp_nist_method();  
#if !defined(OPENSSL_NO_EC2M)  
           } else if (id == s_GF2m_simple) {  
               method = EC_GF2m_simple_method();  
#endif  
           }  
#在new_curve = EC_GROUP_new_curve_GFp;下一行新增 #if !defined(OPENSSL_NO_EC2M)  
#在new_curve = EC_GROUP_new_curve_GF2m;下一行新增 #endif  
-------------------------如下--------------------------------
           if (id == s_GFp) {  
               new_curve = EC_GROUP_new_curve_GFp;  
#if !defined(OPENSSL_NO_EC2M)  
           } else if (id == s_GF2m) {  
               new_curve = EC_GROUP_new_curve_GF2m;  
#endif  
           } else {  
               ossl_raise(rb_eArgError, "unknown symbol, must be :GFp or :GF2m");  
           }  


修改以上兩個地方以後,就能正常安裝了!
# rubygems-2.6.13.tgz   (https://rubygems.org/pages/download/)
cd /apps/software 
gzip -d rubygems-2.6.13.tgz
tar -xvf rubygems-2.6.13.tar 
cd rubygems-2.6.13
ruby setup.rb 
cp bin/gem /usr/local/bin 
#yum install rubygems(安裝os自帶的1.8.7)
#download地址:http://rubygems.org/gems/redis/versions/3.0.0  
https://rubygems.org/gems/redis/versions/3.2.2
gem install -l /apps/software/redis-3.2.2.gem
下載redis-trib.py-master.zip:https://github.com/Negashev/redis-trib.py(注意這個py是支援密碼的)
unzip redis-trib.py-master.zip
cd redis-trib.py-master
python setup.py install
3.redis安裝
下載redis 3.2.10:
https://redis.io/download
tar -zxvf redis-3.2.10.tar.gz
cp -r redis-3.2.10 /usr/local/redis3.2
cd /usr/local/redis3.2
make & make install
三臺機建立如下目錄:
mkdir -p /apps/dbdat/redis_data_6379
mkdir -p /apps/dbdat/redis_data_6380 
mkdir -p /apps/svr/redis3.0/bin
mkdir -p /apps/conf/redis
mkdir -p /apps/logs/redis
mkdir -p /apps/run/redis
cd /usr/local/redis3.2/src
cp mkreleasehdr.sh /apps/svr/redis3.0/bin/
cp redis-trib.rb /apps/svr/redis3.0/bin/
cp redis-server /apps/svr/redis3.0/bin/
cp redis-sentinel /apps/svr/redis3.0/bin/
cp redis-cli /apps/svr/redis3.0/bin/
cp redis-benchmark /apps/svr/redis3.0/bin/
cp redis-check-rdb /apps/svr/redis3.0/bin/
cp redis-check-aof /apps/svr/redis3.0/bin/
同樣把/apps/svr/redis3.0/bin/下的檔案copy到32和33兩臺機對應目錄下,redis3.0_cluster_6379.conf和redis3.0_cluster_6380.conf配置檔案同位放到三臺機的/apps/conf/redis下。
三臺機執行如下授權操作:
chown -R apps:apps /apps/svr
chown -R apps:apps /apps/dbdat
chown -R apps:apps /apps/conf
chown -R apps:apps /apps/logs
chown -R apps:apps /apps/run
若要更改密碼,則使用如下:
cd /apps/conf/redis/
 sed -i "s/masterauth abc/masterauth Lsredis921/g" redis*.conf
 sed -i "s/requirepass abc/requirepass Lsredis921/g" redis*.conf
cluster相關的引數:
bind 0.0.0.0
protected-mode no
cluster-enabled yes
cluster-config-file /apps/conf/redis/nodes-6379.conf
cluster-node-timeout 20000
appendonly yes
daemonize yes
cluster-require-full-coverage yes  #表示當負責一個插槽的主庫下線且沒有相應的從庫進行故障恢復時,叢集不可用,預設為yes
maxclients 10000
maxmemory 2gb
maxmemory-policy allkeys-lru
dir /apps/dbdat/redis_data_6379
port 6379
save ""
appendfilename appendonly6379.aof
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
appendfsync everysec
appendfilename appendonly6379.aof
pidfile /apps/run/redis/redis6379.pid
logfile /apps/logs/redis/redis6379.log
dbfilename dump6379.rdb
啟動redis server:
   /apps/svr/redis3.0/bin/redis-server /apps/conf/redis/redis3.0_cluster_6379.conf
   /apps/svr/redis3.0/bin/redis-server /apps/conf/redis/redis3.0_cluster_6380.conf  
4.建立叢集:(https://github.com/Negashev/redis-trib.py/blob/master/README.md)
只在任一臺主機上執行,我們這裡在192.168.65.31上執行:
 redis-trib.py create --password abc 192.168.65.31:6379  192.168.65.32:6379 192.168.65.33:6379   #  新增主庫,abc 替換為真實的密碼
Redis-trib 0.5.0 Copyright (c) HunanTV Platform developers
Redis-trib 0.0.1 by Negash
INFO:root:Instance at 192.168.65.31:6379 checked
INFO:root:Instance at 192.168.65.32:6379 checked
INFO:root:Instance at 192.168.65.33:6379 checked
INFO:root:Add 5462 slots to 192.168.65.31:6379
INFO:root:Add 5461 slots to 192.168.65.32:6379
INFO:root:Add 5461 slots to 192.168.65.33:6379
報錯:
Traceback (most recent call last):
  File "/usr/local/bin/redis-trib.py", line 5, in <module>
    from pkg_resources import load_entry_point
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2603, in <module>
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 666, in require
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 565, in resolve
pkg_resources.DistributionNotFound: click
或如下報錯:
Traceback (most recent call last):
  File "/usr/local/bin/redis-trib.py", line 5, in <module>
    from pkg_resources import load_entry_point
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2603, in <module>
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 666, in require
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 565, in resolve
pkg_resources.DistributionNotFound: six>=1.7.0
解決方法:升級setuptools
pip install -U setuptools
為每個主庫,新增從庫   
redis-trib.py replicate --password abc --master-addr 192.168.65.31:6379 --slave-addr 192.168.65.32:6380    # abc 替換為真實的密碼
redis-trib.py replicate --password abc --master-addr 192.168.65.32:6379 --slave-addr 192.168.65.33:6380    # abc 替換為真實的密碼
redis-trib.py replicate --password abc --master-addr 192.168.65.33:6379 --slave-addr 192.168.65.31:6380    # abc 替換為真實的密碼
檢視叢集狀態:
redis-trib.py list --password abc --addr  192.168.65.32:6379    # abc 替換為真實的密碼
Total 6 nodes, 3 masters, 0 fail
M  192.168.65.31:6379 master 5462
 S 192.168.65.32:6380 slave 192.168.65.31:6379
M  192.168.65.32:6379 myself,master 5461
 S 192.168.65.33:6380 slave 192.168.65.32:6379
M  192.168.65.33:6379 master 5461
 S 192.168.65.31:6380 slave 192.168.65.33:6379
其它命令檢視:cluster info、info cluster 
若需重新配置:
pkill redis-server
1.刪掉叢集配置檔案 
rm -rf /apps/conf/redis/nodes-*
2.刪掉redis資料檔案 
附cluster相關命令如下:
CLUSTER info:列印叢集的資訊。
CLUSTER nodes:列出叢集當前已知的所有節點(node)的相關資訊。
CLUSTER meet <ip> <port>:將ip和port所指定的節點新增到叢集當中。
CLUSTER addslots <slot> [slot ...]:將一個或多個槽(slot)指派(assign)給當前節點。
CLUSTER delslots <slot> [slot ...]:移除一個或多個槽對當前節點的指派。
CLUSTER slots:列出槽位、節點資訊。
CLUSTER slaves <node_id>:列出指定節點下面的從節點資訊。
CLUSTER replicate <node_id>:將當前節點設定為指定節點的從節點。
CLUSTER saveconfig:手動執行命令儲存儲存叢集的配置檔案,叢集預設在配置修改的時候會自動儲存配置檔案。
CLUSTER keyslot <key>:列出key被放置在哪個槽上。
CLUSTER flushslots:移除指派給當前節點的所有槽,讓當前節點變成一個沒有指派任何槽的節點。
CLUSTER countkeysinslot <slot>:返回槽目前包含的鍵值對數量。
CLUSTER getkeysinslot <slot> <count>:返回count個槽中的鍵。
CLUSTER setslot <slot> node <node_id> 將槽指派給指定的節點,如果槽已經指派給另一個節點,那麼先讓另一個節點刪除該槽,然後再進行指派。  
CLUSTER setslot <slot> migrating <node_id> 將本節點的槽遷移到指定的節點中。  
CLUSTER setslot <slot> importing <node_id> 從 node_id 指定的節點中匯入槽 slot 到本節點。  
CLUSTER setslot <slot> stable 取消對槽 slot 的匯入(import)或者遷移(migrate)。 
CLUSTER failover:手動進行故障轉移。
CLUSTER forget <node_id>:從叢集中移除指定的節點,這樣就無法完成握手,過期時為60s,60s後兩節點又會繼續完成握手。
CLUSTER reset [HARD|SOFT]:重置叢集資訊,soft是清空其他節點的資訊,但不修改自己的id,hard還會修改自己的id,不傳該引數則使用soft方式。
CLUSTER count-failure-reports <node_id>:列出某個節點的故障報告的長度。
CLUSTER SET-CONFIG-EPOCH:設定節點epoch,只有在節點加入叢集前才能設定。