1. 程式人生 > >Redis高可用-Cluster-測試

Redis高可用-Cluster-測試



I.高可用測試



II.關閉所有節點

ps -ef|grep redis 
kill -9 888
 

II.刪除

cd /usr/local/redis3/cluster/7111/
-rw-r--r--. 1 root root 352415 3月  29 11:42 appendonly.aof
-rw-r--r--. 1 root root   7008 3月  29 11:46 dump.rdb
-rw-r--r--. 1 root root    750 3月  29 10:53 nodes.conf
cd /usr/local/redis3/cluster/7112/
cd /usr/local/redis3/cluster/7113/
cd /usr/local/redis3/cluster/7114/
cd /usr/local/redis3/cluster/7115/
cd /usr/local/redis3/cluster/7116/


保留redis-7111.conf
#刪除redis*之外的所有檔案
rm -f !(redis*) 


II.啟動所有節點

/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7111/redis-7111.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7112/redis-7112.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7113/redis-7113.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7114/redis-7114.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7115/redis-7115.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7116/redis-7116.conf 
 檢查啟動
ps -ef|grep redis 


II.重啟啟動叢集

# cd /usr/local/src/redis-3.0.3/src/ 
# cp redis-trib.rb /usr/local/bin/redis-trib 
/usr/local/bin/redis-trib  create  --replicas  1 10.1.125.152:7111 10.1.125.89:7112 10.1.125.212:7113 10.1.125.203:7114 10.1.125.129:7115 10.1.125.180:7116


II.登入slave5 

-登入叢集必須加-c 
[[email protected] 7116]# /usr/local/redis3/bin/redis-cli -c -p 7116
127.0.0.1:7116> set name dingsai
-> Redirected to slot [5798] located at 10.1.125.89:7112
OK
10.1.125.89:7112> get name
"dingsai"


II.登入master

/usr/local/redis3/bin/redis-cli -c -p 7111
127.0.0.1:7112> get name
"dingsai"


II.檢查節點

在任意節點上執行
[[email protected] local]# /usr/local/bin/redis-trib check 10.1.125.152:7111
Connecting to node 10.1.125.152:7111: OK
Connecting to node 10.1.125.212:7113: OK
Connecting to node 10.1.125.180:7116: OK
Connecting to node 10.1.125.203:7114: OK
Connecting to node 10.1.125.129:7115: OK
Connecting to node 10.1.125.89:7112: OK
>>> Performing Cluster Check (using node 10.1.125.152:7111)
M: adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18 10.1.125.212:7113
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116
   slots: (0 slots) slave
   replicates e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18
S: 510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114
   slots: (0 slots) slave
   replicates adaebf848247181fba9d50886a8712bf2a38a9d1
S: e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115
   slots: (0 slots) slave
   replicates 4ac21be2d805505cbedee871161cf856607a29f0
M: 4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.






II.模擬使用者使用redis

 
package com.ding.study.redis;

import java.util.HashSet;
import java.util.Set;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
import redis.clients.jedis.JedisPoolConfig;

/**
 * 
 * @author daniel
 * @email [email protected]
 * @time 2016-3-29 下午3:41:31
 */
public class RedisClusterFailoverTest {
	private static final Log log = LogFactory.getLog(RedisClusterFailoverTest.class);

	public static void main(String[] args) {
		
		// 資料庫連結池配置
		JedisPoolConfig config = new JedisPoolConfig();  
        config.setMaxTotal(100);  
        config.setMaxIdle(50);  
        config.setMinIdle(20);  
        config.setMaxWaitMillis(6 * 1000);  
        config.setTestOnBorrow(true);  
		
		// Redis叢集的節點集合
		Set<HostAndPort> jedisClusterNodes = new HashSet<HostAndPort>();
		jedisClusterNodes.add(new HostAndPort("10.1.125.152", 7111));
		jedisClusterNodes.add(new HostAndPort("10.1.125.89", 7112));
		jedisClusterNodes.add(new HostAndPort("10.1.125.212", 7113));
		jedisClusterNodes.add(new HostAndPort("10.1.125.203", 7114));
		jedisClusterNodes.add(new HostAndPort("10.1.125.129", 7115));
		jedisClusterNodes.add(new HostAndPort("10.1.125.180", 7116));
		
		try {
			
			// 根據節點集創叢集連結物件
			//JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes);
			// 叢集各節點集合,超時時間,最多重定向次數,連結池
			JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes, 5000, 100, config);
			int num = 1000;
			String key = "dingsai";
			String value = "";
			int count = 1;

			while(true){
				for (int i=1; i <= num; i++){
					try {
						// 存資料
				      jedisCluster.set(key+i, "Value"+i); 
						
						// 取資料
						 value = jedisCluster.get(key+i); 
						log.info(key+i + "=" + value);
						if (value == null || "".equals(value)){
							log.error("===>break" + key+i + " value is null");
							break;
						}
					} catch (Exception e) {
						log.error("====>", e);
						Thread.sleep(3000);
						continue;
					}
					// 刪除資料
					//jedisCluster.del(key+i); 
					//value = jedisCluster.get(key+i); 
					//log.info(key+i + "=" + value);
				}
				log.info("===================================>count:" + count);
				if (value == null || "".equals(value)){
					break;
				}
				count++;
				Thread.sleep(1000);
			}
		} catch (Exception e) {
			log.error("====>", e);
		}
		
	}
}


 






II.關閉master節點10.1.125.212

[[email protected] local]# ps -ef|grep redis
root     37230     1  3 14:45 ?        00:00:43 /usr/local/redis3/bin/redis-server *:7113 [cluster]
root     38794 22391  0 15:07 pts/2    00:00:00 grep --color=auto redis
[[email protected] local]# kill -9 37230
客戶端很大的延遲,等了好幾秒才進行操作




II.再次檢視叢集狀態

127.0.0.1:7116> cluster nodes
a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116 myself,master - 0 0 7 connected 10923-16383
4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112 master - 0 1459235330419 2 connected 5461-10922
adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111 master - 0 1459235328412 1 connected 0-5460
e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115 slave 4ac21be2d805505cbedee871161cf856607a29f0 0 1459235329416 5 connected
e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18 10.1.125.212:7113 master,fail - 1459235250356 1459235244712 3 disconnected
510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114 slave adaebf848247181fba9d50886a8712bf2a38a9d1 0 1459235327407 4 connected
127.0.0.1:7116> 


[[email protected] local]# /usr/local/bin/redis-trib check 10.1.125.152:7111
Connecting to node 10.1.125.152:7111: OK
Connecting to node 10.1.125.180:7116: OK
Connecting to node 10.1.125.203:7114: OK
Connecting to node 10.1.125.129:7115: OK
Connecting to node 10.1.125.89:7112: OK
>>> Performing Cluster Check (using node 10.1.125.152:7111)
M: adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
S: 510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114
   slots: (0 slots) slave
   replicates adaebf848247181fba9d50886a8712bf2a38a9d1
S: e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115
   slots: (0 slots) slave
   replicates 4ac21be2d805505cbedee871161cf856607a29f0
M: 4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.




II.啟動master3(slave2)節點

 /usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7113/redis-7113.conf 


 

II.再次檢視叢集狀態

[[email protected] local]# /usr/local/bin/redis-trib check 10.1.125.152:7111
Connecting to node 10.1.125.152:7111: OK
Connecting to node 10.1.125.212:7113: OK
Connecting to node 10.1.125.180:7116: OK
Connecting to node 10.1.125.203:7114: OK
Connecting to node 10.1.125.129:7115: OK
Connecting to node 10.1.125.89:7112: OK
>>> Performing Cluster Check (using node 10.1.125.152:7111)
M: adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18 10.1.125.212:7113
   slots: (0 slots) slave
   replicates a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a
M: a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114
   slots: (0 slots) slave
   replicates adaebf848247181fba9d50886a8712bf2a38a9d1
S: e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115
   slots: (0 slots) slave
   replicates 4ac21be2d805505cbedee871161cf856607a29f0
M: 4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


127.0.0.1:7116> cluster nodes
a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116 myself,master - 0 0 7 connected 10923-16383
4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112 master - 0 1459235640927 2 connected 5461-10922
adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111 master - 0 1459235643946 1 connected 0-5460
e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115 slave 4ac21be2d805505cbedee871161cf856607a29f0 0 1459235641934 5 connected
e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18 10.1.125.212:7113 slave a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 0 1459235642943 7 connected
510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114 slave adaebf848247181fba9d50886a8712bf2a38a9d1 0 1459235636891 4 connected




 



I.JedisCluster 常見錯誤



II.重定向次數過多預設5次
redis.clients.jedis.exceptions.JedisClusterMaxRedirectionsException: Too many Cluster redirections  
預設超時2s,重定向5次 初始化JedisCluster 配置maxRedirections 
 // 叢集各節點集合,超時時間,最多重定向次數,連結池
 JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes, 5000, 100, config);


II.叢集不可用,高可用切換的時候可能報錯.
redis.clients.jedis.exceptions.JedisClusterException: CLUSTERDOWN The cluster is down 
 重試即可
 
II.超時 
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out 

修改超時時間