Centos7.0上用docker下搭建codis叢集
阿新 • • 發佈：2019-01-31
1.環境
| 引數 | 值 |
|---|---|
| 宿主機ip | 10.210.234.203 |
| 宿主機核心版本 | 3.10.0-123.el7.x86_64 |
| 宿主機cpu | 4個單核cpu |
| docker版本 | 1.9.1 |
| 網路模式 | host |
| 使用者 | root |
2.執行方式
- 1.在docker容器中執行codis服務程式
- 2.在宿主機上儲存配置檔案，docker配置檔案對映到宿主機上
3.開始搭建
1.在宿主機上建立以下目錄：
mkdir -p /data1/codis/config; mkdir -p /data1/codis/logs; mkdir -p /data1/codis/pid; mkdir -p /data1/codis/out; mkdir -p /data1/codis/storage; mkdir -p /data1/codis/data;
目錄說明:
| 目錄 | 說明 |
|---|---|
| config | 配置檔案目錄 |
| logs | 日誌檔案目錄 |
| pid | pid檔案目錄 |
| assets | 存放FE檔案(js,css) |
| stroge | codis元資訊儲存目錄 |
| data | 儲存redis資料檔案 |
2.啟動docker容器
docker run -it --net=host --name=codis_container -v /data1:/data1 centos /bin/bash
(如果本地沒有centos的映象,會從遠端拉取,稍微會慢一點)
說明： --name: 容器的名稱 --net: 容器與主機的連線方式 -v /data1:/data1 目錄對映，程式配置檔案及日誌都會在這個目錄中，docker中訪問/data1目錄會直接訪問宿主機的/data1目錄
3.進入容器,下載安裝codis二進位制檔案包(這裡不使用原始碼安裝的方式)
# 解壓
tar -zxvf codis3.2.1-go1.7.6-linux.tar.gz
# 拷貝目錄到/usr/local/bin/codis目錄
mv codis3.2.1-go1.7.6-linux /usr/local/bin/codis
# 拷貝assets目錄到宿主機(這個是web介面,拷出來方便修改)在宿主機下執行
docker cp codis_container:/usr/local/bin/codis/assets /data1/codis/assets
4.啟動FE
1.進入到容器:docker exec -it codis_container
2.執行命令：
/usr/local/bin/codis/codis-fe \
--assets-dir=/data1/codis/assets \
--filesystem=/data1/codis/storage \
--log=/data1/codis/logs/fe.log \
--pidfile=/data1/codis/pid/fe.pid \
--log-level=INFO \
--listen=0.0.0.0:9090 \
>/data1/codis/out/fe.log 2>/dev/null &說明：
--assets 指定web介面所在目錄
--filesystem 以檔案的形式儲存codis叢集元資訊
--log fe日誌檔案
--pidfile pid檔案
--listen web服務埠
3.訪問：http://x.x.x.x:9090 出現以下介面則說明啟動成功,如果失敗則檢視log日誌
5.啟動dashboard
1.在宿主機/data1/codis/config目錄新增dashboard.toml檔案
2.內容如下
coordinator_name = "filesystem"
coordinator_addr = "/data1/codis/storage"
product_name = "codis-demo"
product_auth = ""
admin_addr = "0.0.0.0:18080"
migration_method = "semi-async"
migration_parallel_slots = 100
migration_async_maxbulks = 200
migration_async_maxbytes = "32mb"
migration_async_numkeys = 500
migration_timeout = "30s"
sentinel_client_timeout = "10s"
sentinel_quorum = 2
sentinel_parallel_syncs = 1
sentinel_down_after = "30s"
sentinel_failover_timeout = "5m"
sentinel_notification_script = ""
sentinel_client_reconfig_script = ""3.在codis_container中執行
/usr/local/bin/codis/codis-dashboard \
--config=/data1/codis/config/dashboard.toml \
--log=/data1/codis/logs/dashboard.log \
--log-level=INFO \
--pidfile=/data1/codis/pid/dashboard.pid &4.成功web介面中看到如下，失敗檢視日誌資訊
6.啟動proxy
1.在宿主機/data1/codis/config目錄新增proxy.toml檔案
2.新增如下內容
product_name = "codis-demo"
product_auth = ""
session_auth = ""
admin_addr = "0.0.0.0:11080"
proto_type = "tcp4"
proxy_addr = "0.0.0.0:19000"
jodis_name = ""
jodis_addr = ""
jodis_auth = ""
jodis_timeout = "20s"
jodis_compatible = false
proxy_datacenter = ""
proxy_max_clients = 1000
proxy_max_offheap_size = "1024mb"
proxy_heap_placeholder = "256mb"
backend_ping_period = "5s"
backend_recv_bufsize = "128kb"
backend_recv_timeout = "30s"
backend_send_bufsize = "128kb"
backend_send_timeout = "30s"
backend_max_pipeline = 20480
backend_primary_only = false
backend_primary_parallel = 1
backend_replica_parallel = 1
backend_keepalive_period = "75s"
backend_number_databases = 16
session_recv_bufsize = "128kb"
session_recv_timeout = "30m"
session_send_bufsize = "64kb"
session_send_timeout = "30s"
session_max_pipeline = 10000
session_keepalive_period = "75s"
session_break_on_failure = false
metrics_report_server = ""
metrics_report_period = "1s"
metrics_report_influxdb_server = ""
metrics_report_influxdb_period = "1s"
metrics_report_influxdb_username = ""
metrics_report_influxdb_password = ""
metrics_report_influxdb_database = ""
metrics_report_statsd_server = ""
metrics_report_statsd_period = "1s"
metrics_report_statsd_prefix = ""3.執行命令
/usr/local/bin/codis/codis-proxy \
--config=/data1/codis/config/proxy.toml \
--dashboard=127.0.0.1:18080 \
--log=/data1/codis/logs/proxy.log \
--pidfile=/data1/codis/pid/proxy.pid \
> /data1/codis/out/proxy.log 2>/dev/null &
4.成功看到如下資訊
7.啟動redis服務組(這裡啟動兩組)
1.在/data1/codis/config新增redis.conf檔案
2.新增內容：
protected-mode no
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
loglevel notice
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump.rdb"
dir "/data1/codis/data"
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yesgroup1:
master
/usr/local/bin/codis/codis-server /data1/codis/config/redis.conf \
--port 6379 \
--pidfile /data1/codis/pid/redis_6379.pid \
--logfile /data1/codis/logs/redis_6379.logslave1
/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port 16379 \
--slaveof 127.0.0.1 6379 \
--pidfile /data1/codis/pid/redis_1637.pid \
--logfile /data1/codis/logs/redis_16379.logslave2
/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port 26379 \
--slaveof 10.210.234.203 6379 \
--pidfile /data1/codis/pid/redis_26379.pid \
--logfile /data1/codis/logs/redis_26379.loggroup2:
master:
/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port 6380 \
--pidfile /data1/codis/pid/redis_6380.pid \
--logfile /data1/codis/logs/redis_6380.logslave1:
/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port 16380 \
--slaveof 127.0.0.1 6380 \
--pidfile /data1/codis/pid/redis_16380.pid \
--logfile /data1/codis/logs/redis_16380.logslave2
/usr/local/bin/codis/codis-server \
/data1/codis/config/redis.conf \
--port 26380 \
--slaveof 127.0.0.1 6380 \
--pidfile /data1/codis/pid/redis_26380.pid \
--logfile /data1/codis/logs/redis_26380.log7.新增分組新增到叢集中
8.新增sentinel
1.新增sentinel.conf.001到/data1/codis/config
protected-mode no
port 25379
dir /data1/codis/data
sentinel monitor mymaster1 127.0.0.1 6379 2
sentinel monitor mymaster2 127.0.0.1 6380 2
sentinel down-after-milliseconds mymaster1 30000
sentinel down-after-milliseconds mymaster2 30000
sentinel parallel-syncs mymaster1 1
sentinel parallel-syncs mymaster2 1
sentinel failover-timeout mymaster1 180000
sentinel failover-timeout mymaster2 180000
sentinel client-reconfig-script mymaster1 /data1/codis/sentinal.sh
sentinel client-reconfig-script mymaster2 /data1/codis/sentinal.sh2.新增sentinel.conf.002到/data1/codis/config
protected-mode no
port 25380
dir /data1/codis/data
sentinel monitor mymaster1 127.0.0.1 6379 2
sentinel monitor mymaster2 127.0.0.1 6380 2
sentinel down-after-milliseconds mymaster1 30000
sentinel down-after-milliseconds mymaster2 30000
sentinel parallel-syncs mymaster1 1
sentinel parallel-syncs mymaster2 1
sentinel failover-timeout mymaster1 180000
sentinel failover-timeout mymaster2 180000
sentinel client-reconfig-script mymaster1 /data1/codis/sentinal.sh
sentinel client-reconfig-script mymaster2 /data1/codis/sentinal.sh3.新增sentinel.conf.003到/data1/codis/config
protected-mode no
port 25381
dir /data1/codis/data
sentinel monitor mymaster1 127.0.0.1 6379 2
sentinel monitor mymaster2 127.0.0.1 6380 2
sentinel down-after-milliseconds mymaster1 30000
sentinel down-after-milliseconds mymaster2 30000
sentinel parallel-syncs mymaster1 1
sentinel parallel-syncs mymaster2 1
sentinel failover-timeout mymaster1 180000
sentinel failover-timeout mymaster2 180000
sentinel client-reconfig-script mymaster1 /data1/codis/sentinal.sh
sentinel client-reconfig-script mymaster2 /data1/codis/sentinal.sh4.新增到哨兵通知指令碼到/data1/codis
vi /data1/codis/sentinal.sh
新增如下內容：
#!/bin/bash
echo $1 $2 $3 $4 $5 $6 $7 >> ./notice.txt當redis故障遷移時完成時會呼叫：
格式：/data1/codis/sentinal.sh master-name role state from-ip from-port to-ip to-port
指令碼必須以 #!/bin/bash 開頭，一般會用來報警,通知客戶端更新主伺服器ip什麼的
chmod +x /data1/codis/sentinal.sh
5.啟動哨兵：
/usr/local/bin/codis/redis-sentinel /data1/codis/config/sentinel.conf.003 \
>/dev/null 2>/dev/null &/usr/local/bin/codis/redis-sentinel \
/data1/codis/config/sentinel.conf.002 \
>/dev/null 2>/dev/null &/usr/local/bin/codis/redis-sentinel \
/data1/codis/config/sentinel.conf.001 \
>/dev/null 2>/dev/null &5.新增到叢集中
9.啟動域名服務(在宿主機上安裝)
1.yum -y install bind bind-utils bind-chroot
2.vim /etc/named.conf 修改如下圖所示
3 .vi /etc/named.rfc1912.zones 新增一組域名服務
4.vim /var/named/named.service.com 新增內容
$TTL 1D
@ IN SOA @ test01.service. (
0 ; serial
1D ; refresh
1H ; retry
1W ; expire
3H ) ; minimum
@ IN NS test01.service.
test01.service. IN A 10.210.234.203
redis19000.huiyuan.comm.service. IN A 10.210.234.203
redis19001.huiyuan.comm.service. IN A 10.210.234.203
redis19001.huiyuan.activity.service. IN A 10.210.234.2035.重啟named伺服器 (如果service不存在，執行yum install initscripts )
service named restart
6.在/etc/resolv.conf檔案中新增一行
nameserver 10.210.234.203
7.ping redis19000.huiyuan.comm.service 則說明配置成功
8.此時redis服務資訊：
主機： redis19000.huiyuan.comm.service
埠： 19000
9.登入redis服務
redis-cli -h redis19000.huiyuan.comm.service -p 19000