zookeeper叢集管理(1) zookeeper叢集部署
阿新 • • 發佈：2019-02-17
zookeeper是一個分散式(叢集)應用程式協調系統，具有分散式以及開源的特性，也是大資料hadoop生態中的一個基礎服務，但不去不止用於hadoop系統。本次實驗記錄zookeeper叢集的部署，擴容，縮容，基本操作以及監控，本文件將實驗結果記錄下來，作為研究檔案，供將來差缺補漏。
1. zookeeper叢集結構與伺服器資訊
測試叢集採用3個節點的虛擬機器進行構建(虛擬機器構建在openstack平臺之上)，節點資訊如下：
| 主機名稱 | 別名序號 | IP地址 | 系統 | 角色 |
|---|---|---|---|---|
| zookeeper-node-1 | server.1 | CentOS7 | 10.120.67.19 | zookeeper |
| zookeeper-node-2 | server.2 | CentOS7 | 10.120.67.20 | zookeeper |
| zookeeper-node-3 | server.3 | CentOS7 | 10.120.67.21 | zookeeper |
叢集部署採用3個節點，之後會在建立2個節點用於擴容，縮容測試
2. 伺服器準備工作
(1) 關閉selinux 和 firewalld(iptables)
這裡便於測試將selinux與firewalld直接關閉服務，實際產線中一般會關閉selinux，firewalld根據實際需要配置（如果在內網裡也可以直接關閉，實際情況很多組織都是這麼做的）
~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
~]# setenforce 0
~]# systemctl stop firewalld.service
~]# systemctl disable firewalld.service(2) 配置主機名稱與/etc/hosts檔案
zookeeper叢集的3臺伺服器標號從1到3，每臺主機的名稱命名規範如下
~]# hostnamectl set-hostname zookeeper-node-${編號}3個節點的hosts檔案配置如下(其實沒有用到這個，配置檔案直接配置的IP地址)
~]# cat /etc/hosts
...
10.120.67.19 zookeeper-node-1
10.120.67.20 zookeeper-node-2
10.120.67.21 zookeeper-node-3(3) Linux系統引數調整
沒有去研究zookeeper系統的核心調優，這裡將所在組織虛擬機器模板中的引數直接使用，先用著，不合適再進行調整
~]# cat /etc/sysctl.conf
kernel.sysrq = 0
kernel.core_uses_pid = 1
kernel.msgmax = 1048560
kernel.msgmnb = 1073741824
kernel.shmall = 4294967296
kernel.shmmax = 68719476736
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.wmem_max = 16777216
net.core.rmem_max = 16777216
net.core.somaxconn = 256
net.core.optmem_max = 20480
net.core.somaxconn = 32768
net.core.netdev_max_backlog = 32768
net.ipv4.tcp_wmem = 8192 436600 16777216
net.ipv4.tcp_rmem = 32768 436600 16777216
net.ipv4.tcp_mem = 786432 1048576 1572864
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_retries2 = 5
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_syn_retries = 3
net.ipv4.tcp_keepalive_time = 1200
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.tcp_max_syn_backlog = 65536
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.lo.arp_ignore = 0
net.ipv4.conf.lo.arp_announce = 0
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_announce = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.ip_local_port_range = 1024 65000
vm.swappiness = 0
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1(4) 節點NTP時鐘同步
我的測試叢集將10.120.67.9伺服器作為時鐘源，zookeeper叢集配置如下（所有節點）
~]# vi /etc/chrony.conf
server 10.120.67.9 iburst
~]# systemctl restart chronyd.service
~]# timedatectl
...
NTP enabled: yes
NTP synchronized: yes
~]# chronyc sources
210 Number of sources = 1
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 10.120.67.9 2 6 17 19 -25us[ -113us] +/- 104ms(5)所有節點安裝JDK
這裡採用官方網站下載JDK檔案，解壓安裝的方式安裝。檔案：jdk-8u172-linux-x64.tar.gz，所有節點統一部署配置，下載地址:JDK
~]# mkdir /usr/java
~]# tar zxvf jdk-8u172-linux-x64.tar.gz
~]# mv jdk1.8.0_172 /usr/java/
~]# vi /etc/profile #[最後追加java環境變數]
export JAVA_HOME=/usr/java/jdk1.8.0_172
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
~]# source /etc/profile
~]# java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)3. 安裝配置zookeeper叢集
下載zookeeper程式，下載地址：zookeeper
~]# tar zxvf zookeeper-3.4.12.tar.gz
~]# mv zookeeper-3.4.12 /opt/app/install/
~]# ln -s /opt/app/install/zookeeper-3.4.12 /opt/app/zookeeper
~]# ll /opt/app/
lrwxrwxrwx 1 root root 33 May 22 18:38 zookeeper -> /opt/app/install/zookeeper-3.4.12
# 配置(3個節點一樣的配置)
~]# cd /opt/app/zookeeper/conf
conf]# mkdir -p /data/zookeeper
conf]# cp zoo_sample.cfg zoo.cfg
conf]# vi zoo.cfg #此處將註釋行與空行省略掉了
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/zookeeper
clientPort=2181
server.1=10.120.67.19:2888:3888
server.2=10.120.67.20:2888:3888
server.3=10.120.67.21:2888:3888
# 配置資料目錄以及myid檔案 [各個節點分別對應執行]
zookeeper-node-1 ~]# echo 1 > /data/zookeeper/myid
zookeeper-node-2 ~]# echo 2 > /data/zookeeper/myid
zookeeper-node-3 ~]# echo 3 > /data/zookeeper/myid4. 啟動zookeeper叢集
在節點1（zookeeper-node-1）上執行服務啟動
~]# /opt/app/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/app/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
~]# /opt/app/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/app/zookeeper/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
# 【檢視日誌 zookeeper.out檔案可以瞭解到時因為其他2個節點沒有起來導致的。依次啟動剩餘節點即可】
//zookeeper-node-2
~]# /opt/app/zookeeper/bin/zkServer.sh start
~]# /opt/app/zookeeper/bin/zkServer.sh status
...
Mode: leader
//zookeeper-node-3
~]# /opt/app/zookeeper/bin/zkServer.sh start
~]# /opt/app/zookeeper/bin/zkServer.sh status
...
Mode: follower
//zookeeper-node-1 #【回頭再看下節點1 已經正常了】
~]# /opt/app/zookeeper/bin/zkServer.sh status
...
Mode: follower5. 服務使用測試
~]# /opt/app/zookeeper/bin/zkCli.sh -server 10.120.67.19:2181
[zk: 10.120.67.19:2181(CONNECTED) 0] ls /
[zookeeper]
[zk: 10.120.67.19:2181(CONNECTED) 1] create /demo 'test'
Created /demo
[zk: 10.120.67.19:2181(CONNECTED) 2] ls /
[zookeeper, demo]
~]# /opt/app/zookeeper/bin/zkCli.sh -server 10.120.67.20:2181
[zk: 10.120.67.20:2181(CONNECTED) 0] ls /
[zookeeper, demo]如上，就完成了一個3節點的zookeeper叢集的配置，可以提供服務了。