etcd的搭建

前言

這裡記錄下如何搭建etcd

單機

在etcd的releases中有安裝指令碼,安裝指令碼

這裡放一個docker的安裝指令碼

  1. rm -rf /tmp/etcd-data.tmp && mkdir -p /tmp/etcd-data.tmp && \
  2. docker rmi quay.io/coreos/etcd:v3.5.0 || true && \
  3. docker run \
  4. -p 2379:2379 \
  5. -p 2380:2380 \
  6. --mount type=bind,source=/tmp/etcd-data.tmp,destination=/etcd-data \
  7. --name etcd-gcr-v3.5.0 \
  8. quay.io/coreos/etcd:v3.5.0 \
  9. /usr/local/bin/etcd \
  10. --name s1 \
  11. --data-dir /etcd-data \
  12. --listen-client-urls http://0.0.0.0:2379 \
  13. --advertise-client-urls http://0.0.0.0:2379 \
  14. --listen-peer-urls http://0.0.0.0:2380 \
  15. --initial-advertise-peer-urls http://0.0.0.0:2380 \
  16. --initial-cluster s1=http://0.0.0.0:2380 \
  17. --initial-cluster-token tkn \
  18. --initial-cluster-state new \
  19. --log-level info \
  20. --logger zap \
  21. --log-outputs stderr

叢集

這裡準備了三臺centos7機器

主機 ip
etcd-1 192.168.56.111
etcd-2 192.168.56.112
etcd-3 192.168.56.113

首先在每臺機器中安裝etcd,這裡寫了安裝的指令碼

  1. $ cat etcd.sh
  2. ETCD_VER=v3.5.0
  3. # choose either URL
  4. GOOGLE_URL=https://storage.googleapis.com/etcd
  5. GITHUB_URL=https://github.com/etcd-io/etcd/releases/download
  6. DOWNLOAD_URL=${GITHUB_URL}
  7. rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
  8. rm -rf /opt/etcd && mkdir -p /opt/etcd
  9. curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
  10. tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /opt/etcd --strip-components=1
  11. rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz

賦予執行許可權

  1. $ chmod +x etcd.sh

在每臺機器中都執行下

  1. $ ./etcd.sh
  2. % Total % Received % Xferd Average Speed Time Time Time Current
  3. Dload Upload Total Spent Left Speed
  4. 100 636 100 636 0 0 1328 0 --:--:-- --:--:-- --:--:-- 1330
  5. 100 18.4M 100 18.4M 0 0 717k 0 0:00:26 0:00:26 --:--:-- 775k
  6. ...

建立etcd配置檔案

  1. $ mkdir /etc/etcd
  2. $ vi /etc/etcd/conf.yml

節點1

  1. name: etcd-1
  2. data-dir: /opt/etcd/data
  3. listen-client-urls: http://192.168.56.111:2379,http://127.0.0.1:2379
  4. advertise-client-urls: http://192.168.56.111:2379,http://127.0.0.1:2379
  5. listen-peer-urls: http://192.168.56.111:2380
  6. initial-advertise-peer-urls: http://192.168.56.111:2380
  7. initial-cluster: etcd-1=http://192.168.56.111:2380,etcd-2=http://192.168.56.112:2380,etcd-3=http://192.168.56.113:2380
  8. initial-cluster-token: etcd-cluster-token
  9. initial-cluster-state: new

節點2

  1. name: etcd-2
  2. data-dir: /opt/etcd/data
  3. listen-client-urls: http://192.168.56.112:2379,http://127.0.0.1:2379
  4. advertise-client-urls: http://192.168.56.112:2379,http://127.0.0.1:2379
  5. listen-peer-urls: http://192.168.56.112:2380
  6. initial-advertise-peer-urls: http://192.168.56.112:2380
  7. initial-cluster: etcd-1=http://192.168.56.111:2380,etcd-2=http://192.168.56.112:2380,etcd-3=http://192.168.56.113:2380
  8. initial-cluster-token: etcd-cluster-token
  9. initial-cluster-state: new

節點3

  1. name: etcd-3
  2. data-dir: /opt/etcd/data
  3. listen-client-urls: http://192.168.56.113:2379,http://127.0.0.1:2379
  4. advertise-client-urls: http://192.168.56.113:2379,http://127.0.0.1:2379
  5. listen-peer-urls: http://192.168.56.113:2380
  6. initial-advertise-peer-urls: http://192.168.56.113:2380
  7. initial-cluster: etcd-1=http://192.168.56.111:2380,etcd-2=http://192.168.56.112:2380,etcd-3=http://192.168.56.113:2380
  8. initial-cluster-token: etcd-cluster-token
  9. initial-cluster-state: new

配置項說明:

  • --name:etcd叢集中的節點名,這裡可以隨意,可區分且不重複就行

  • --listen-peer-urls:監聽的用於節點之間通訊的url,可監聽多個,叢集內部將通過這些url進行資料互動(如選舉,資料同步等)

  • --initial-advertise-peer-urls:建議用於節點之間通訊的url,節點間將以該值進行通訊

  • --listen-client-urls:監聽的用於客戶端通訊的url,同樣可以監聽多個

  • --advertise-client-urls:建議使用的客戶端通訊 url,該值用於 etcd 代理或 etcd 成員與 etcd 節點通訊

  • --initial-cluster-token: etcd-cluster-1,節點的 token 值,設定該值後集群將生成唯一 id,併為每個節點也生成唯一 id,當使用相同配置檔案再啟動一個叢集時,只要該 token 值不一樣,etcd 叢集就不會相互影響

  • --initial-cluster:也就是叢集中所有的 initial-advertise-peer-urls 的合集

  • --initial-cluster-state:new,新建叢集的標誌

更新etcd系統預設配置

當前使用的是etcd v3版本,系統預設的是v2,通過下面命令修改配置。

  1. $ vi /etc/profile
  2. # 在末尾追加
  3. export ETCDCTL_API=3
  4. # 然後更新
  5. $ source /etc/profile

啟動

  1. $ ./etcd --config-file=/etc/etcd/conf.yml

配置ETCD為啟動服務

編輯/usr/lib/systemd/system/etcd.service

  1. $ cat /usr/lib/systemd/system/etcd.service
  2. [Unit]
  3. Description=EtcdServer
  4. After=network.target
  5. After=network-online.target
  6. Wants=network-online.target
  7. [Service]
  8. Type=notify
  9. WorkingDirectory=/opt/etcd/
  10. # User=etcd
  11. ExecStart=/opt/etcd/etcd --config-file=/etc/etcd/conf.yml
  12. Restart=on-failure
  13. LimitNOFILE=65536
  14. [Install]
  15. WantedBy=multi-user.target

更新啟動:

  1. $ systemctl daemon-reload
  2. $ systemctl enable etcd
  3. $ systemctl start etcd
  4. $ systemctl restart etcd
  5. $ systemctl status etcd.service -l

測試下

複製etcd二進位制檔案到/usr/local/bin/

  1. $ cp /opt/etcd/etcd* /usr/local/bin/

首先設定ETCD_ENDPOINTS

  1. # export ETCDCTL_API=3
  2. # export ETCD_ENDPOINTS=192.168.56.111:2379,192.168.56.112:2379,192.168.56.113:2379

檢視狀態

  1. $ etcdctl --endpoints=${ETCD_ENDPOINTS} --write-out=table member list
  2. +------------------+---------+--------+----------------------------+--------------------------------------------------+------------+
  3. | ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | IS LEARNER |
  4. +------------------+---------+--------+----------------------------+--------------------------------------------------+------------+
  5. | 90d224ceb3098d7 | started | etcd-2 | http://192.168.56.112:2380 | http://127.0.0.1:2379,http://192.168.56.112:2379 | false |
  6. | 3b23fbb7d9c7cd10 | started | etcd-1 | http://192.168.56.111:2380 | http://127.0.0.1:2379,http://192.168.56.111:2379 | false |
  7. | 7909c74e3f5ffafa | started | etcd-3 | http://192.168.56.113:2380 | http://127.0.0.1:2379,http://192.168.56.113:2379 | false |
  8. +------------------+---------+--------+----------------------------+--------------------------------------------------+------------+
  9. $ etcdctl --endpoints=${ETCD_ENDPOINTS} --write-out=table endpoint health
  10. +---------------------+--------+------------+-------+
  11. | ENDPOINT | HEALTH | TOOK | ERROR |
  12. +---------------------+--------+------------+-------+
  13. | 192.168.56.111:2379 | true | 6.558088ms | |
  14. | 192.168.56.113:2379 | true | 6.543104ms | |
  15. | 192.168.56.112:2379 | true | 7.405801ms | |
  16. +---------------------+--------+------------+-------+
  17. $ etcdctl --endpoints=${ETCD_ENDPOINTS} --write-out=table endpoint status
  18. +---------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
  19. | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
  20. +---------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
  21. | 192.168.56.111:2379 | 3b23fbb7d9c7cd10 | 3.5.0 | 20 kB | true | false | 2 | 19 | 19 | |
  22. | 192.168.56.112:2379 | 90d224ceb3098d7 | 3.5.0 | 20 kB | false | false | 2 | 19 | 19 | |
  23. | 192.168.56.113:2379 | 7909c74e3f5ffafa | 3.5.0 | 20 kB | false | false | 2 | 19 | 19 | |
  24. +---------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

在etcd-1中watch一個key,然後再etcd-2中對key設定一個值

  1. [root@centos7-1 ~]# etcdctl watch test
  2. PUT
  3. test
  4. xiaoming
  5. [root@centos7-3 ~]# etcdctl put test xiaoming
  6. OK

參考

【ETCD叢集安裝配置】https://zhuanlan.zhihu.com/p/46477992

【Install】https://etcd.io/docs/v3.5/install/

【徹底搞懂 etcd 系列文章(三):etcd 叢集運維部署】https://developer.aliyun.com/article/765312