03-創建高可用 etcd 集群
阿新 • • 發佈:2018-02-06
reload 行為 from bin 文件 ted all 客戶 新版本
本文檔記錄自己的學習歷程!
創建高可用 etcd 集群
kuberntes 系統使用 etcd 存儲所有數據,本文檔介紹部署一個三節點高可用 etcd 集群的步驟,這三個節點使用以下機器:
- 192.168.1.121
- 192.168.1.122
- 192.168.1.123
TLS 認證文件
需要為 etcd 集群創建加密通信的 TLS 證書,這裏復用之前創建的 kubernetes 證書
# cp ca.pem kubernetes-key.pem kubernetes.pem /etc/kubernetes/ssl #之前執行過cp *.pem /etc/kubernetes/ssl就忽略
- kubernetes 證書的
hosts
下載二進制文件
到 https://github.com/coreos/etcd/releases
頁面下載最新版本的二進制文件
# https://github.com/coreos/etcd/releases/download/v3.1.5/etcd-v3.1.5-linux-amd64.tar.gz
# tar -xvf etcd-v3.1.4-linux-amd64.tar.gz
# cp etcd-v3.1.4-linux-amd64/etcd* /usr/bin
創建 etcd 的 systemd unit 文件
[Unit]
Description= Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/usr/bin/etcd \
--name=${ETCD_NAME} --data-dir=${ETCD_DATA_DIR} --listen-peer-urls=${ETCD_LISTEN_PEER_URLS} --advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} --initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} --initial-cluster=${ETCD_INITIAL_CLUSTER} --initial-cluster-state=${ETCD_INITIAL_CLUSTER_STATE} --listen-client-urls=${ETCD_LISTEN_CLIENT_URLS}
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
- 指定
etcd
的工作目錄和數據目錄為/var/lib/etcd
需在啟動服務前創建這個目錄;
完整 unit 文件見:etcd.service
配置文件在/etc/etcd/etcd.conf
# [member]
ETCD_NAME=infra1
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.1.121:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.1.121:2379,http://127.0.0.1:2379"
# #[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.121:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.1.121:2379"
ETCD_INITIAL_CLUSTER="infra1=https://192.168.1.121:2380,infra2=https://192.168.1.122:2380,infra3=https://192.168.1.123:2380"
#[security]
ETCD_CERT_FILE="/etc/kubernetes/ssl/kubernetes.pem"
ETCD_KEY_FILE="/etc/kubernetes/ssl/kubernetes-key.pem"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_TRUSTED_CA_FILE="/etc/kubernetes/ssl/ca.pem"
ETCD_AUTO_TLS="true"
ETCD_PEER_CERT_FILE="/etc/kubernetes/ssl/kubernetes.pem"
ETCD_PEER_KEY_FILE="/etc/kubernetes/ssl/kubernetes-key.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/kubernetes/ssl/ca.pem"
ETCD_PEER_AUTO_TLS="true"
- 為了保證通信安全,需要指定 etcd 的公私鑰(cert-file和key-file)、Peers 通信的公私鑰和 CA 證書(peer-cert-file、peer-key-file、peer-trusted-ca-file)、客戶端的CA證書(trusted-ca-file);
- 創建
kubernetes.pem
證書時使用的kubernetes-csr.json
文件的hosts
字段包含所有 etcd 節點的IP,否則證書校驗會出錯; --initial-cluster-state
值為new
時,--name
的參數值必須位於--initial-cluster
列表中;- 這是192.168.1.121節點的配置,其他兩個etcd節點只要將上面的IP地址改成相應節點的IP地址,ETCD_NAME改成配置文件中定義的即可。
啟動 etcd 服務
# mv etcd.service /etc/systemd/system/
# systemctl daemon-reload
# systemctl enable etcd
# systemctl start etcd
# systemctl status etcd
在所有的 kubernetes master 節點重復上面的步驟,直到所有機器上的 etcd 服務都已正常啟動。
驗證服務
在任一 kubernetes master 機器上執行如下命令:
# etcdctl \
--ca-file=/etc/kubernetes/ssl/ca.pem --cert-file=/etc/kubernetes/ssl/kubernetes.pem --key-file=/etc/kubernetes/ssl/kubernetes-key.pem cluster-health
2017-07-24 15:28:43.051637 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
2017-07-24 15:28:43.052674 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
member 669bc6472fb13679 is healthy: got healthy result from https://192.168.1.121:2379
member aba9edaf5d433902 is healthy: got healthy result from https://192.168.1.122:2379
member d250ef9d0d70c7c9 is healthy: got healthy result from https://192.168.1.123:2379
cluster is healthy
結果最後一行為 cluster is healthy
時表示集群服務正常。
03-創建高可用 etcd 集群