K8s叢集部署（二）------ Master節點部署
阿新 • • 發佈：2018-11-14
Master節點要部署三個服務：API Server、Scheduler、Controller Manager。
apiserver提供叢集管理的REST API介面，包括認證授權、資料校驗以 及叢集狀態變更等
只有API Server才直接操作etcd
其他模組通過API Server查詢或修改資料
提供其他模組之間的資料互動和通訊的樞紐
scheduler負責分配排程Pod到叢集內的node節點
監聽kube-apiserver，查詢還未分配Node的Pod
根據排程策略為這些Pod分配節點
controller-manager由一系列的控制器組成，它通過apiserver監控整個 叢集的狀態，並確保叢集處於預期的工作狀態
1.部署Kubernetes API服務部署
0.準備軟體包
cd /usr/local/src/kubernetes cp server/bin/kube-apiserver /opt/kubernetes/bin/ \ cp server/bin/kube-controller-manager /opt/kubernetes/bin/ \ cp server/bin/kube-scheduler /opt/kubernetes/bin/
1.建立生成CSR的 JSON 配置檔案
[[email protected] kubernetes]# cd /usr/local/src/ssl/ [[email protected]-master ssl]# vim kubernetes-csr.json { "CN": "kubernetes", "hosts": [ "127.0.0.1", "10.0.3.225", #Master IP地址 "10.1.0.1", ？？？ "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "BeiJing", "L": "BeiJing", "O": "k8s", "OU": "System" } ] }
2.生成 kubernetes 證書和私鑰
cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \ -ca-key=/opt/kubernetes/ssl/ca-key.pem \ -config=/opt/kubernetes/ssl/ca-config.json \ -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes #拷貝證書到其他節點 cp kubernetes*.pem /opt/kubernetes/ssl/ scp kubernetes*.pem 10.0.3.226:/opt/kubernetes/ssl/ scp kubernetes*.pem 10.0.3.227:/opt/kubernetes/ssl/
3.建立 kube-apiserver 使用的客戶端 token 檔案
[[email protected] ssl]# head -c 16 /dev/urandom | od -An -t x | tr -d ' ' 4c7d89749d1e1a15e5fe55eb5e8446ec [[email protected]-master ssl]# vim /opt/kubernetes/ssl/bootstrap-token.csv 4c7d89749d1e1a15e5fe55eb5e8446ec,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
4.建立基礎使用者名稱/密碼認證配置
vim /opt/kubernetes/ssl/basic-auth.csv admin,admin,1 readonly,readonly,2
5.部署Kubernetes API Server
vim /usr/lib/systemd/system/kube-apiserver.service [Unit] Description=Kubernetes API Server Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=network.target [Service] ExecStart=/opt/kubernetes/bin/kube-apiserver \ --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction \ --bind-address=10.0.3.225 \ --insecure-bind-address=127.0.0.1 \ --authorization-mode=Node,RBAC \ --runtime-config=rbac.authorization.k8s.io/v1 \ --kubelet-https=true \ --anonymous-auth=false \ --basic-auth-file=/opt/kubernetes/ssl/basic-auth.csv \ --enable-bootstrap-token-auth \ --token-auth-file=/opt/kubernetes/ssl/bootstrap-token.csv \ --service-cluster-ip-range=10.1.0.0/16 \ --service-node-port-range=20000-40000 \ --tls-cert-file=/opt/kubernetes/ssl/kubernetes.pem \ --tls-private-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \ --client-ca-file=/opt/kubernetes/ssl/ca.pem \ --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \ --etcd-cafile=/opt/kubernetes/ssl/ca.pem \ --etcd-certfile=/opt/kubernetes/ssl/kubernetes.pem \ --etcd-keyfile=/opt/kubernetes/ssl/kubernetes-key.pem \ --etcd-servers=https://10.0.3.225:2379,https://10.0.3.226:2379,https://10.0.3.227:2379 \ --enable-swagger-ui=true \ --allow-privileged=true \ --audit-log-maxage=30 \ --audit-log-maxbackup=3 \ --audit-log-maxsize=100 \ --audit-log-path=/opt/kubernetes/log/api-audit.log \ --event-ttl=1h \ --v=2 \ --logtostderr=false \ --log-dir=/opt/kubernetes/log Restart=on-failure RestartSec=5 Type=notify LimitNOFILE=65536 [Install] WantedBy=multi-user.target
6.啟動API Server服務
systemctl daemon-reload systemctl enable kube-apiserver systemctl start kube-apiserver
檢視API Server服務狀態 systemctl status kube-apiserver
[[email protected] ssl]# netstat -lntup|grep kube-apiser
tcp 0 0 10.0.3.225:6443 0.0.0.0:* LISTEN 27784/kube-apiserve
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 27784/kube-apiserve
部署ControllerManager服務
剛才安裝包已經拷貝過去了，直接配置系統服務即可。
[[email protected] ssl]# vim /usr/lib/systemd/system/kube-controller-manager.service [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] ExecStart=/opt/kubernetes/bin/kube-controller-manager \ --address=127.0.0.1 \ --master=http://127.0.0.1:8080 \ --allocate-node-cidrs=true \ --service-cluster-ip-range=10.1.0.0/16 \ --cluster-cidr=10.2.0.0/16 \ --cluster-name=kubernetes \ --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \ --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \ --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \ --root-ca-file=/opt/kubernetes/ssl/ca.pem \ --leader-elect=true \ --v=2 \ --logtostderr=false \ --log-dir=/opt/kubernetes/log Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target
啟動Controller Manager
systemctl daemon-reload systemctl enable kube-controller-manager systemctl start kube-controller-manager
#檢視狀態 systemctl status kube-controller-manager [[email protected]-master ssl]# netstat -lntup|grep kube-controll tcp 0 0 127.0.0.1:10252 0.0.0.0:* LISTEN 27899/kube-controll
部署Kubernetes Scheduler
[[email protected] ssl]# vim /usr/lib/systemd/system/kube-scheduler.service [Unit] Description=Kubernetes Scheduler Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] ExecStart=/opt/kubernetes/bin/kube-scheduler \ --address=127.0.0.1 \ --master=http://127.0.0.1:8080 \ --leader-elect=true \ --v=2 \ --logtostderr=false \ --log-dir=/opt/kubernetes/log Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target
啟動Kubernetes Scheduler
systemctl daemon-reload systemctl enable kube-scheduler systemctl start kube-scheduler #檢視服務狀態 systemctl status kube-scheduler [[email protected]-master ssl]# netstat -lntup|grep kube-schedule tcp 0 0 127.0.0.1:10251 0.0.0.0:* LISTEN 27955/kube-schedule
部署kubectl 命令列工具
kubectl是通過API Server來管理k8s叢集的，kubectl和API Server之間通訊也需要證書認證。kubectl 只在Master管理節點安裝，下面來生成證書。
1.準備二進位制命令包
[[email protected] ssl]# cd /usr/local/src/kubernetes/client/bin [[email protected]-master bin]# cp kubectl /opt/kubernetes/bin/
2.建立 admin 證書籤名請求
[[email protected] bin]# cd /usr/local/src/ssl/ [[email protected]-master ssl]# vim admin-csr.json { "CN": "admin", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "BeiJing", "L": "BeiJing", "O": "system:masters", "OU": "System" } ] }
3.生成 admin 證書和私鑰
[[email protected] ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \ > -ca-key=/opt/kubernetes/ssl/ca-key.pem \ > -config=/opt/kubernetes/ssl/ca-config.json \ > -profile=kubernetes admin-csr.json | cfssljson -bare admin 2018/11/14 10:11:02 [INFO] generate received request 2018/11/14 10:11:02 [INFO] received CSR 2018/11/14 10:11:02 [INFO] generating key: rsa-2048 2018/11/14 10:11:02 [INFO] encoded CSR 2018/11/14 10:11:03 [INFO] signed certificate with serial number 725437256018406250545228596363344942073012526422 2018/11/14 10:11:03 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for websites. For more information see the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org); specifically, section 10.2.3 ("Information Requirements"). #會生成4個檔案 [[email protected]-master ssl]# ls -l admin* -rw-r--r-- 1 root root 1009 Nov 14 10:11 admin.csr -rw-r--r-- 1 root root 229 Nov 14 10:10 admin-csr.json -rw------- 1 root root 1679 Nov 14 10:11 admin-key.pem -rw-r--r-- 1 root root 1399 Nov 14 10:11 admin.pem
#移動到ssl證書目錄
[[email protected] ssl]# mv admin*.pem /opt/kubernetes/ssl/
4.設定叢集引數
[[email protected] ssl]# kubectl config set-cluster kubernetes \ > --certificate-authority=/opt/kubernetes/ssl/ca.pem \ > --embed-certs=true \ > --server=https://10.0.3.225:6443 Cluster "kubernetes" set.
5.設定客戶端認證引數
[[email protected] ssl]# kubectl config set-credentials admin \ > --client-certificate=/opt/kubernetes/ssl/admin.pem \ > --embed-certs=true \ > --client-key=/opt/kubernetes/ssl/admin-key.pem User "admin" set.
6.設定上下文引數
[[email protected] ssl]# kubectl config set-context kubernetes \ > --cluster=kubernetes \ > --user=admin Context "kubernetes" created.
7.設定預設上下文
[[email protected] ssl]# kubectl config use-context kubernetes Switched to context "kubernetes".
#敲了一大堆命令，其實是在家目錄.kube/ 生成一個config配置檔案，kubectl和API Server通訊就要使用到這個檔案。 其他節點想要執行kubectl 就要把這個檔案拷貝過去
[email protected] ~]# cat .kube/config
8.使用kubectl工具
[[email protected]master ssl]# kubectl get cs NAME STATUS MESSAGE ERROR controller-manager Healthy ok scheduler Healthy ok etcd-1 Healthy {"health": "true"} etcd-0 Healthy {"health": "true"} etcd-2 Healthy {"health": "true"}