kubeadm安裝高可用kubernetes v1.14.1

阿新 • • 發佈：2019-05-17

root rec nload 環境 ret token reat iptable wall

前言

步驟跟之前安裝1.13版本的是一樣的
區別就在於kubeadm init的configuration file
目前kubeadm init with configuration file已經處於beta階段了,在1.15版本已經進入到了v1beta2版本
雖然還沒到GA版，但是相對於手動配置k8s集群，kubeadm不但簡化了步驟，而且還減少了手動部署的出錯的概率，何樂而不為呢

環境介紹：

系統版本：CentOS 7.6
內核：4.18.7-1.el7.elrepo.x86_64
Kubernetes: v1.14.1
Docker-ce: 18.09

Keepalived保證apiserever服務器的IP高可用
Haproxy實現apiserver的負載均衡
master x3 && etcd x3 保證k8s集群可用性

192.168.1.1        master
192.168.1.2        master2
192.168.1.3        master3
192.168.1.4        Keepalived + Haproxy
192.168.1.5        Keepalived + Haproxy
192.168.1.6        etcd1
192.168.1.7        etcd2
192.168.1.8        etcd3
192.168.1.9        node1
192.168.1.10       node2
192.168.1.100      VIP、apiserver的地址

一、準備工作

為方便操作，所有操作均以root用戶執行
以下操作僅在kubernetes集群節點執行即可

關閉selinux和防火墻

sed -ri ‘s#(SELINUX=).*#\1disabled#‘ /etc/selinux/config
setenforce 0

systemctl disable firewalld
systemctl stop firewalld

關閉swap

swapoff -a

配置轉發相關參數，否則可能會出錯

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
#vm.swappiness=0
EOF

sysctl --system

加載ipvs模塊

cat << EOF > /etc/sysconfig/modules/ipvs.modules 
#!/bin/bash
ipvs_modules_dir="/usr/lib/modules/\`uname -r\`/kernel/net/netfilter/ipvs"
for i in \`ls \$ipvs_modules_dir | sed  -r ‘s#(.*).ko.*#\1#‘\`; do
    /sbin/modinfo -F filename \$i  &> /dev/null
    if [ \$? -eq 0 ]; then
        /sbin/modprobe \$i
    fi
done
EOF

chmod +x /etc/sysconfig/modules/ipvs.modules 
bash /etc/sysconfig/modules/ipvs.modules

安裝cfssl

#在master節點安裝即可！！！

wget -O /bin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget -O /bin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget -O /bin/cfssl-certinfo  https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
for cfssl in `ls /bin/cfssl*`;do chmod +x $cfssl;done;

安裝kubernetes阿裏雲鏡像

cat << EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum install -y  kubelet-1.14.1 kubeadm-1.14.1 kubectl-1.14.1

安裝docker，並幹掉docker0網橋

wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum install -y docker-ce

mkdir /etc/docker/
cat << EOF > /etc/docker/daemon.json
{   
    "exec-opts": ["native.cgroupdriver=systemd"],
    "registry-mirrors": ["https://registry.docker-cn.com"],
    "live-restore": true,
    "default-shm-size": "128M",
    "bridge": "none",
    "max-concurrent-downloads": 10,
    "oom-score-adjust": -1000,
    "debug": false
}   
EOF 

#重啟docker
systemctl daemon-reload
systemctl enable docker
systemctl restart docker

配置hosts文件

#為所有節點配置hosts文件

192.168.1.1    master
192.168.1.2    master2
192.168.1.3    master3
192.168.1.4    lb1
192.168.1.5    lb2
192.168.1.6    etcd1
192.168.1.7    etcd2
192.168.1.8    etcd3
192.168.1.9    node1
192.168.1.10   node2

二、配置etcd

配置etcd的證書

mkdir -pv $HOME/ssl && cd $HOME/ssl

cat << EOF > ca-config.json
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "kubernetes": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "87600h"
      }
    }
  }
}
EOF

cat << EOF > etcd-ca-csr.json
{
  "CN": "etcd",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Shenzhen",
      "L": "Shenzhen",
      "O": "etcd",
      "OU": "Etcd Security"
    }
  ]
}
EOF

cat << EOF > etcd-csr.json
{
    "CN": "etcd",
    "hosts": [
      "127.0.0.1",
      "192.168.1.6",
      "192.168.1.7",
      "192.168.1.8"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "Shenzhen",
            "L": "Shenzhen",
            "O": "etcd",
            "OU": "Etcd Security"
        }
    ]
}
EOF

#生成證書並復制證書至其他etcd節點

cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare etcd-ca
cfssl gencert -ca=etcd-ca.pem -ca-key=etcd-ca-key.pem -config=ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd

mkdir -pv /etc/etcd/ssl
cp etcd*.pem /etc/etcd/ssl
mkdir -pv /etc/kubernetes/pki/etcd
cp etcd*.pem /etc/kubernetes/pki/etcd

scp -r /etc/etcd 192.168.1.6:/etc/
scp -r /etc/etcd 192.168.1.7:/etc/
scp -r /etc/etcd 192.168.1.8:/etc/

etcd1主機啟動etcd

yum install -y etcd 

cat << EOF > /etc/etcd/etcd.conf
#[Member]
#ETCD_CORS=""
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
ETCD_LISTEN_PEER_URLS="https://192.168.1.6:2380"
ETCD_LISTEN_CLIENT_URLS="https://127.0.0.1:2379,https://192.168.1.6:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
ETCD_NAME="etcd1"
#ETCD_SNAPSHOT_COUNT="100000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_QUOTA_BACKEND_BYTES="0"
#ETCD_MAX_REQUEST_BYTES="1572864"
#ETCD_GRPC_KEEPALIVE_MIN_TIME="5s"
#ETCD_GRPC_KEEPALIVE_INTERVAL="2h0m0s"
#ETCD_GRPC_KEEPALIVE_TIMEOUT="20s"
#
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.6:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://127.0.0.1:2379,https://192.168.1.6:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_DISCOVERY_SRV=""
ETCD_INITIAL_CLUSTER="etcd1=https://192.168.1.6:2380,etcd2=https://192.168.1.7:2380,etcd3=https://192.168.1.8:2380"
ETCD_INITIAL_CLUSTER_TOKEN="BigBoss"
#ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_STRICT_RECONFIG_CHECK="true"
#ETCD_ENABLE_V2="true"
#
#[Proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[Security]
ETCD_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
#ETCD_CLIENT_CERT_AUTH="false"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-ca.pem"
#ETCD_AUTO_TLS="false"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
#ETCD_PEER_CLIENT_CERT_AUTH="false"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-ca.pem"
#ETCD_PEER_AUTO_TLS="false"
#
#[Logging]
#ETCD_DEBUG="false"
#ETCD_LOG_PACKAGE_LEVELS=""
#ETCD_LOG_OUTPUT="default"
#
#[Unsafe]
#ETCD_FORCE_NEW_CLUSTER="false"
#
#[Version]
#ETCD_VERSION="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[Profiling]
#ETCD_ENABLE_PPROF="false"
#ETCD_METRICS="basic"
#
#[Auth]
#ETCD_AUTH_TOKEN="simple"
EOF

chown -R etcd.etcd /etc/etcd
systemctl enable etcd
systemctl start etcd

etcd2主機啟動etcd

yum install -y etcd 

cat << EOF > /etc/etcd/etcd.conf
#[Member]
#ETCD_CORS=""
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
ETCD_LISTEN_PEER_URLS="https://192.168.1.7:2380"
ETCD_LISTEN_CLIENT_URLS="https://127.0.0.1:2379,https://192.168.1.7:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
ETCD_NAME="etcd2"
#ETCD_SNAPSHOT_COUNT="100000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_QUOTA_BACKEND_BYTES="0"
#ETCD_MAX_REQUEST_BYTES="1572864"
#ETCD_GRPC_KEEPALIVE_MIN_TIME="5s"
#ETCD_GRPC_KEEPALIVE_INTERVAL="2h0m0s"
#ETCD_GRPC_KEEPALIVE_TIMEOUT="20s"
#
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.7:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://127.0.0.1:2379,https://192.168.1.7:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_DISCOVERY_SRV=""
ETCD_INITIAL_CLUSTER="etcd1=https://192.168.1.6:2380,etcd2=https://192.168.1.7:2380,etcd3=https://192.168.1.8:2380"
ETCD_INITIAL_CLUSTER_TOKEN="BigBoss"
#ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_STRICT_RECONFIG_CHECK="true"
#ETCD_ENABLE_V2="true"
#
#[Proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[Security]
ETCD_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
#ETCD_CLIENT_CERT_AUTH="false"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-ca.pem"
#ETCD_AUTO_TLS="false"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
#ETCD_PEER_CLIENT_CERT_AUTH="false"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-ca.pem"
#ETCD_PEER_AUTO_TLS="false"
#
#[Logging]
#ETCD_DEBUG="false"
#ETCD_LOG_PACKAGE_LEVELS=""
#ETCD_LOG_OUTPUT="default"
#
#[Unsafe]
#ETCD_FORCE_NEW_CLUSTER="false"
#
#[Version]
#ETCD_VERSION="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[Profiling]
#ETCD_ENABLE_PPROF="false"
#ETCD_METRICS="basic"
#
#[Auth]
#ETCD_AUTH_TOKEN="simple"
EOF

chown -R etcd.etcd /etc/etcd
systemctl enable etcd
systemctl start etcd

etcd3主機啟動etcd

yum install -y etcd 

cat << EOF > /etc/etcd/etcd.conf
#[Member]
#ETCD_CORS=""
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
ETCD_LISTEN_PEER_URLS="https://192.168.1.8:2380"
ETCD_LISTEN_CLIENT_URLS="https://127.0.0.1:2379,https://192.168.1.8:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
ETCD_NAME="etcd3"
#ETCD_SNAPSHOT_COUNT="100000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_QUOTA_BACKEND_BYTES="0"
#ETCD_MAX_REQUEST_BYTES="1572864"
#ETCD_GRPC_KEEPALIVE_MIN_TIME="5s"
#ETCD_GRPC_KEEPALIVE_INTERVAL="2h0m0s"
#ETCD_GRPC_KEEPALIVE_TIMEOUT="20s"
#
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.8:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://127.0.0.1:2379,https://192.168.1.8:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_DISCOVERY_SRV=""
ETCD_INITIAL_CLUSTER="etcd1=https://192.168.1.6:2380,etcd2=https://192.168.1.7:2380,etcd3=https://192.168.1.8:2380"
ETCD_INITIAL_CLUSTER_TOKEN="BigBoss"
#ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_STRICT_RECONFIG_CHECK="true"
#ETCD_ENABLE_V2="true"
#
#[Proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[Security]
ETCD_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
#ETCD_CLIENT_CERT_AUTH="false"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-ca.pem"
#ETCD_AUTO_TLS="false"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
#ETCD_PEER_CLIENT_CERT_AUTH="false"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-ca.pem"
#ETCD_PEER_AUTO_TLS="false"
#
#[Logging]
#ETCD_DEBUG="false"
#ETCD_LOG_PACKAGE_LEVELS=""
#ETCD_LOG_OUTPUT="default"
#
#[Unsafe]
#ETCD_FORCE_NEW_CLUSTER="false"
#
#[Version]
#ETCD_VERSION="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[Profiling]
#ETCD_ENABLE_PPROF="false"
#ETCD_METRICS="basic"
#
#[Auth]
#ETCD_AUTH_TOKEN="simple"
EOF

chown -R etcd.etcd /etc/etcd
systemctl enable etcd
systemctl start etcd

檢查etcd集群

etcdctl --endpoints "https://192.168.1.6:2379,https://192.168.1.7:2379,https://192.168.1.8:2379"   --ca-file=/etc/etcd/ssl/etcd-ca.pem  --cert-file=/etc/etcd/ssl/etcd.pem   --key-file=/etc/etcd/ssl/etcd-key.pem   cluster-health

[[email protected] ~]# etcdctl --endpoints "https://192.168.1.6:2379,https://192.168.1.7:2379,https://192.168.1.8:2379"   --ca-file=/etc/etcd/ssl/etcd-ca.pem  > --cert-file=/etc/etcd/ssl/etcd.pem   --key-file=/etc/etcd/ssl/etcd-key.pem   cluster-health
member 3639deb1869a1bda is healthy: got healthy result from https://127.0.0.1:2379
member b75e13f1faa57bd8 is healthy: got healthy result from https://127.0.0.1:2379
member e31fec5bb4c882f2 is healthy: got healthy result from https://127.0.0.1:2379

配置keepalived

在lb1機器上配置

yum install -y keepalived

cat << EOF > /etc/keepalived/keepalived.conf
! Configuration File for keepalived
    global_defs {
        notification_email {
            [email protected]      #發送郵箱
        }
        notification_email_from [email protected]    #郵箱地址   
        smtp_server 127.0.0.1   #郵件服務器地址
        smtp_connect_timeout 30 
        router_id node1         #主機名，每個節點不同即可
        vrrp_mcast_group4 224.0.100.100    #組播地址
    }       

vrrp_instance VI_1 {
    state MASTER        #在另一個節點上為BACKUP
    interface eth0      #IP地址漂移到的網卡
    virtual_router_id 6 #多個節點必須相同
    priority 100        #優先級，備用節點的值必須低於主節點的值
    advert_int 1        #通告間隔1秒
    authentication {
        auth_type PASS      #預共享密鑰認證
        auth_pass 571f97b2  #密鑰
    }
    virtual_ipaddress {
        192.168.1.100/24    #VIP地址
    }
}   
EOF

systemctl enable keepalived
systemctl start keepalived

在lb2主機配置

yum install -y keepalived

cat << EOF > /etc/keepalived/keepalived.conf
! Configuration File for keepalived
    global_defs {
        notification_email {
            [email protected]      #發送郵箱
        }
        notification_email_from [email protected]    #郵箱地址   
        smtp_server 127.0.0.1   #郵件服務器地址
        smtp_connect_timeout 30 
        router_id node2         #主機名，每個節點不同即可
        vrrp_mcast_group4 224.0.100.100  #組播地址
    }       

vrrp_instance VI_1 {
    state BACKUP        #在另一個節點上為MASTER
    interface eth0      #IP地址漂移到的網卡
    virtual_router_id 6 #多個節點必須相同
    priority 80        #優先級，備用節點的值必須低於主節點的值
    advert_int 1        #通告間隔1秒
    authentication {
        auth_type PASS      #預共享密鑰認證
        auth_pass 571f97b2  #密鑰
    }
    virtual_ipaddress {
        192.168.1.100/24    #漂移過來的IP地址
    }
}   
EOF

systemctl enable keepalived
systemctl start keepalived

配置Haproxy

在lb1主機上

yum install  -y haproxy

cat << EOF > /etc/haproxy/haproxy.cfg
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

defaults
    mode                    tcp
    log                     global
    retries                 3
    timeout connect         10s
    timeout client          1m
    timeout server          1m

frontend kubernetes
    bind *:6443
    mode tcp
    default_backend kubernetes-master

backend kubernetes-master
    balance roundrobin
    server master  192.168.1.1:6443 check maxconn 2000
    server master2 192.168.1.2:6443 check maxconn 2000
    server master3 192.168.1.3:6443 check maxconn 2000
EOF

systemctl enable haproxy
systemctl start haproxy

在lb2主機上

yum install  -y haproxy

cat << EOF > /etc/haproxy/haproxy.cfg
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

defaults
    mode                    tcp
    log                     global
    retries                 3
    timeout connect         10s
    timeout client          1m
    timeout server          1m

frontend kubernetes
    bind *:6443
    mode tcp
    default_backend kubernetes-master

backend kubernetes-master
    balance roundrobin
    server master  192.168.1.1:6443 check maxconn 2000
    server master2 192.168.1.2:6443 check maxconn 2000
    server master3 192.168.1.3:6443 check maxconn 2000
EOF

systemctl enable haproxy
systemctl start haproxy

初始化master

初始化master1

#kubeadm init配置文件參考：https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file

cd $HOME
cat << EOF > /root/kubeadm-init.yaml
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
# 本地的api server監聽地址和端口
localAPIEndpoint:
  advertiseAddress: 192.168.1.1
  bindPort: 6443
# 本節點加入集群的註冊信息,也就是kubectl get node看到的信息
nodeRegistration:
# 如果不填寫name字段，則默認使用主機名，名字最好是是集群唯一
#  name: master1
  criSocket: /var/run/dockershim.sock
# 汙點，NoSchedule表示不調度Pod到這臺node上
# 詳細信息參考：https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
# 集群名字
clusterName: kubernetes
# controller訪問api server的地址
# 如果是多個master的集群，這裏就要寫前端lb的地址
controlPlaneEndpoint: "192.168.1.100:6443"
apiServer:
# 此處填所有的masterip和lbip和其它你可能需要通過它訪問apiserver的地址和域名或者主機名等  
  certSANs:
  - "master"
  - "master2"
  - "master3"
  - "192.168.1.1"
  - "192.168.1.2"
  - "192.168.1.3"
  - "192.168.1.4"
  - "192.168.1.5"
  - "192.168.1.100"
  - "127.0.0.1"

  timeoutForControlPlane: 4m0s
certificatesDir: /etc/kubernetes/pki
dns:
  type: CoreDNS
etcd:
# 讓k8s自行啟動etcd，多master的集群必須使用external
#  local:
#    imageRepository: "k8s.gcr.io"
#    dataDir: "/var/lib/etcd"
# 外部的etcd，所有的api server都要連接
  external:
    endpoints:
    - "https://192.168.1.6:2379"
    - "https://192.168.1.7:2379"
    - "https://192.168.1.8:2379"
    caFile: "/etc/kubernetes/pki/etcd/etcd-ca.pem"
    certFile: "/etc/kubernetes/pki/etcd/etcd.pem"
    keyFile: "/etc/kubernetes/pki/etcd/etcd-key.pem"
imageRepository: k8s.gcr.io
kubernetesVersion: v1.14.1
networking:
# service網段
  serviceSubnet: "10.96.0.0/12"
# pod網絡網段
  podSubnet: "10.100.0.1/24"
  dnsDomain: "cluster.local"
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: "systemd"
# 如果swap啟用了,是否將kubelet判定為啟動失敗
failSwapOn: false
EOF

systemctl enable kubelet
kubeadm config images pull --config kubeadm-init.yaml
kubeadm init --config /root/kubeadm-init.yaml

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

cat << EOF > /etc/profile.d/kubernetes.sh 
source <(kubectl completion bash)
EOF
source /etc/profile.d/kubernetes.sh 

scp -r /etc/kubernetes/pki 192.168.1.2:/etc/kubernetes/
scp -r /etc/kubernetes/pki 192.168.1.3:/etc/kubernetes/

初始化master2

cd /etc/kubernetes/pki/
rm -fr apiserver.crt apiserver.key
cd $HOME
cat << EOF > /root/kubeadm-init.yaml
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.1.2
  bindPort: 6443
nodeRegistration:
#  name: master1
  criSocket: /var/run/dockershim.sock
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
clusterName: kubernetes
controlPlaneEndpoint: "192.168.1.100:6443"
apiServer:
  certSANs:
  - "master"
  - "master2"
  - "master3"
  - "192.168.1.1"
  - "192.168.1.2"
  - "192.168.1.3"
  - "192.168.1.4"
  - "192.168.1.5"
  - "192.168.1.100"
  - "127.0.0.1"

  timeoutForControlPlane: 4m0s
certificatesDir: /etc/kubernetes/pki
dns:
  type: CoreDNS
etcd:
  external:
    endpoints:
    - "https://192.168.1.6:2379"
    - "https://192.168.1.7:2379"
    - "https://192.168.1.8:2379"
    caFile: "/etc/kubernetes/pki/etcd/etcd-ca.pem"
    certFile: "/etc/kubernetes/pki/etcd/etcd.pem"
    keyFile: "/etc/kubernetes/pki/etcd/etcd-key.pem"
imageRepository: k8s.gcr.io
kubernetesVersion: v1.14.1
networking:
  serviceSubnet: "10.96.0.0/12"
  podSubnet: "10.100.0.1/24"
  dnsDomain: "cluster.local"
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: "systemd"
failSwapOn: false
EOF

systemctl enable kubelet
kubeadm config images pull --config kubeadm-init.yaml
kubeadm init --config /root/kubeadm-init.yaml

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

cat << EOF > /etc/profile.d/kubernetes.sh 
source <(kubectl completion bash)
EOF
source /etc/profile.d/kubernetes.sh

初始化master3

cd /etc/kubernetes/pki/
rm -fr apiserver.crt apiserver.key
cd $HOME
cat << EOF > /root/kubeadm-init.yaml
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.1.3
  bindPort: 6443
nodeRegistration:
#  name: master1
  criSocket: /var/run/dockershim.sock
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
clusterName: kubernetes
controlPlaneEndpoint: "192.168.1.100:6443"
apiServer:
  certSANs:
  - "master"
  - "master2"
  - "master3"
  - "192.168.1.1"
  - "192.168.1.2"
  - "192.168.1.3"
  - "192.168.1.4"
  - "192.168.1.5"
  - "192.168.1.100"
  - "127.0.0.1"

  timeoutForControlPlane: 4m0s
certificatesDir: /etc/kubernetes/pki
dns:
  type: CoreDNS
etcd:
  external:
    endpoints:
    - "https://192.168.1.6:2379"
    - "https://192.168.1.7:2379"
    - "https://192.168.1.8:2379"
    caFile: "/etc/kubernetes/pki/etcd/etcd-ca.pem"
    certFile: "/etc/kubernetes/pki/etcd/etcd.pem"
    keyFile: "/etc/kubernetes/pki/etcd/etcd-key.pem"
imageRepository: k8s.gcr.io
kubernetesVersion: v1.14.1
networking:
  serviceSubnet: "10.96.0.0/12"
  podSubnet: "10.100.0.1/24"
  dnsDomain: "cluster.local"
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: "systemd"
failSwapOn: false
EOF

systemctl enable kubelet
kubeadm config images pull --config kubeadm-init.yaml
kubeadm init --config /root/kubeadm-init.yaml

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# 配置kubectl命令提示
cat << EOF > /etc/profile.d/kubernetes.sh 
source <(kubectl completion bash)
EOF
source /etc/profile.d/kubernetes.sh

將所有node節點加入集群

獲取加入集群的token

#在master主機執行獲取join命令
kubeadm token create --print-join-command

[[email protected] ~]# kubeadm token create --print-join-command
kubeadm join 192.168.1.100:6443 --token zpru0r.jkvrdyy2caexr8kk --discovery-token-ca-cert-hash sha256:a45c091dbd8a801152aacd877bcaaaaf152697bfa4536272c905a83612b3bf22

其他的步驟參考：kubeadm安裝kubernetes v1.11.3 HA多主高可用並啟用ipvs

技術分享圖片

kubeadm安裝高可用kubernetes v1.14.1

root rec nload 環境 ret token reat iptable wall 前言步驟跟之前安裝1.13版本的是一樣的區別就在於kubeadm init的configuration file 目前kubeadm init with configura

kubernetes實戰(二十六)：kubeadm 安裝高可用 k8s v1.16.x dashboard 2.x

1、基本配置　　基本配置、核心升級、基本服務安裝參考https://www.cnblogs.com/dukuan/p/10278637.html，或者參考《再也不踩坑的Kubernetes實戰指南》第一章第一節 2、Kubernetes元件安裝　　所有節點安裝Kubeadm、Kubectl、

Kubeadm建立高可用Kubernetes v1.12.0叢集 (轉)

節點規劃主機名 IP Role k8s-master01 10.3.1.20 etcd、Master、Node、keepalived k8s-master02 10.3.1.21 etcd、Master、Node、keepa

kubeadm創建高可用kubernetes v1.12.0集群

架構 r.js 慎用 led validate block boot ade 組件通信節點規劃主機名 IP Role k8s-master01 10.3.1.20 etcd、Master、Node、keepalived k8s-master02 10.

用kubeadm建立高可用kubernetes集群后，如何重新新增控制平面

叢集資訊叢集版本：1.13.1 3個控制平面，2個worker節點 k8s-001：10.0.3.4 k8s-002：10.0.3.5 k8s-003：10.0.3.6 k8s-004：10.0.3.7 k8s-005：10.0.3.8 vip：：10.0.3.9 場景出於某些原因刪除了k8s-001

kubeadm部署高可用叢集Kubernetes 1.14.1版本

Kubernetes高可用叢集部署部署架構： Master 元件： kube-apiserver Kubernetes API，叢集的統一入口，各元件協調者，以HTTP API提供介面服務，所有物件資源的增刪改查和監聽操作都交給APIServer處理後再提交給Etcd儲存。

使用kubeadm安裝安全高可用kubernetes叢集

安裝包地址如非高可用安裝請忽略此教程，直接看產品頁的三步安裝。單個master流程：單master視訊教程解壓後在master 上 cd shell && sh init.sh ,然後sh master.sh（注意因為指令碼用的相

Kubernetes v1.9.1 單機版本一鍵安裝腳本

k8s kubernetes etcd flanneld #!/bin/bash # ---------------------------------------- # kubernetes v1.9.1 單機一鍵部署腳本 # 用於實驗環境 # CentOS 7.2.1511下測試OK #

使用kubeadm快速部署Kubernetes(v1.12.1)叢集---來源：馬哥教育馬哥原創

使用kubeadm快速部署Kubernetes(v1.12.1)叢集------來源：馬哥教育馬哥原創 Kubernetes技術已經成為了原生雲技術的事實標準，它是目前基礎軟體領域最為熱門的分散式排程和管理平臺。於是，Kubernetes也幾乎成了時下開發工程師和運維工程師必備的技能之一。一、主機環境預

國內環境Kubernetes v1.12.1的安裝與配置

一、些重要概念 Cluster：計算、儲存、網路資源的總和。Kubernetes的各種基於容器的應用都是執行在這些資源上的。 Master：Kubernetes的大腦，負責排程各種計算資源。Master可以是物理機或虛擬機器，多個Master可以同時執行，並實現HA。Master

kubernetes1.9.2基於kubeadm的高可用安裝HA_Kubernetes中文社群

使用kubeadm安裝安全高可用kubernetes叢集 [安裝包地址] 非高可用安裝三步即可在master上 cd shell && init.sh && master.sh 在node上 init.sh 在node上執行master輸出的join命令

CentOS7.5中安裝kubernetes - v1.12.1

文章目錄 1 機器環境 1.1 機器1 1.2 機器2 1.3 機器3 2 軟體介紹 2.1 作業系統 2.2 Docker 2.3 etcd 2.4 flanneld

DOCKER學習（二）--Kubernetes(v1.12.1)叢集安裝

本文使用Kubeadm安裝Kubernetes kubeadm是Kubernetes官方提供的用於快速安裝Kubernetes叢集的工具，伴隨Kubernetes每個版本的釋出都會同步更新，kubeadm會對叢集配置方面的一些實踐做調整，通過實驗kubeadm可以學習到

再探使用kubeadm部署高可用的k8s集群-01引言

data- mode etcd [1] working -s device master 基本再探使用kubeadm部署高可用的k8s集群-01引言 2018/1/26 提示僅供測試用途前言：高可用一直是重要的話題，需要持續研究。最近關註到 k8s 官網文檔有更新，其中

構建生產環境可用的高可用kubernetes集群

直接地址 ofo 影響 yun 客戶除了腳本網絡 kubernetes集群三步安裝構建生產環境可用的高可用kubernetes集群 | sealos項目地址特性 [x] 支持任意節點的etcd集群自動構建，且etcd集群使用安全證書，通過static pod方

使用kubeadm安裝單節點kubernetes叢集，在vmware虛擬機器centos7

關閉防火牆 systemctl stop firewalld && sudo systemctl disable firewalld 安裝docker-ce 17.06 yum remove docker \ docker-c

MySQL---高可用（修改版1.0）

MySQL高可用因為主從複製是由Master伺服器負責寫操作，因而會存在單點失敗的隱患，所以需要提高HA高可用性本文介紹了以下方法實現MySQL的高可用性一、MHA高可用（Master High Availability）原理：建立一個Manager管理者，它

centos7使用kubeadm配置高可用k8s叢集

CountingStars_ 關注 2018.08.12 09:06* 字數 464 閱讀 88評論 0喜歡 0 簡介使用kubeadm配置多master節點，實現高可用。安裝實

高可用服務設計概述[1]

1 負載均衡與反向代理當我們的應用單例項不能支撐使用者請求時，就需要擴容，從一天伺服器擴容到兩臺、幾十臺、幾百臺。然而使用者訪問時是通過如http://www.jd.com的方式訪問，在請求時，瀏覽器首先會查詢DNS伺服器獲取對應的IP，然後通過此IP訪問對

使用kuberspay部署高可用kubernetes叢集

0 環境環境：主機名 IP k8s-node01 172.16.120.151 k8s-node02 172.16.120.152 k8s-node03 172.16.120.153

kubeadm安裝高可用kubernetes v1.14.1

環境介紹：

一、準備工作

二、配置etcd

配置keepalived

配置Haproxy

初始化master

將所有node節點加入集群

其他的步驟參考：kubeadm安裝kubernetes v1.11.3 HA多主高可用並啟用ipvs

相關推薦