kubernetes 1.14安裝部署EFK日誌收集系統
簡單介紹:
EFK 組合插件是k8s項目的一個日誌解決方案,它包括三個組件:Elasticsearch, Fluentd, Kibana。相對於ELK這樣的架構,k8s官方推行了EFK,可能Fluentd相對於Logstash更加輕量級吧。
Elasticsearch 是日誌存儲和日誌搜索引擎,Fluentd 負責把k8s集群的日誌發送給 Elasticsearch, Kibana 則是可視化界面查看和檢索存儲在 Elasticsearch 的數據。
GitHub官網:https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch
測試環境:
System | Hostname | IP |
CentOS 7.6 | k8s-master | 138.138.82.14 |
CentOS 7.6 | k8s-node1 | 138.138.82.15 |
CentOS 7.6 | k8s-node2 | 138.138.82.16 |
註:以下是簡單的安裝部署操作,實際用於生產環境的話還要根據不同的後端存儲進而修改yaml中的相應存儲路徑。
操作步驟:
1. 獲取yaml文件
從github官網下載:https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch
或者下載整個tar包:https://storage.googleapis.com/kubernetes-release/release/v1.14.1/kubernetes-server-linux-amd64.tar.gz
解壓後yanl文件位置: kubernetes/cluster/addons/fluentd-elasticsearch/
復制yaml文件到另一處進行修改操作:
[[email protected] ~]# mkdir efk [[email protected]-master ~]# cp kubernetes/cluster/addons/fluentd-elasticsearch/*.yaml efk/ [[email protected] ~]# cd efk/ [[email protected] efk]# ll 總用量 36 -rw-r--r-- 1 root root 382 5月 5 13:55 es-service.yaml -rw-r--r-- 1 root root 2906 5月 5 14:44 es-statefulset.yaml -rw-r--r-- 1 root root 16124 5月 5 13:55 fluentd-es-configmap.yaml -rw-r--r-- 1 root root 2702 5月 5 13:55 fluentd-es-ds.yaml -rw-r--r-- 1 root root 1052 5月 5 13:55 kibana-deployment.yaml -rw-r--r-- 1 root root 354 5月 5 13:55 kibana-service.yaml
2. 準備鏡像(在所有主機上)
註:因為被墻的緣故,加上鏡像文件比較大,啟動會費時,可以提前單獨下載好所需鏡像。
①修改 es-statefulset.yaml (https://www.docker.elastic.co/#)
②修改 fluentd-es-ds.yaml
③無需修改 kibana-deployment.yaml
說明:關於kibana鏡像以-oss結尾的版本好奇(default release that contains the commercially licensed code. The release that contains only open source will have the -oss
appended to it.):
https://www.elastic.co/guide/en/kibana/6.0/docker.html
https://discuss.elastic.co/t/what-are-the-differences-between-the-kibana-oss-and-non-oss-build/152364/3
附上鏡像拉取文本:
~]# docker pull docker.elastic.co/elasticsearch/elasticsearch:6.6.2 ~]# docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/fluentd-elasticsearch:v2.4.0 ~]# docker pull docker.elastic.co/kibana/kibana-oss:6.6.1View Code
3. 發布yaml文件到k8s中
[[email protected] efk]# kubectl apply -f .
4. 查看各pod狀態
[[email protected] efk]# kubectl get po -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES calico-node-b78m4 1/1 Running 0 7d1h 138.138.82.16 k8s-node2 <none> <none> calico-node-r5mlj 1/1 Running 0 7d1h 138.138.82.14 k8s-master <none> <none> calico-node-z5tdh 1/1 Running 0 7d1h 138.138.82.15 k8s-node1 <none> <none> coredns-fb8b8dccf-6mgks 1/1 Running 0 7d1h 192.168.0.7 k8s-master <none> <none> coredns-fb8b8dccf-cbtlx 1/1 Running 0 7d1h 192.168.0.6 k8s-master <none> <none> elasticsearch-logging-0 1/1 Running 0 10m 192.168.2.11 k8s-node2 <none> <none> elasticsearch-logging-1 1/1 Running 0 10m 192.168.1.17 k8s-node1 <none> <none> etcd-k8s-master 1/1 Running 0 7d1h 138.138.82.14 k8s-master <none> <none> fluentd-es-v2.4.0-fm9n2 1/1 Running 0 10m 192.168.1.16 k8s-node1 <none> <none> fluentd-es-v2.4.0-xjx6h 1/1 Running 0 10m 192.168.2.10 k8s-node2 <none> <none> kibana-logging-f4d99b69f-hc4h9 1/1 Running 0 10m 192.168.1.15 k8s-node1 <none> <none> kube-apiserver-k8s-master 1/1 Running 0 7d1h 138.138.82.14 k8s-master <none> <none> kube-controller-manager-k8s-master 1/1 Running 1 7d1h 138.138.82.14 k8s-master <none> <none> kube-proxy-c9xd2 1/1 Running 0 7d1h 138.138.82.14 k8s-master <none> <none> kube-proxy-fp2r2 1/1 Running 0 7d1h 138.138.82.15 k8s-node1 <none> <none> kube-proxy-lrsw7 1/1 Running 0 7d1h 138.138.82.16 k8s-node2 <none> <none> kube-scheduler-k8s-master 1/1 Running 1 7d1h 138.138.82.14 k8s-master <none> <none> kubernetes-dashboard-5f7b999d65-45vlk 1/1 Running 0 6d7h 192.168.1.14 k8s-node1 <none> <none> metrics-server-7579f696d8-pgcc4 1/1 Running 0 6d22h 192.168.1.12 k8s-node1 <none> <none>
順利的話,稍等一小會就成功running狀態了。
kubernetes 1.14安裝部署EFK日誌收集系統