1. 程式人生 > >kubernetes 1.14安裝部署EFK日誌收集系統

kubernetes 1.14安裝部署EFK日誌收集系統

closed display yam tab configMap .gz class fluent border

簡單介紹

EFK 組合插件是k8s項目的一個日誌解決方案,它包括三個組件:Elasticsearch, Fluentd, Kibana。相對於ELK這樣的架構,k8s官方推行了EFK,可能Fluentd相對於Logstash更加輕量級吧。

Elasticsearch 是日誌存儲和日誌搜索引擎,Fluentd 負責把k8s集群的日誌發送給 Elasticsearch, Kibana 則是可視化界面查看和檢索存儲在 Elasticsearch 的數據。

GitHub官網:https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch

測試環境

System Hostname IP
CentOS 7.6 k8s-master 138.138.82.14
CentOS 7.6 k8s-node1 138.138.82.15
CentOS 7.6 k8s-node2 138.138.82.16

:以下是簡單的安裝部署操作,實際用於生產環境的話還要根據不同的後端存儲進而修改yaml中的相應存儲路徑。

操作步驟

1. 獲取yaml文件

從github官網下載:https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch

或者下載整個tar包:https://storage.googleapis.com/kubernetes-release/release/v1.14.1/kubernetes-server-linux-amd64.tar.gz

解壓後yanl文件位置: kubernetes/cluster/addons/fluentd-elasticsearch/

復制yaml文件到另一處進行修改操作:

[[email protected] ~]# mkdir efk
[[email protected]-master ~]# cp kubernetes/cluster/addons/fluentd-elasticsearch/*
.yaml efk/ [[email protected] ~]# cd efk/ [[email protected] efk]# ll 總用量 36 -rw-r--r-- 1 root root 382 5月 5 13:55 es-service.yaml -rw-r--r-- 1 root root 2906 5月 5 14:44 es-statefulset.yaml -rw-r--r-- 1 root root 16124 5月 5 13:55 fluentd-es-configmap.yaml -rw-r--r-- 1 root root 2702 5月 5 13:55 fluentd-es-ds.yaml -rw-r--r-- 1 root root 1052 5月 5 13:55 kibana-deployment.yaml -rw-r--r-- 1 root root 354 5月 5 13:55 kibana-service.yaml

2. 準備鏡像(在所有主機上)

:因為被墻的緣故,加上鏡像文件比較大,啟動會費時,可以提前單獨下載好所需鏡像。

①修改 es-statefulset.yaml https://www.docker.elastic.co/#

技術分享圖片

②修改 fluentd-es-ds.yaml

技術分享圖片

無需修改 kibana-deployment.yaml

技術分享圖片

說明:關於kibana鏡像以-oss結尾的版本好奇(default release that contains the commercially licensed code. The release that contains only open source will have the -oss appended to it.):

https://www.elastic.co/guide/en/kibana/6.0/docker.html

https://discuss.elastic.co/t/what-are-the-differences-between-the-kibana-oss-and-non-oss-build/152364/3

附上鏡像拉取文本:

技術分享圖片
~]# docker pull docker.elastic.co/elasticsearch/elasticsearch:6.6.2
~]# docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/fluentd-elasticsearch:v2.4.0
~]# docker pull docker.elastic.co/kibana/kibana-oss:6.6.1
View Code

3. 發布yaml文件到k8s中

[[email protected] efk]# kubectl apply -f .

4. 查看各pod狀態

[[email protected] efk]# kubectl get po -n kube-system -o wide
NAME                                    READY   STATUS    RESTARTS   AGE     IP              NODE         NOMINATED NODE   READINESS GATES
calico-node-b78m4                       1/1     Running   0          7d1h    138.138.82.16   k8s-node2    <none>           <none>
calico-node-r5mlj                       1/1     Running   0          7d1h    138.138.82.14   k8s-master   <none>           <none>
calico-node-z5tdh                       1/1     Running   0          7d1h    138.138.82.15   k8s-node1    <none>           <none>
coredns-fb8b8dccf-6mgks                 1/1     Running   0          7d1h    192.168.0.7     k8s-master   <none>           <none>
coredns-fb8b8dccf-cbtlx                 1/1     Running   0          7d1h    192.168.0.6     k8s-master   <none>           <none>
elasticsearch-logging-0                 1/1     Running   0          10m     192.168.2.11    k8s-node2    <none>           <none>
elasticsearch-logging-1                 1/1     Running   0          10m     192.168.1.17    k8s-node1    <none>           <none>
etcd-k8s-master                         1/1     Running   0          7d1h    138.138.82.14   k8s-master   <none>           <none>
fluentd-es-v2.4.0-fm9n2                 1/1     Running   0          10m     192.168.1.16    k8s-node1    <none>           <none>
fluentd-es-v2.4.0-xjx6h                 1/1     Running   0          10m     192.168.2.10    k8s-node2    <none>           <none>
kibana-logging-f4d99b69f-hc4h9          1/1     Running   0          10m     192.168.1.15    k8s-node1    <none>           <none>
kube-apiserver-k8s-master               1/1     Running   0          7d1h    138.138.82.14   k8s-master   <none>           <none>
kube-controller-manager-k8s-master      1/1     Running   1          7d1h    138.138.82.14   k8s-master   <none>           <none>
kube-proxy-c9xd2                        1/1     Running   0          7d1h    138.138.82.14   k8s-master   <none>           <none>
kube-proxy-fp2r2                        1/1     Running   0          7d1h    138.138.82.15   k8s-node1    <none>           <none>
kube-proxy-lrsw7                        1/1     Running   0          7d1h    138.138.82.16   k8s-node2    <none>           <none>
kube-scheduler-k8s-master               1/1     Running   1          7d1h    138.138.82.14   k8s-master   <none>           <none>
kubernetes-dashboard-5f7b999d65-45vlk   1/1     Running   0          6d7h    192.168.1.14    k8s-node1    <none>           <none>
metrics-server-7579f696d8-pgcc4         1/1     Running   0          6d22h   192.168.1.12    k8s-node1    <none>           <none>

順利的話,稍等一小會就成功running狀態了。

kubernetes 1.14安裝部署EFK日誌收集系統