ELK日誌分析系統搭建
注：/usr/local/src為原始碼安裝包存放目錄。 /data/為資料儲存、解壓目錄。
準備工作：下載elasticsearch、filebeat、Kibana_Hanization、elasticsearch-head、jdk 、kibana、logstash、node
wget https://download.oracle.com/otn-pub/java/jdk/8u191-b12/2787e4a523244c269598db4e85c51e0c/jdk-8u191-linux-x64.tar.gz
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.1.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.5.1-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.1.tar.gz
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.4.1-linux-x86_64.tar.gz
git clone https://github.com/mobz/elasticsearch-head.git
wget https://npm.taobao.org/mirrors/node/latest-v4.x/node-v4.4.7-linux-x64.tar.gz
git clone https://github.com/anbai-inc/Kibana_Hanization.git
一、 安裝elasticsearch
1.1 安裝Java
[[email protected] data]# rpm -qa |grep java
tzdata-java-2015g-1.el7.noarch
javapackages-tools-3.4.1-11.el7.noarch
java-1.8.0-openjdk-headless-1.8.0.65-3.b17.el7.x86_64
java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64
java-1.8.0-openjdk-1.8.0.65-3.b17.el7.x86_64
java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64
python-javapackages-3.4.1-11.el7.noarch
[
[[email protected] data]# rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64
[[email protected] data]# rpm -e --nodeps java-1.8.0-openjdk-1.8.0.65-3.b17.el7.x86_64
[[email protected] data]# rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.65-3.b17.el7.x86_64
[[email protected] data]# rpm -qa |grep java
tzdata-java-2015g-1.el7.noarch
javapackages-tools-3.4.1-11.el7.noarch
python-javapackages-3.4.1-11.el7.noarch
[[email protected] ~]# mkdir /data
[[email protected] src]# tar xf jdk-8u191-linux-x64.tar.gz -C /data/
[[email protected] jdk]# vim /etc/profile
在末尾新增以下內容
export JAVA_HOME=/data/jdk
export JAVA_BIN=/data/jdk/bin
export JAVA_LIB=/data/jdk/lib
export JAVA_JRE=/data/jdk/jre
[[email protected] jdk]# ln -s /data/jdk/bin/java /usr/bin/
[[email protected] jdk]# java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
[[email protected] config]# mkdir /data/{es-data,es-logs}
修改配置檔案，注意“：”後面有空格，
[[email protected] config]# vim elasticsearch.yml
path.data: /data/es-data
path.logs: /data/es-logs
network.host: 0.0.0.0
http.port: 9200
*在network項最後新增，後面要呼叫到以下資訊，所以一起新增。但是在啟動elasticsearch時會報錯，所以新增時記得登出掉，在前面新增“#”，這樣啟動elasticsearch就不會報錯了。在安裝elasticsearch-head後再取消“#”就可以了。
http.cors.enabled:true
http.cors.allow-origin:""
後面安裝elasticsearch-head-master需要呼叫到
[[email protected] data]# vim /etc/security/limits.conf
在最後新增以下資訊， **
- soft nofile 65536
- hard nofile 65536
- soft nproc 65536
- hard nproc 65536
[[email protected] data]# vim /etc/sysctl.conf
新增以下內容
vm.max_map_count=655360
儲存後執行下
[[email protected] data]# sysctl -p
修改完後需要重啟，因為修改limits.conf內容需要重啟才能生效。
[[email protected] data]# reboot
修改許可權，由於不能用root使用者執行，需要建立新使用者，並授權。
[[email protected] data]# chown -R elk:elk elasticsearch
[[email protected] data]# chown -R elk:elk es-data
[[email protected] data]# chown -R elk:elk es-logs
重新登入，切換普通使用者
[[email protected] config]# su elk
[[email protected] config]$ cd ..
執行elasticsearch
[[email protected] elasticsearch]$ bin/elasticsearch
檢查是否啟動成功
檢視監聽埠
[[email protected] ~]# netstat -ntlp |grep java
tcp6 0 0 :::9200 ::: LISTEN 3469/java
tcp6 0 0 :::9300 ::: LISTEN 3469/java
說明正常啟動
在沒有關閉防火牆的情況下，記得開放9200、9300和9100（elasticsearch-head-master用的）所以一起開放了
[[email protected] ~]# firewall-cmd --zone=public --add-port=9200/tcp --permanent
success
[[email protected] ~]# firewall-cmd --zone=public --add-port=9300/tcp --permanent
success
[[email protected] ~]# firewall-cmd --zone=public --add-port=9100/tcp --permanent
success
[[email protected] ~]# firewall-cmd --reload
success
[[email protected] ~]#
URL測試
[[email protected] ~]# curl 172.16.41.32:9200
{
"name" : "g79q1--",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "On9wUDkERpmJDcbql6jEkw",
"version" : {
"number" : "6.5.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "8c58350",
"build_date" : "2018-11-16T02:22:42.182257Z",
"build_snapshot" : false,
"lucene_version" : "7.5.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
[[email protected] ~]#
正常返回資料說明服務正常使用。
1.2 下面開始安裝elasticsearch-head-master，由於elasticsearch沒有網頁介面所以需要安裝這個外掛
下載elasticsearch-head
[[email protected] data]# git clone https://github.com/mobz/elasticsearch-head.git
或
[[email protected] data]# wget https://github.com/mobz/elasticsearch-head/archive/master.zip
下載elasticsearch-head依賴外掛node
[[email protected] src]# wget https://npm.taobao.org/mirrors/node/latest-v4.x/node-v4.4.7-linux-x64.tar.gz
[[email protected] src]# tar xf node-v4.4.7-linux-x64.tar.gz -C /data/
[[email protected] data]# cd elasticsearch-head/
[[email protected] elasticsearch-head]# vim Gruntfile.js
在connect項新增“hostname: '主機IP'”，注意在keeplive: true後面記得加上“，”及“:”後面加空格。否則啟動服務時會報語法錯誤。
connect: {
server: {
options: {
port: 9100,
base: '.',
keepalive: true,
hostname: '172.16.41.32',
}
}
}
*新增node環境變數
[[email protected] node]# vim /etc/profile
新增以下內容
export NODE_HOME=/data/node
export NODE_PATH=/data/node/lib/node_modules
export PATH=$NODE_HOME/bin:$NODE_PATH:$PATH
修改完profile檔案記得讓配置生效
[[email protected] node]# source /etc/profile
把之前新增的兩條命令前的“#”去掉，注意：“:”後面有空格
[[email protected] config]# vim elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: ""
進入到elasticsearch-head的安裝目錄，因為elasticsearch-head是通過grunt啟動的，所以需要安裝grunt外掛
[[email protected] elasticsearch-head]# npm install -g grunt-cli
npm WARN engine [email protected]: wanted: {"node":">= 4.5.0"} (current: {"node":"4.4.7","npm":"2.15.8"})
/data/node/bin/grunt -> /data/node/lib/node_modules/grunt-cli/bin/grunt
[email protected] /data/node/lib/node_modules/grunt-cli
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected] ([email protected], [email protected])
└── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected])
[[email protected] elasticsearch-head]#
在elasticsearch-head目錄下面執行命令下載所有的依賴
[[email protected] elasticsearch-head]# npm install**
[email protected] node_modules/grunt-contrib-jasmine
├── [email protected]
├── [email protected]
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
└── [email protected] ([email protected], [email protected], [email protected], [email protected])
[[email protected] elasticsearch-head]#
通過後臺程序進行啟動
[[email protected] elasticsearch-head]# nohup grunt server &
[1] 5254
[[email protected] elasticsearch-head]# nohup: 忽略輸入並把輸出追加到"nohup.out"
現在可以通過網頁進行瀏覽
預設這裡會顯示“叢集健康值：未連線”，不要緊張，我們只需要將“localhost”修改成主機IP地址，點選“連線”就可以顯示正常了。
到此elasticsearch配置完成。
二、 logstasho安裝
2.1 手動安裝logstash
在logstash/config目錄建立一個.conf配置檔案，名字自己定義。我這使用的是default.conf，新增以下內容：
#監聽5044埠作為輸入
input {
beats {
port => "5044"
}
}
#資料過濾
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
geoip {
source => "clientip"
}
#輸出配置為本機的9200埠，這是ElasticSerach服務的監聽埠
output {
elasticsearch {
hosts => ["本機IP地址:9200"]
}
}
~
進入filebeat外掛目錄
[[email protected] data]# cd filebeat/
後臺啟動
[[email protected] filebeat]# ./filebeat &
檢視啟動情況
[[email protected] filebeat]# ps -ef |grep filebeat
root 70376 48524 0 16:19 pts/3 00:00:00 ./filebeat
root 70391 48524 0 16:20 pts/3 00:00:00 grep --color=auto filebeat
[[email protected] filebeat]# tail logs/filebeat
2018-11-27T16:19:36+08:00 INFO Loading Prospectors: 1
2018-11-27T16:19:36+08:00 INFO Prospector with previous states loaded: 5
2018-11-27T16:19:36+08:00 INFO Starting Registrar
2018-11-27T16:19:36+08:00 INFO Starting prospector of type: log; id: 17005676086519951868
2018-11-27T16:19:36+08:00 INFO Start sending events to output
2018-11-27T16:19:36+08:00 INFO Loading and starting Prospectors completed. Enabled prospectors: 1
2018-11-27T16:19:36+08:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2018-11-27T16:20:06+08:00 INFO Non-zero metrics in the last 30s: publish.events=5 registrar.states.current=5 registrar.states.update=5 registrar.writes=1
2018-11-27T16:20:36+08:00 INFO No non-zero metrics in the last 30s
2018-11-27T16:21:06+08:00 INFO No non-zero metrics in the last 30s
啟動正常
開放埠
[[email protected] logstash]# firewall-cmd --add-port=5044/tcp --permanent
success
[[email protected] logstash]# firewall-cmd --add-port=9600/tcp --permanent
success
[[email protected] logstash]# firewall-cmd --reload
配置path.data路徑，要不然啟動logstash會報錯
[[email protected] logstash]# bin/logstash -f config-mysql-path.data=/data/es-logstash-log
後臺啟動logstash
[[email protected] logstash]# bin/logstash -f config/default.conf &
檢視監聽資訊
[[email protected] config]# netstat -ntlup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 172.16.41.32:9100 0.0.0.0: LISTEN 47943/grunt
tcp 0 0 192.168.122.1:53 0.0.0.0: LISTEN 2854/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0: LISTEN 1566/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0: LISTEN 1571/cupsd
tcp 0 0 127.0.0.1:25 0.0.0.0: LISTEN 2070/master
tcp 0 0 172.16.41.32:5601 0.0.0.0: LISTEN 73103/bin/../node/b
tcp6 0 0 :::9200 ::: LISTEN 48329/java
tcp6 0 0 :::5044 ::: LISTEN 71717/java
tcp6 0 0 :::9300 ::: LISTEN 48329/java
tcp6 0 0 :::22 ::: LISTEN 1566/sshd
tcp6 0 0 ::1:631 ::: LISTEN 1571/cupsd
tcp6 0 0 ::1:25 ::: LISTEN 2070/master
tcp6 0 0 127.0.0.1:9600 ::: LISTEN 71717/java
udp 0 0 192.168.122.1:53 0.0.0.0: 2854/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0: 2854/dnsmasq
udp 0 0 0.0.0.0:53381 0.0.0.0: 939/avahi-daemon: r
udp 0 0 0.0.0.0:5353 0.0.0.0: 939/avahi-daemon: r
udp 0 0 127.0.0.1:323 0.0.0.0: 960/chronyd
udp6 0 0 ::1:323 :::* 960/chronyd
[[email protected] config]#
測試logstash訪問情況，返回以下內容說明logstash正常啟動。
[[email protected] elasticsearch]# curl http://172.16.41.32:9200/_search?pretty
{
"took" : 2771,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1116,
"max_score" : 1.0,
"hits" : [
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "IVZAVGcB6PZDlVvdZ8IE",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.518Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "Successfully initialized wpa_supplicant",
"offset" : 40,
"source" : "/var/log/wpa_supplicant.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "KFZAVGcB6PZDlVvdZ8IE",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.518Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "Successfully initialized wpa_supplicant",
"offset" : 80,
"source" : "/var/log/wpa_supplicant.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "K1ZAVGcB6PZDlVvdZ8IE",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.519Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "Nov 15 11:14:47 Updated: systemd-sysv-219-57.el7_5.3.x86_64",
"offset" : 412,
"source" : "/var/log/yum.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "L1ZAVGcB6PZDlVvdZ8IE",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.535Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "[\u001B[32m OK \u001B[0m] Reached target Paths.",
"offset" : 95,
"source" : "/var/log/boot.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "MVZAVGcB6PZDlVvdZ8IE",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.535Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "X.Org X Server 1.17.2",
"offset" : 36,
"source" : "/var/log/Xorg.1.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "OVZAVGcB6PZDlVvdZ8IE",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.567Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "Release Date: 2015-06-16",
"offset" : 61,
"source" : "/var/log/Xorg.1.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "QVZAVGcB6PZDlVvdZ8IE",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.568Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "[ 23.508] Current version of pixman: 0.32.6",
"offset" : 658,
"source" : "/var/log/Xorg.0.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "TVZAVGcB6PZDlVvdZ8IE",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.568Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "[ 23.597] (==) No Layout section. Using the first Screen section.",
"offset" : 1266,
"source" : "/var/log/Xorg.0.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "V1ZAVGcB6PZDlVvdbsKB",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.568Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "[\u001B[32m OK \u001B[0m] Reached target Initrd Default Target.",
"offset" : 881,
"source" : "/var/log/boot.log",
"type" : "log"
}
},
{
"_index" : "filebeat-2018.11.27",
"_type" : "log",
"_id" : "XVZAVGcB6PZDlVvdbsKB",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-11-27T08:19:13.568Z",
"beat" : {
"hostname" : "localhost.localdomain",
"name" : "localhost.localdomain",
"version" : "5.4.1"
},
"input_type" : "log",
"message" : "[\u001B[32m OK \u001B[0m] Stopped Cleaning Up and Shutting Down Daemons.",
"offset" : 1218,
"source" : "/var/log/boot.log",
"type" : "log"
}
}
]
}
}
正常啟動
三、 配置kibana
[[email protected] data]# cd kibana/config
修改配置檔案，去除“#”，並修改相應內容。
[[email protected] config]# vim kibana.yml
[[email protected] config]# egrep -v "^#|^$" kibana.yml
server.port: 5601
server.host: "172.16.41.32"
elasticsearch.url: "http://172.16.41.32:9200"
開放埠
[[email protected] config]# firewall-cmd --add-port=5601/tcp --permanent
success
[[email protected] config]# firewall-cmd --reload
success
後臺啟動kibana
[[email protected] kibana]# bin/kibana &
檢視埠監聽情況
[[email protected] kibana]# netstat -ntulp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 172.16.41.32:9100 0.0.0.0: LISTEN 47943/grunt
tcp 0 0 192.168.122.1:53 0.0.0.0: LISTEN 2854/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0: LISTEN 1566/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0: LISTEN 1571/cupsd
tcp 0 0 127.0.0.1:25 0.0.0.0: LISTEN 2070/master
tcp 0 0 172.16.41.32:5601 0.0.0.0: LISTEN 73103/bin/../node/b
tcp6 0 0 :::9200 ::: LISTEN 48329/java
tcp6 0 0 :::5044 ::: LISTEN 71717/java
tcp6 0 0 :::9300 ::: LISTEN 48329/java
tcp6 0 0 :::22 ::: LISTEN 1566/sshd
tcp6 0 0 ::1:631 ::: LISTEN 1571/cupsd
tcp6 0 0 ::1:25 ::: LISTEN 2070/master
tcp6 0 0 127.0.0.1:9600 ::: LISTEN 71717/java
udp 0 0 192.168.122.1:53 0.0.0.0: 2854/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0: 2854/dnsmasq
udp 0 0 0.0.0.0:53381 0.0.0.0: 939/avahi-daemon: r
udp 0 0 0.0.0.0:5353 0.0.0.0: 939/avahi-daemon: r
udp 0 0 127.0.0.1:323 0.0.0.0: 960/chronyd
udp6 0 0 ::1:323 :::* 960/chronyd
結尾返回以下內容，說明正常啟動。
log [09:43:22.040] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [09:43:29.129] [warning][reporting] Enabling the Chromium sandbox provides an additional layer of protection.
log [09:43:29.287] [info][migrations] Creating index .kibana_1.
log [09:43:31.804] [info][migrations] Pointing alias .kibana to .kibana_1.
log [09:43:33.246] [info][migrations] Finished in 3959ms.
log [09:43:33.251] [info][listening] Server running at http://172.16.41.32:5601
log [09:43:36.724] [info][status][plugin:[email protected]] Status changed from red to green - Ready
2.2 漢化
預設情況下kibana是英文版，所以我們要進行漢化。
下載漢化包
[[email protected] data]# git clone https://github.com/anbai-inc/Kibana_Hanization.git
[[email protected] data]# cd Kibana_Hanization/
進行翻譯
[[email protected] Kibana_Hanization]# python main.py ../kibana/
等待翻譯完成。
2.3 開啟網頁進行瀏覽測試