前言

搭建ELK系統有兩種方式
1、元件獨立安裝（更深入瞭解ELK系統的工作流程）
2、使用docker容器安裝（這種方式配置更簡單，快捷方便）

本系列文章使用元件獨立安裝的方式，如果你想使用docker容器安裝，請跳過本教程

環境需求

本教程使用CentOS 6.9
Elasticsearch5.x版本不能以root使用者執行，所以需增加普通使用者
這裡新增使用者：elk

useradd elk

JDK環境安裝

Elasticsearch是基於Java的，需要安裝最新的JDK 1.8
如果自帶低版本的JDK，執行以下命令檢視

java -version

請刪除再進行安裝新版本

yum remove java

下載JDK1.8

wget http://download.oracle.com/otn-pub/java/jdk/8u144-b01/090f390dda5b47b9b721c7dfaa008135/jdk-8u144-linux-x64.tar.gz
#wget命令可能下載不了，自行想辦法下載

解壓JDK1.8

#解壓到指定目錄
tar -zxvf ./jdk-8u144-linux-x64.tar.gz -C /usr/local/
#最終結果/usr/local/jdk1.8.0_144

配置環境變數

vi /etc/profile

新增

JAVA_HOME=/usr/local/jdk1.8.0_144
JRE_HOME=/usr/local/jdk1.8.0_144/jre
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
CLASSPATH=:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib

export PATH=$PATH:$JAVA_HOME:$JRE_HOME:$CLASSPATH
 

安裝Elasticsearch5.5.2

下載安裝包

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.2.tar.gz

解壓安裝包

#解壓到指定目錄
tar -zxvf ./elasticsearch-5.5.2.tar.gz -C /usr/local/
#最終結果/usr/local/elasticsearch-5.5.2
#改變許可權
chown -R elk:elk /usr/local/elasticsearch-5.5.2/

直接執行elasticsearch

#切換使用者
su elk
#進入目錄
cd /usr/local/elasticsearch-5.5.2
#執行
./bin/elasticsearch
 

此時可能會出現一大堆異常

java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed
        at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:363) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:638) ~[elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:245) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:111) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:194) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.5.2.jar:5.5.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.5.2.jar:5.5.2]
[2017-08-30T10:29:44,497][INFO ][o.e.n.Node               ] [] initializing ...
[2017-08-30T10:29:45,180][INFO ][o.e.e.NodeEnvironment    ] [2NJyxQe] using [1] data paths, mounts [[/ (/dev/mapper/VolGroup-lv_root)]], net usable_space [14.9gb], net total_space [17.1gb], spins? [possibly], types [ext4]
[2017-08-30T10:29:45,183][INFO ][o.e.e.NodeEnvironment    ] [2NJyxQe] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-08-30T10:29:45,197][INFO ][o.e.n.Node               ] node name [2NJyxQe] derived from node ID [2NJyxQewTSmbEGY0YlXTSg]; set [node.name] to override
[2017-08-30T10:29:45,201][INFO ][o.e.n.Node               ] version[5.5.2], pid[1828], build[b2f0c09/2017-08-14T12:33:14.154Z], OS[Linux/2.6.32-696.el6.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_144/25.144-b01]
[2017-08-30T10:29:45,205][INFO ][o.e.n.Node               ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.5.2]
[2017-08-30T10:29:48,468][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [aggs-matrix-stats]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [ingest-common]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [lang-expression]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [lang-groovy]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [lang-mustache]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [lang-painless]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [parent-join]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [percolator]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [reindex]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [transport-netty3]
[2017-08-30T10:29:48,469][INFO ][o.e.p.PluginsService     ] [2NJyxQe] loaded module [transport-netty4]
[2017-08-30T10:29:48,470][INFO ][o.e.p.PluginsService     ] [2NJyxQe] no plugins loaded
[2017-08-30T10:29:55,362][INFO ][o.e.d.DiscoveryModule    ] [2NJyxQe] using discovery type [zen]
[2017-08-30T10:29:56,597][INFO ][o.e.n.Node               ] initialized
[2017-08-30T10:29:56,597][INFO ][o.e.n.Node               ] [2NJyxQe] starting ...
[2017-08-30T10:29:57,256][INFO ][o.e.t.TransportService   ] [2NJyxQe] publish_address {192.168.1.138:9300}, bound_addresses {192.168.1.138:9300}
[2017-08-30T10:29:57,296][INFO ][o.e.b.BootstrapChecks    ] [2NJyxQe] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-08-30T10:29:57,412][ERROR][o.e.b.Bootstrap          ] [2NJyxQe] node validation exception
[4] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: max number of threads [1024] for user [elk] is too low, increase to at least [2048]
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[4]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
[2017-08-30T10:29:57,435][INFO ][o.e.n.Node               ] [2NJyxQe] stopping ...
[2017-08-30T10:29:57,534][INFO ][o.e.n.Node               ] [2NJyxQe] stopped
[2017-08-30T10:29:57,534][INFO ][o.e.n.Node               ] [2NJyxQe] closing ...
[2017-08-30T10:29:57,552][INFO ][o.e.n.Node               ] [2NJyxQe] closed

異常問題一

[4] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: max number of threads [1024] for user [elk] is too low, increase to at least [2048]
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[4]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

這裡有4個問題，需要逐一解決

問題[1]

原因：無法建立本地檔案問題,使用者最大可建立檔案數太小

解決方案：
切換到root使用者，編輯limits.conf配置檔案， 新增類似如下內容：

vi /etc/security/limits.conf

#新增如下內容:
*  soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

問題[2]

解決：切換到root使用者，修改90-nproc.conf配置檔案。

vi /etc/security/limits.d/90-nproc.conf 

#修改如下內容：
* soft nproc 1024
#修改為
* soft nproc 2048

問題[3]
解決：切換到root使用者修改配置sysctl.conf

vi /etc/sysctl.conf
#新增下面配置：
vm.max_map_count=655360
#並執行命令：
sysctl -p

問題[4]

[4]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

#和以下異常是統一原因
java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed
....

問題原因：因為Centos6不支援SecComp，而ES5.5.2預設bootstrap.system_call_filter為true進行檢測
所以導致檢測失敗，失敗後直接導致ES不能啟動。
詳見 ：https://github.com/elastic/elasticsearch/issues/22899

解決方法：
修改配置

vi ./config/elasticsearch.yml

bootstrap.system_call_filter為false，注意要在Memory下面:

bootstrap.memory_lock: false 
bootstrap.system_call_filter: false

切換root使用者，重新啟動elasticsearch

su root
./bin/elasticsearch

如無意外，啟動正常

本機命令訪問

    curl http://127.0.0.1:9200

如出現以下內容，則啟動正常

{
  "name" : "2NJyxQe",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "HR3s74ntRXKZi0qal3BE0A",
  "version" : {
    "number" : "5.5.2",
    "build_hash" : "b2f0c09",
    "build_date" : "2017-08-14T12:33:14.154Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

如果在虛擬機器裡安裝elasticsearch，想使用外部瀏覽器訪問
需修改配置

vi ./config/elasticsearch.yml

network.host: 192.168.1.138 
http.port: 9200

# 增加新的引數，這樣head外掛可以訪問es
http.cors.enabled: true
http.cors.allow-origin: "*"

重新啟動elasticsearch，然後在window使用瀏覽器訪問

http://192.168.1.138:9200/

如果訪問出現問題，注意防火牆問題
關閉防火牆

service iptables stop

或防火牆配置中新增埠

su root
vi /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 9200 -j ACCEPT
service iptables restart