ELK部署生產實踐部署（1）
### 日誌采集前規範解決事項：
1、開發人員不能登錄線上服務器查看詳細日誌。
2、各個系統都有日誌，日誌數據分散難以查找。
3、日誌數據量大、查詢速度慢
4、日誌數據大量延遲
5、服務器時間不同步，導致日期錯誤
### 解決問題
1. 方便快速查看各種日誌
2. 故障發生，處理故障時才去查看日誌，沒有完整的日誌告警機制
3. 節點多、日誌分散、收集日誌難度加大、沒有統一規範存取路徑
4. 運行日誌、錯誤日誌、需要固定存放位置
###
部署環境
[[email protected]_agent logfile]# cat /etc/redhat-release
CentOS Linux release 7.0.1406 (Core)
### ELKstack
- [ ] 運行流程： 收集--->>存儲 ----->>搜索+統計+展示------->>報警，數據分析
### Elastic Serch 是一個基於Lucene搜索服務器，提供了一個分布式多用戶能力的全文搜索引擎。
### ES特點
- [ ] cluster 集群
- 集群中有多個節點，其中有一個為主節點，這個主節點是可以通過選舉產生的，主節點是對於集群內部管理的。
## ELKsatack介紹，架構圖如下
- [x] 對於日誌來說，最常見的需求就是收集、存儲、查詢、展示，開源社區正好有相對應的開源項目：logstash（收集）、elasticsearch（存儲+搜索）、kibana（展示），我們將這三個組合起來的技術稱之為ELKStack，所以說ELKStack指的是Elasticsearch、Logstash、Kibana技術棧的結合，一個通用的架構如下圖所示：
## ElkStack部署
#### Elasticsearch、需要Java環境，所以直接使用yum安裝。
1. 安裝java、並檢查是否安裝成功
```
[[email protected]_agent ~]# yum install java -y
[[email protected]_agent ~]# java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)
[[email protected]_agent ~]#
```
2. 下載並安裝GPG key
[[email protected]_agent ~]# rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
3. 編輯yum倉庫
[[email protected]_agent ~]# vim /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
4. yum安裝需要配置limits
[[email protected]_agent ~]# vim /etc/security/limits.conf
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
5. 安裝ElasticSearch
[[email protected]_agent ~]# yum install -y elasticsearch logstash kibana
## 配置Elasticsearch
[[email protected]_agent ~]# vim /etc/elasticsearch/elasticsearch.yml
[[email protected]_agent ~]# mkdir -p /data/es-data ##創建存放目錄
cluster.name: elkcluster ##集群名稱
node.name: elk-server01 ##節點名稱
path.data: /data/es-data ##修改目錄存放路徑
path.logs: /var/log/elasticsearch/ ##修改日誌存放路徑
bootstrap.memory_lock: true 不使用交換分區，鎖住內存
network.host: 172.16.1.200 監聽主機
http.port: 9200 打開監聽端口
### 查看更改配置文件的內容
[[email protected]_agent ~]# grep ‘^[a-z]‘ /etc/elasticsearch/elasticsearch.yml
cluster.name: elkcluster
node.name: elk-server01
path.data: /data/es-data
path.logs: /var/log/elasticsearch/
bootstrap.memory_lock: true
network.host: 172.16.1.200
http.port: 9200
action.destructive_requires_name: false
discovery.zen.ping.unicast.hosts: ["172.16.1.200", "172.16.1.201"]
[[email protected]_agent ~]#
### 啟動elasticsearch
```
[[email protected]_agent ~]# systemctl start elasticsearch
### 查看啟動狀態、啟動失敗
[[email protected]_agent ~]# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2017-06-26 14:52:39 EDT; 2s ago
Docs: http://www.elastic.co
Process: 20760 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_HOME} -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR} -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
Process: 20759 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
Main PID: 20760 (code=exited, status=1/FAILURE)
Jun 26 14:52:39 master_agent elasticsearch[20760]: at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:212)
Jun 26 14:52:39 master_agent elasticsearch[20760]: at org.elasticsearch.bootstrap.Security.configure(Security.java:118)
Jun 26 14:52:39 master_agent elasticsearch[20760]: at org.elasticsearch.bootstrap.Bootstrap.setupSecurity(Bootstrap.java:212)
Jun 26 14:52:39 master_agent elasticsearch[20760]: at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:183)
Jun 26 14:52:39 master_agent elasticsearch[20760]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
Jun 26 14:52:39 master_agent elasticsearch[20760]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:45)
Jun 26 14:52:39 master_agent elasticsearch[20760]: Refer to the log for complete error details.
Jun 26 14:52:39 master_agent systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jun 26 14:52:39 master_agent systemd[1]: Unit elasticsearch.service entered failed state.
Jun 26 14:52:39 master_agent systemd[1]: elasticsearch.service failed.
[[email protected]_agent ~]#
### 解決辦法
- [x] 查看日誌
- [[email protected]_agent ~]# tail -100 /var/log/elasticsearch/elkcluster.log
- [x] 修改目錄所屬權限
- [[email protected]_agent ~]# chown -R elasticsearch:elasticsearch /data/es-data/
### 查看啟動
[[email protected]_agent ~]# systemctl start elasticsearch
[[email protected]_agent ~]# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: active (running) since Mon 2017-06-26 14:56:06 EDT; 2s ago
Docs: http://www.elastic.co
Process: 20802 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
Main PID: 20803 (java)
CGroup: /system.slice/elasticsearch.service
└─20803 /bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFrac...
### 檢查端口是否開啟
[[email protected]_agent ~]# systemctl restart elasticsearch
[[email protected]_agent ~]# ss -ntlp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 50 *:3306 *:* users:(("mysqld",19594,14))
LISTEN 0 128 *:22 *:* users:(("sshd",1372,3))
LISTEN 0 50 ::ffff:172.16.1.201:9200 :::* users:(("java",21205,91))
LISTEN 0 50 ::ffff:172.16.1.201:9300 :::* users:(("java",21205,79))
LISTEN 0 128 :::22 :::* users:(("sshd",1372,4))
[[email protected]_agent ~]#
### 瀏覽器訪問查看下結果
http://172.16.1.201:9200/
內容如下：
{
"name" : "elk-server01",
"cluster_name" : "elkcluster",
"cluster_uuid" : "bkJEwJGARXq2Ki2xWa0oTQ",
"version" : {
"number" : "2.4.6",
"build_hash" : "5376dca9f70f3abef96a77f4bb22720ace8240fd",
"build_timestamp" : "2017-07-18T12:17:44Z",
"build_snapshot" : false,
"lucene_version" : "5.5.4"
},
"tagline" : "You Know, for Search"
}
本文出自 “一杯水” 博客，請務必保留此出處http://6528161.blog.51cto.com/6518161/1952126
ELK部署生產實踐部署（1）