1.MySQL慢查詢日誌格式：

# Time: 181109 15:04:08
# User@Host: tvpayrcdev[tvpayrcdev] @  [172.16.14.51]  Id: 8960747
# Query_time: 35.918265  Lock_time: 0.000141 Rows_sent: 1  Rows_examined: 11699162
SET timestamp=1541747048;
select count(*) from trade_risk_control_record

2.MySQL慢查詢日誌已通過rsyslog實時傳輸到logstash作為Indexer的節點。

二、logstash配置文件

input部分

input {
  file {
    type => "logstash-rc-mysql-slow"
        path => "/opt/data/logs/localhost-172.16.14.35/db1-slow.log"
        codec => multiline {
          pattern => "^# Time:"
          negate => true
          what => "previous"
        }
        stat_interval => 1
                discover_interval => 1
                start_position=>"end"
                sincedb_path => "/dev/null"
  }
}
 

filter部分

if [type] == "logstash-rc-mysql-slow" {
  grok {
    patterns_dir => ["/usr/local/logstash/etc/conf.d/patterns/mysql"]
    match => { "message" => "%{LONGQUERYLOG}" }
  }
  date {
    match => ["timestamp","UNIX"]
  }
  mutate {
    convert => [ "query_time", "float" ]
    convert => [ " lock_time", "float" ]
    remove_field => "message"
    remove_field => "timestamp"
       }
  } 
 

output部分

if [type] == "logstash-rc-mysql-slow" {
             elasticsearch {
                            hosts => ["172.16.1.25","172.16.1.26","172.16.1.27"]
                            index => ‘logstash-mysql_slow_log-%{+YYYY-MM-dd}‘
                            codec=>plain{charset=>"UTF-8"}
                          }
        }

三、kibana展示

1.創建索引

2.發現數據

包括字段：

3.繪制visualize
例1：統計數量排名前10的sql語句及對應的查詢時間

ELK分析MySQL慢查詢日誌並生成圖像