1. 程式人生 > >prometheus 監控 之 nginx 應用

prometheus 監控 之 nginx 應用

用Prometheus進行nginx的監控可以自動的對相關server_name和upstream進行監控,你也可以自定義Prometheus的資料標籤,實現對不同機房和不同專案的nginx進行監控。

監控Nginx主要用到以下三個模組:
  • nginx-module-vts:Nginx virtual host traffic status module,Nginx的監控模組,能夠提供JSON格式的資料產出。
  • nginx-vts-exporter:Simple server that scrapes Nginx vts stats and exports them via HTTP for Prometheus consumption。主要用於收集Nginx的監控資料,並給Prometheus提供監控介面,預設埠號9913。
  • Prometheus:監控Nginx-vts-exporter提供的Nginx資料,並存儲在時序資料庫中,可以使用PromQL對時序資料進行查詢和聚合。
1.nginx-module-vts模組的編譯

首先檢視nginx安裝了那些模組:

[[email protected] conf]#./sbin/nginx  -V

下載nginx-module-vts模組,並新增模組到nginx:

[[email protected] conf]# cd /usr/local/src
[[email protected] conf]# git clone git://github.com/vozlt/nginx-module-vts.git
[
[email protected]
conf]# ./configure --prefix=/usr/local/nginx --with-http_ssl_module --add-module=//usr/local/src/nginx-module-vts [[email protected] conf]# make ##編譯 不要make install, 不然會覆蓋

替換nginx二進位制檔案:

[[email protected] conf]# mv /usr/local/nginx/sbin/nginx /usr/local/nginx/sbin/nginx.bak
[[email protected]
conf]# cp ./objs/nginx /usr/local/nginx/sbin/ [[email protected] conf]# pkill -9 nginx ## 關閉nginx [[email protected]x conf]# /usr/local/nginx/sbin/nginx ## 啟動nginx

修改nginx.conf配置,實驗安裝是否成功:

http {
    ...
    vhost_traffic_status_zone;  
    vhost_traffic_status_filter_by_host on;
    ...
    server {
        ...   
        location /status {
               vhost_traffic_status_display;
               vhost_traffic_status_display_format html;
        }
}

配置解析: 1.1 開啟vhost過濾: vhost_traffic_status_filter_by_host on; 開啟此功能,在Nginx配置有多個server_name的情況下,會根據不同的server_name進行流量的統計,否則預設會把流量全部計算到第一個server_name上。

1.2.在不想統計流量的server區域禁用vhost_traffic_status,配置示例:

server {
...
vhost_traffic_status off;
...
}

假如nginx沒有規範配置server_name或者無需進行監控的server上,那麼建議在此vhost上禁用統計監控功能。否則會出現“127.0.0.1”,hostname等的域名監控資訊。

開啟瀏覽器訪問ip/status: 在這裡插入圖片描述

2.安裝nginx-vts-exporter
[[email protected] conf]# wget -O nginx-vts-exporter-0.5.zip https://github.com/hnlq715/nginx-vts-exporter/archive/v0.5.zip
[[email protected] conf]# unzip nginx-vts-exporter-0.5.zip
[[email protected] conf]# mv nginx-vts-exporter-0.5  /usr/local/prometheus/nginx-vts-exporter
[[email protected] conf]# chmod +x /usr/local/prometheus/nginx-vts-exporter/bin/nginx-vts-exporter

使用supervisor程序管理工具啟動nginx-vts-exporter(安裝supervisor請參考:CentOS7 supervisor安裝、配置、實戰):

[[email protected] conf]# yum install epel-release -y
[[email protected] conf]# yum install supervisor -y
[[email protected] conf]# tail -n 25 /etc/supervisord.conf 
……
[include]
files = supervisord.d/*.ini

[program:nginx_exporter]
command=/usr/local/prometheus/nginx-vts-exporter/bin/nginx-vts-exporter -nginx.scrape_uri=http://10.100.xx.xxx/status/format/json
stdout_logfile=/tmp/prometheus/nginx-vts-exporter.log
autostart=true
autorestart=true
startsecs=5
priority=1
user=root
stopasgroup=true
killasgroup=true

[program:node_exporter]
command=/usr/local/bin/node_exporter
stdout_logfile=/tmp/prometheus/prometheus.log
autostart=true
autorestart=true
startsecs=5
priority=1
user=root
stopasgroup=true
killasgroup=true
……
[[email protected] conf]# chkconfig supervisord on
[[email protected] conf]# service supervisord start
[[email protected] conf]#  supervisorctl status
nginx_exporter                   RUNNING   pid 28008, uptime 0:05:57
node_exporter                    RUNNING   pid 28007, uptime 0:05:57

防火牆開啟相應的埠:

[[email protected] conf]# vi /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 9913 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 9100 -j ACCEPT
[[email protected] conf]# service iptables restart

開啟瀏覽器訪問http://10.100.xx.xxx/9913: 在這裡插入圖片描述

3.配置prometheus 新增nginx監控

修改prometheus配置(我的環境是使用consul動態的管理配置,請參考我的另一篇博文prometheus配置檔案動態管理


[[email protected] conf]#  vi prometheus.yml  ##prometheus 主配置檔案
……
##新增
- job_name: 'nginx'
  file_sd_configs:
  - refresh_interval: 1m
    files:
    - ./conf.d/nginx*.json
……
[[email protected] conf]# vim nginx-discovery.ctmpl  ## 建立nginx 自動發現模版檔案 nginx-discovery.ctmpl 
[
{$ range tree "prometheus/nginx" $}
{
"targets": ["{$ .Value $}"],
"labels": {
"instance": "{$ .Key $}"
}
},
{$ end $}
{
"targets": ["10.100.xxx.xxx:9090"],
"labels": {
"instance": "prometheus01"
}
}
] 
[[email protected] conf]# vim consul-template.conf  ## consul 主配置檔案
……
## 新增
template {
source = "/usr/local/prometheus/templates/nginx-discovery.ctmpl"
destination = "/usr/local/prometheus/conf.d/nginx-discovery.json"
command = ""
backup = true
command_timeout = "60s"
left_delimiter = "{$"
right_delimiter = "$}"
wait {
min = "2s"
max = "20s"
}
}
……
[[email protected] conf]#  curl  -XPOST localhost:9090/-/reload ## 重新載入prometheus
[[email protected] conf]#  supervisorctl restart consul-template  ## 重新載入consul-template

配置consul 實現動態管理 監控nginx檔案: 在這裡插入圖片描述 在這裡插入圖片描述

來看看利用consul-template 動態生成的nginx 監控配置檔案:

[[email protected] conf]# cat nginx-discovery.json
[

{
"targets": ["10.100.xx.xxx:9913"],
"labels": {
"instance": "pre-nginx_10.100.xx.xxx"
}
},

{
"targets": ["10.100.xx.xxx:9913"],
"labels": {
"instance": "pre-nginx_10.100.xx.xxx"
}
},

{
"targets": ["10.100.xx.xxx:9090"],
"labels": {
"instance": "prometheus01"
}
}
]

prometheus監控介面:

在這裡插入圖片描述

大功告成!!