prometheus mesos docker tomcat jvm

說明：基於環境mesos+marathon+docker+prometheus+grafana監控tomcat
一、配置環境
(1)dockerfile

from tomcat
COPY tomcat-users.xml /usr/local/tomcat/conf/
COPY server.xml /usr/local/tomcat/conf/
COPY context.xml /usr/local/tomcat/webapps/manager/META-INF/
COPY catalina.sh /usr/local/tomcat/bin/
COPY jmx_prometheus_javaagent-0.2.0.jar catalina.sh /usr/local/tomcat/
COPY config.yaml /usr/local/tomcat/
RUN mkdir /usr/local/tomcat/webapps/docker
COPY health.jsp /usr/local/tomcat/webapps/docker/health.jsp
ENV LANG C.UTF-8
RUN rm -rf /usr/local/tomcat/webapps/ROOT && rm -rf /usr/local/tomcat/webapps/docs && rm -rf /usr/local/tomcat/webapps/examples && rm -rf /usr/local/tomcat/webapps/host-manager && rm -rf /usr/local/tomcat/webapps/manager
EXPOSE 8080 1234
 

註釋：基於tomcat8官方鏡像，對於server.xml，tomcat-users.xml，context.xm自己配置

(2)catalina.sh
增加：JAVA_OPTS="$JAVA_OPTS -Duser.timezone=GMT+08 -javaagent:$PWD/jmx_prometheus_javaagent-0.2.0.jar=1234:$PWD/config.yaml -Djava.protocol.handler.pkgs=org.apache.catalina.webresources"
(3)創建docker鏡像
docker build -t 192.168.10.81:5000/tomcat8:mytest .

二、prometheus配置
(1)添加監控

- job_name: tomcat
    static_configs:
      - targets: [‘192.168.10.81:5032‘]
        labels:
          instance: mytest

說明： - targets: [‘192.168.10.81:5032‘] IP為創建虛擬機的mesos-slaveIP，端口為marathon-lb的端口

三、驗證

curl http://192.168.10.81:5032/metrics

#HELP jmx_config_reload_failure_total Number of times configuration have failed to be reloaded.

#TYPE jmx_config_reload_failure_total counter
jmx_config_reload_failure_total 0.0
#HELP jvm_memory_bytes_used Used bytes of a given JVM memory area.
#TYPE jvm_memory_bytes_used gauge
jvm_memory_bytes_used{area="heap",} 6.60999448E8
jvm_memory_bytes_used{area="nonheap",} 2.8583308E8
#HELP jvm_memory_bytes_committed Committed (bytes) of a given JVM memory area.
#TYPE jvm_memory_bytes_committed gauge
jvm_memory_bytes_committed{area="heap",} 8.45676544E8
jvm_memory_bytes_committed{area="nonheap",} 3.04611328E8
#HELP jvm_memory_bytes_max Max (bytes) of a given JVM memory area.
#TYPE jvm_memory_bytes_max gauge
jvm_memory_bytes_max{area="heap",} 2.14695936E9
jvm_memory_bytes_max{area="nonheap",} -1.0
#HELP jvm_memory_pool_bytes_used Used bytes of a given JVM memory pool.
#TYPE jvm_memory_pool_bytes_used gauge
jvm_memory_pool_bytes_used{pool="Code Cache",} 8.7307776E7
jvm_memory_pool_bytes_used{pool="Metaspace",} 1.7892812E8
jvm_memory_pool_bytes_used{pool="Compressed Class Space",} 1.9597184E7
jvm_memory_pool_bytes_used{pool="PS Eden Space",} 130976.0
jvm_memory_pool_bytes_used{pool="PS Survivor Space",} 131072.0
jvm_memory_pool_bytes_used{pool="PS Old Gen",} 6.607374E8
#HELP jvm_memory_pool_bytes_committed Committed bytes of a given JVM memory pool.
#TYPE jvm_memory_pool_bytes_committed gauge
jvm_memory_pool_bytes_committed{pool="Code Cache",} 9.4633984E7
jvm_memory_pool_bytes_committed{pool="Metaspace",} 1.87957248E8
jvm_memory_pool_bytes_committed{pool="Compressed Class Space",} 2.2020096E7
jvm_memory_pool_bytes_committed{pool="PS Eden Space",} 524288.0
jvm_memory_pool_bytes_committed{pool="PS Survivor Space",} 524288.0
jvm_memory_pool_bytes_committed{pool="PS Old Gen",} 8.44627968E8
#HELP jvm_memory_pool_bytes_max Max bytes of a given JVM memory pool.
#TYPE jvm_memory_pool_bytes_max gauge
jvm_memory_pool_bytes_max{pool="Code Cache",} 2.5165824E8
jvm_memory_pool_bytes_max{pool="Metaspace",} -1.0
jvm_memory_pool_bytes_max{pool="Compressed Class Space",} 1.073741824E9
jvm_memory_pool_bytes_max{pool="PS Eden Space",} 524288.0
jvm_memory_pool_bytes_max{pool="PS Survivor Space",} 524288.0
jvm_memory_pool_bytes_max{pool="PS Old Gen",} 2.145910784E9
#HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
#TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 242475.12
#HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
#TYPE process_start_time_seconds gauge
process_start_time_seconds 1.518703782033E9
#HELP process_open_fds Number of open file descriptors.
#TYPE process_open_fds gauge
process_open_fds 112.0
#HELP process_max_fds Maximum number of open file descriptors.
#TYPE process_max_fds gauge
process_max_fds 1048576.0
#HELP process_virtual_memory_bytes Virtual memory size in bytes.
#TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 1.0415075328E10
#HELP process_resident_memory_bytes Resident memory size in bytes.
#TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 1.464459264E9
#HELP jvm_threads_current Current thread count of a JVM
#TYPE jvm_threads_current gauge
jvm_threads_current 95.0
#HELP jvm_threads_daemon Daemon thread count of a JVM
#TYPE jvm_threads_daemon gauge
jvm_threads_daemon 92.0
#HELP jvm_threads_peak Peak thread count of a JVM
#TYPE jvm_threads_peak gauge
jvm_threads_peak 136.0
#HELP jvm_threads_started_total Started thread count of a JVM
#TYPE jvm_threads_started_total counter
jvm_threads_started_total 1583.0
#HELP jvm_threads_deadlocked Cycles of JVM-threads that are in deadlock waiting to acquire object monitors or ownable synchronizers
#TYPE jvm_threads_deadlocked gauge
jvm_threads_deadlocked 0.0
#HELP jvm_threads_deadlocked_monitor Cycles of JVM-threads that are in deadlock waiting to acquire object monitors
#TYPE jvm_threads_deadlocked_monitor gauge
jvm_threads_deadlocked_monitor 0.0
#HELP jvm_classes_loaded The number of classes that are currently loaded in the JVM
#TYPE jvm_classes_loaded gauge
jvm_classes_loaded 27069.0
#HELP jvm_classes_loaded_total The total number of classes that have been loaded since the JVM has started execution
#TYPE jvm_classes_loaded_total counter
jvm_classes_loaded_total 33493.0
#HELP jvm_classes_unloaded_total The total number of classes that have been unloaded since the JVM has started execution
#TYPE jvm_classes_unloaded_total counter
jvm_classes_unloaded_total 6424.0
#HELP jvm_info JVM version info
#TYPE jvm_info gauge
jvm_info{version="1.8.0_151-8u151-b12-1~deb9u1-b12",vendor="Oracle Corporation",} 1.0
#HELP tomcat_bytesreceived_total Tomcat global bytesReceived
#TYPE tomcat_bytesreceived_total counter
tomcat_bytesreceived_total{port="8080",protocol="http-nio",} 4799.0
tomcat_bytesreceived_total{port="8009",protocol="ajp-nio",} 0.0
#HELP tomcat_errorcount_total Tomcat global errorCount
#TYPE tomcat_errorcount_total counter
tomcat_errorcount_total{port="8080",protocol="http-nio",} 0.0
tomcat_errorcount_total{port="8009",protocol="ajp-nio",} 0.0
#HELP tomcat_requestcount_total Tomcat global requestCount
#TYPE tomcat_requestcount_total counter
tomcat_requestcount_total{port="8080",protocol="http-nio",} 74407.0
tomcat_requestcount_total{port="8009",protocol="ajp-nio",} 0.0
#HELP tomcat_session_sessioncounter_total Tomcat session sessionCounter total
#TYPE tomcat_session_sessioncounter_total counter
tomcat_session_sessioncounter_total{context="/supercode",host="localhost",} 0.0
tomcat_session_sessioncounter_total{context="/docker",host="localhost",} 60463.0
tomcat_session_sessioncounter_total{context="/",host="localhost",} 9526.0
#HELP tomcat_threadpool_pollerthreadcount Tomcat threadpool pollerThreadCount
#TYPE tomcat_threadpool_pollerthreadcount gauge
tomcat_threadpool_pollerthreadcount{port="8009",protocol="ajp-nio",} 2.0
tomcat_threadpool_pollerthreadcount{port="8080",protocol="http-nio",} 2.0
#HELP tomcat_session_rejectedsessions_total Tomcat session rejectedSessions total
#TYPE tomcat_session_rejectedsessions_total counter
tomcat_session_rejectedsessions_total{context="/supercode",host="localhost",} 0.0
tomcat_session_rejectedsessions_total{context="/docker",host="localhost",} 0.0
tomcat_session_rejectedsessions_total{context="/",host="localhost",} 0.0
#HELP tomcat_session_expiredsessions_total Tomcat session expiredSessions total
#TYPE tomcat_session_expiredsessions_total counter
tomcat_session_expiredsessions_total{context="/supercode",host="localhost",} 0.0
tomcat_session_expiredsessions_total{context="/docker",host="localhost",} 60341.0
tomcat_session_expiredsessions_total{context="/",host="localhost",} 9487.0
#HELP tomcat_session_processingtime_total Tomcat session processingTime total
#TYPE tomcat_session_processingtime_total counter
tomcat_session_processingtime_total{context="/supercode",host="localhost",} 61.0
tomcat_session_processingtime_total{context="/docker",host="localhost",} 2382.0
tomcat_session_processingtime_total{context="/",host="localhost",} 467.0
#HELP tomcat_maxtime_total Tomcat global maxTime
#TYPE tomcat_maxtime_total counter
tomcat_maxtime_total{port="8080",protocol="http-nio",} 10850.0
tomcat_maxtime_total{port="8009",protocol="ajp-nio",} 0.0
#HELP tomcat_servlet_processingtime_total Tomcat servlet processingTime total
#TYPE tomcat_servlet_processingtime_total counter
tomcat_servlet_processingtime_total{module="localhost/supercode",servlet="grailsDispatcherServlet",} 110025.0
tomcat_servlet_processingtime_total{module="localhost/docker",servlet="default",} 0.0
tomcat_servlet_processingtime_total{module="localhost/supercode",servlet="default",} 0.0
tomcat_servlet_processingtime_total{module="localhost/docker",servlet="jsp",} 250431.0
tomcat_servlet_processingtime_total{module="localhost/",servlet="default",} 0.0
tomcat_servlet_processingtime_total{module="localhost/",servlet="grailsDispatcherServlet",} 550705.0
tomcat_servlet_processingtime_total{module="localhost/",servlet="groovyPagesServlet",} 0.0
tomcat_servlet_processingtime_total{module="localhost/",servlet="jsp",} 0.0
tomcat_servlet_processingtime_total{module="localhost/supercode",servlet="jsp",} 0.0
tomcat_servlet_processingtime_total{module="localhost/supercode",servlet="groovyPagesServlet",} 0.0
#HELP tomcat_threadpool_connectioncount Tomcat threadpool connectionCount
#TYPE tomcat_threadpool_connectioncount gauge
tomcat_threadpool_connectioncount{port="8009",protocol="ajp-nio",} 1.0
tomcat_threadpool_connectioncount{port="8080",protocol="http-nio",} 1.0
#HELP tomcat_bytessent_total Tomcat global bytesSent
#TYPE tomcat_bytessent_total counter
tomcat_bytessent_total{port="8080",protocol="http-nio",} 3413490.0
tomcat_bytessent_total{port="8009",protocol="ajp-nio",} 0.0
#HELP tomcat_servlet_errorcount_total Tomcat servlet errorCount total
#TYPE tomcat_servlet_errorcount_total counter
tomcat_servlet_errorcount_total{module="localhost/supercode",servlet="grailsDispatcherServlet",} 0.0
tomcat_servlet_errorcount_total{module="localhost/docker",servlet="default",} 0.0
tomcat_servlet_errorcount_total{module="localhost/supercode",servlet="default",} 0.0
tomcat_servlet_errorcount_total{module="localhost/docker",servlet="jsp",} 0.0
tomcat_servlet_errorcount_total{module="localhost/",servlet="default",} 0.0
tomcat_servlet_errorcount_total{module="localhost/",servlet="grailsDispatcherServlet",} 0.0
tomcat_servlet_errorcount_total{module="localhost/",servlet="groovyPagesServlet",} 0.0
tomcat_servlet_errorcount_total{module="localhost/",servlet="jsp",} 0.0
tomcat_servlet_errorcount_total{module="localhost/supercode",servlet="jsp",} 0.0
tomcat_servlet_errorcount_total{module="localhost/supercode",servlet="groovyPagesServlet",} 0.0
#HELP tomcat_servlet_maxtime_total Tomcat servlet maxTime total
#TYPE tomcat_servlet_maxtime_total counter
tomcat_servlet_maxtime_total{module="localhost/supercode",servlet="grailsDispatcherServlet",} 10850.0
tomcat_servlet_maxtime_total{module="localhost/docker",servlet="default",} 0.0
tomcat_servlet_maxtime_total{module="localhost/supercode",servlet="default",} 0.0
tomcat_servlet_maxtime_total{module="localhost/docker",servlet="jsp",} 4425.0
tomcat_servlet_maxtime_total{module="localhost/",servlet="default",} 0.0
tomcat_servlet_maxtime_total{module="localhost/",servlet="grailsDispatcherServlet",} 8451.0
tomcat_servlet_maxtime_total{module="localhost/",servlet="groovyPagesServlet",} 0.0
tomcat_servlet_maxtime_total{module="localhost/",servlet="jsp",} 0.0
tomcat_servlet_maxtime_total{module="localhost/supercode",servlet="jsp",} 0.0
tomcat_servlet_maxtime_total{module="localhost/supercode",servlet="groovyPagesServlet",} 0.0
#HELP tomcat_servlet_requestcount_total Tomcat servlet requestCount total
#TYPE tomcat_servlet_requestcount_total counter
tomcat_servlet_requestcount_total{module="localhost/supercode",servlet="grailsDispatcherServlet",} 345.0
tomcat_servlet_requestcount_total{module="localhost/docker",servlet="default",} 0.0
tomcat_servlet_requestcount_total{module="localhost/supercode",servlet="default",} 0.0
tomcat_servlet_requestcount_total{module="localhost/docker",servlet="jsp",} 60465.0
tomcat_servlet_requestcount_total{module="localhost/",servlet="default",} 0.0
tomcat_servlet_requestcount_total{module="localhost/",servlet="grailsDispatcherServlet",} 13597.0
tomcat_servlet_requestcount_total{module="localhost/",servlet="groovyPagesServlet",} 0.0
tomcat_servlet_requestcount_total{module="localhost/",servlet="jsp",} 0.0
tomcat_servlet_requestcount_total{module="localhost/supercode",servlet="jsp",} 0.0
tomcat_servlet_requestcount_total{module="localhost/supercode",servlet="groovyPagesServlet",} 0.0
#HELP tomcat_threadpool_keepalivecount Tomcat threadpool keepAliveCount
#TYPE tomcat_threadpool_keepalivecount gauge
tomcat_threadpool_keepalivecount{port="8009",protocol="ajp-nio",} 0.0
tomcat_threadpool_keepalivecount{port="8080",protocol="http-nio",} 0.0
#HELP tomcat_threadpool_currentthreadcount Tomcat threadpool currentThreadCount
#TYPE tomcat_threadpool_currentthreadcount gauge
tomcat_threadpool_currentthreadcount{port="8009",protocol="ajp-nio",} 10.0
tomcat_threadpool_currentthreadcount{port="8080",protocol="http-nio",} 10.0
#HELP tomcat_threadpool_currentthreadsbusy Tomcat threadpool currentThreadsBusy
#TYPE tomcat_threadpool_currentthreadsbusy gauge
tomcat_threadpool_currentthreadsbusy{port="8009",protocol="ajp-nio",} 0.0
tomcat_threadpool_currentthreadsbusy{port="8080",protocol="http-nio",} 0.0
#HELP tomcat_processingtime_total Tomcat global processingTime
#TYPE tomcat_processingtime_total counter
tomcat_processingtime_total{port="8080",protocol="http-nio",} 950611.0
tomcat_processingtime_total{port="8009",protocol="ajp-nio",} 0.0
#HELP jmx_scrape_duration_seconds Time this JMX scrape took, in seconds.
#TYPE jmx_scrape_duration_seconds gauge
jmx_scrape_duration_seconds 1.82782075
#HELP jmx_scrape_error Non-zero if this scrape failed.
#TYPE jmx_scrape_error gauge
jmx_scrape_error 0.0
#HELP jmx_config_reload_success_total Number of times configuration have successfully been reloaded.
#TYPE jmx_config_reload_success_total counter
jmx_config_reload_success_total 0.0
#HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds.
#TYPE jvm_gc_collection_seconds summary
jvm_gc_collection_seconds_count{gc="PS Scavenge",} 7920380.0
jvm_gc_collection_seconds_sum{gc="PS Scavenge",} 49157.027
jvm_gc_collection_seconds_count{gc="PS MarkSweep",} 1211.0
jvm_gc_collection_seconds_sum{gc="PS MarkSweep",} 404.145

四、grafana畫圖

#cat mytestjvm.json
{
  "__inputs": [
    {
      "name": "DS_PROMETHEUS",
      "label": "Prometheus",
      "description": "",
      "type": "datasource",
      "pluginId": "prometheus",
      "pluginName": "Prometheus"
    }
  ],
  "__requires": [
    {
      "type": "grafana",
      "id": "grafana",
      "name": "Grafana",
      "version": "4.6.3"
    },
    {
      "type": "panel",
      "id": "graph",
      "name": "Graph",
      "version": ""
    },
    {
      "type": "datasource",
      "id": "prometheus",
      "name": "Prometheus",
      "version": "1.0.0"
    },
    {
      "type": "panel",
      "id": "singlestat",
      "name": "Singlestat",
      "version": ""
    }
  ],
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": "-- Grafana --",
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "description": "Dashboard for JVM metrics with Prometheus / JMX Exporter",
  "editable": true,
  "gnetId": 3066,
  "graphTooltip": 0,
  "hideControls": true,
  "id": null,
  "links": [
    {
      "asDropdown": true,
      "icon": "external link",
      "includeVars": true,
      "keepTime": true,
      "tags": [
        "llzgnode"
      ],
      "targetBlank": true,
      "title": "OS",
      "type": "dashboards"
    },
    {
      "asDropdown": true,
      "icon": "external link",
      "includeVars": true,
      "keepTime": true,
      "tags": [
        "mysql"
      ],
      "targetBlank": true,
      "title": "mysql",
      "type": "dashboards"
    },
    {
      "asDropdown": true,
      "icon": "external link",
      "includeVars": true,
      "keepTime": true,
      "tags": [
        "tomcat"
      ],
      "targetBlank": true,
      "title": "jvm",
      "type": "dashboards"
    }
  ],
  "refresh": "30s",
  "rows": [
    {
      "collapse": false,
      "height": 35,
      "panels": [
        {
          "cacheTimeout": null,
          "colorBackground": false,
          "colorValue": false,
          "colors": [
            "#d683ce",
            "#d683ce",
            "#d683ce"
          ],
          "datasource": "${DS_PROMETHEUS}",
          "format": "none",
          "gauge": {
            "maxValue": 100,
            "minValue": 0,
            "show": false,
            "thresholdLabels": false,
            "thresholdMarkers": true
          },
          "id": 13,
          "interval": null,
          "links": [],
          "mappingType": 1,
          "mappingTypes": [
            {
              "name": "value to text",
              "value": 1
            },
            {
              "name": "range to text",
              "value": 2
            }
          ],
          "maxDataPoints": 100,
          "nullPointMode": "connected",
          "nullText": null,
          "postfix": "",
          "postfixFontSize": "50%",
          "prefix": "",
          "prefixFontSize": "50%",
          "rangeMaps": [
            {
              "from": "null",
              "text": "N/A",
              "to": "null"
            }
          ],
          "span": 2,
          "sparkline": {
            "fillColor": "rgba(31, 118, 189, 0.18)",
            "full": false,
            "lineColor": "rgb(31, 120, 193)",
            "show": false
          },
          "tableColumn": "",
          "targets": [
            {
              "expr": "tomcat_requestcount_total{port=\"8080\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "refId": "A"
            }
          ],
          "thresholds": "",
          "title": "requestcount",
          "type": "singlestat",
          "valueFontSize": "80%",
          "valueMaps": [
            {
              "op": "=",
              "text": "N/A",
              "value": "null"
            }
          ],
          "valueName": "total"
        },
        {
          "cacheTimeout": null,
          "colorBackground": false,
          "colorValue": false,
          "colors": [
            "#299c46",
            "rgba(237, 129, 40, 0.89)",
            "#d44a3a"
          ],
          "datasource": "${DS_PROMETHEUS}",
          "format": "none",
          "gauge": {
            "maxValue": 100,
            "minValue": 0,
            "show": false,
            "thresholdLabels": false,
            "thresholdMarkers": true
          },
          "id": 14,
          "interval": null,
          "links": [],
          "mappingType": 1,
          "mappingTypes": [
            {
              "name": "value to text",
              "value": 1
            },
            {
              "name": "range to text",
              "value": 2
            }
          ],
          "maxDataPoints": 100,
          "nullPointMode": "connected",
          "nullText": null,
          "postfix": "",
          "postfixFontSize": "50%",
          "prefix": "",
          "prefixFontSize": "50%",
          "rangeMaps": [
            {
              "from": "null",
              "text": "N/A",
              "to": "null"
            }
          ],
          "span": 2,
          "sparkline": {
            "fillColor": "rgba(31, 118, 189, 0.18)",
            "full": false,
            "lineColor": "rgb(31, 120, 193)",
            "show": false
          },
          "tableColumn": "",
          "targets": [
            {
              "expr": "tomcat_requestcount_total{port=\"8080\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "refId": "A"
            }
          ],
          "thresholds": "",
          "title": "Requestcount Current",
          "type": "singlestat",
          "valueFontSize": "80%",
          "valueMaps": [
            {
              "op": "=",
              "text": "N/A",
              "value": "null"
            }
          ],
          "valueName": "current"
        },
        {
          "cacheTimeout": null,
          "colorBackground": false,
          "colorValue": false,
          "colors": [
            "#299c46",
            "rgba(237, 129, 40, 0.89)",
            "#d44a3a"
          ],
          "datasource": "${DS_PROMETHEUS}",
          "format": "none",
          "gauge": {
            "maxValue": 100,
            "minValue": 0,
            "show": false,
            "thresholdLabels": false,
            "thresholdMarkers": true
          },
          "id": 15,
          "interval": null,
          "links": [],
          "mappingType": 1,
          "mappingTypes": [
            {
              "name": "value to text",
              "value": 1
            },
            {
              "name": "range to text",
              "value": 2
            }
          ],
          "maxDataPoints": 100,
          "nullPointMode": "connected",
          "nullText": null,
          "postfix": "",
          "postfixFontSize": "50%",
          "prefix": "",
          "prefixFontSize": "50%",
          "rangeMaps": [
            {
              "from": "null",
              "text": "N/A",
              "to": "null"
            }
          ],
          "span": 2,
          "sparkline": {
            "fillColor": "rgba(31, 118, 189, 0.18)",
            "full": false,
            "lineColor": "rgb(31, 120, 193)",
            "show": false
          },
          "tableColumn": "",
          "targets": [
            {
              "expr": "tomcat_requestcount_total{port=\"8080\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "refId": "A"
            }
          ],
          "thresholds": "",
          "title": "Requestcount  Average",
          "type": "singlestat",
          "valueFontSize": "80%",
          "valueMaps": [
            {
              "op": "=",
              "text": "N/A",
              "value": "null"
            }
          ],
          "valueName": "avg"
        },
        {
          "cacheTimeout": null,
          "colorBackground": false,
          "colorValue": false,
          "colors": [
            "#299c46",
            "rgba(237, 129, 40, 0.89)",
            "#d44a3a"
          ],
          "datasource": "${DS_PROMETHEUS}",
          "format": "none",
          "gauge": {
            "maxValue": 100,
            "minValue": 0,
            "show": false,
            "thresholdLabels": false,
            "thresholdMarkers": true
          },
          "id": 16,
          "interval": null,
          "links": [],
          "mappingType": 1,
          "mappingTypes": [
            {
              "name": "value to text",
              "value": 1
            },
            {
              "name": "range to text",
              "value": 2
            }
          ],
          "maxDataPoints": 100,
          "nullPointMode": "connected",
          "nullText": null,
          "postfix": "",
          "postfixFontSize": "50%",
          "prefix": "",
          "prefixFontSize": "50%",
          "rangeMaps": [
            {
              "from": "null",
              "text": "N/A",
              "to": "null"
            }
          ],
          "span": 2,
          "sparkline": {
            "fillColor": "rgba(31, 118, 189, 0.18)",
            "full": false,
            "lineColor": "rgb(31, 120, 193)",
            "show": false
          },
          "tableColumn": "",
          "targets": [
            {
              "expr": "tomcat_errorcount_total{port=\"8080\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "refId": "A"
            }
          ],
          "thresholds": "",
          "title": "Errorcount",
          "type": "singlestat",
          "valueFontSize": "80%",
          "valueMaps": [
            {
              "op": "=",
              "text": "N/A",
              "value": "null"
            }
          ],
          "valueName": "total"
        },
        {
          "cacheTimeout": null,
          "colorBackground": false,
          "colorValue": false,
          "colors": [
            "#299c46",
            "rgba(237, 129, 40, 0.89)",
            "#d44a3a"
          ],
          "datasource": "${DS_PROMETHEUS}",
          "format": "bytes",
          "gauge": {
            "maxValue": 100,
            "minValue": 0,
            "show": false,
            "thresholdLabels": false,
            "thresholdMarkers": true
          },
          "id": 18,
          "interval": null,
          "links": [],
          "mappingType": 1,
          "mappingTypes": [
            {
              "name": "value to text",
              "value": 1
            },
            {
              "name": "range to text",
              "value": 2
            }
          ],
          "maxDataPoints": 100,
          "nullPointMode": "connected",
          "nullText": null,
          "postfix": "",
          "postfixFontSize": "50%",
          "prefix": "",
          "prefixFontSize": "50%",
          "rangeMaps": [
            {
              "from": "null",
              "text": "N/A",
              "to": "null"
            }
          ],
          "span": 2,
          "sparkline": {
            "fillColor": "rgba(31, 118, 189, 0.18)",
            "full": false,
            "lineColor": "rgb(31, 120, 193)",
            "show": false
          },
          "tableColumn": "",
          "targets": [
            {
              "expr": "jvm_memory_bytes_max{area=\"heap\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "refId": "A"
            }
          ],
          "thresholds": "",
          "title": "Jvm Max Mem heap",
          "type": "singlestat",
          "valueFontSize": "80%",
          "valueMaps": [
            {
              "op": "=",
              "text": "N/A",
              "value": "null"
            }
          ],
          "valueName": "avg"
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 341,
      "panels": [
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "${DS_PROMETHEUS}",
          "fill": 1,
          "id": 7,
          "legend": {
            "alignAsTable": true,
            "avg": true,
            "current": true,
            "max": true,
            "min": true,
            "rightSide": false,
            "show": true,
            "total": true,
            "values": true
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 6,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "expr": "jvm_memory_bytes_used{area=\"heap\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "legendFormat": "已經使用的堆內存",
              "refId": "A"
            },
            {
              "expr": "jvm_memory_bytes_used{area=\"nonheap\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "legendFormat": "已經使用的非堆內存",
              "refId": "B"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "Mem Used",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "decbytes",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        },
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "${DS_PROMETHEUS}",
          "fill": 1,
          "id": 8,
          "legend": {
            "alignAsTable": true,
            "avg": true,
            "current": true,
            "max": true,
            "min": true,
            "rightSide": false,
            "show": true,
            "total": true,
            "values": true
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 6,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "expr": "jvm_memory_bytes_committed{area=\"heap\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "legendFormat": "已經提交的堆內存",
              "refId": "A"
            },
            {
              "expr": "jvm_memory_bytes_committed{area=\"nonheap\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "legendFormat": "已經提交的非堆內存",
              "refId": "B"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "Mem Committed",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "decbytes",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 370,
      "panels": [
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "${DS_PROMETHEUS}",
          "fill": 1,
          "id": 4,
          "legend": {
            "alignAsTable": true,
            "avg": true,
            "current": true,
            "max": true,
            "min": true,
            "rightSide": true,
            "show": true,
            "total": false,
            "values": true
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 12,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "expr": "jvm_memory_pool_bytes_committed{pool=\"Code Cache\",instance=~\"$instance\"} ",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 1,
              "legendFormat": "Code Cache",
              "refId": "A"
            },
            {
              "expr": "jvm_memory_pool_bytes_committed{pool=\"Metaspace\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 1,
              "legendFormat": "Metaspace",
              "refId": "B"
            },
            {
              "expr": "jvm_memory_pool_bytes_committed{pool=\"Compressed Class Space\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 1,
              "legendFormat": "Compressed Class Space",
              "refId": "C"
            },
            {
              "expr": "jvm_memory_pool_bytes_committed{pool=\"PS Eden Space\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 1,
              "legendFormat": "PS Eden Spac",
              "refId": "D"
            },
            {
              "expr": "jvm_memory_pool_bytes_committed{pool=\"PS Survivor Space\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 1,
              "legendFormat": "PS Survivor Space",
              "refId": "E"
            },
            {
              "expr": "jvm_memory_pool_bytes_committed{pool=\"PS Old Gen\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 1,
              "legendFormat": "PS Old Gen",
              "refId": "F"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "Pool committed",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "bytes",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        },
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "${DS_PROMETHEUS}",
          "fill": 1,
          "id": 9,
          "legend": {
            "alignAsTable": true,
            "avg": true,
            "current": true,
            "max": true,
            "min": true,
            "rightSide": true,
            "show": true,
            "total": false,
            "values": true
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 12,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "expr": "jvm_memory_pool_bytes_used{pool=\"Code Cache\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "Code Cache",
              "refId": "A"
            },
            {
              "expr": "jvm_memory_pool_bytes_used{pool=\"Metaspace\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "Metaspace",
              "refId": "B"
            },
            {
              "expr": "jvm_memory_pool_bytes_used{pool=\"Compressed Class Space\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "Compressed Class Space",
              "refId": "C"
            },
            {
              "expr": "jvm_memory_pool_bytes_used{pool=\"PS Eden Space\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "PS Eden Space",
              "refId": "D"
            },
            {
              "expr": "jvm_memory_pool_bytes_used{pool=\"PS Survivor Space\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "PS Survivor Space",
              "refId": "E"
            },
            {
              "expr": "jvm_memory_pool_bytes_used{pool=\"PS Old Gen\",instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "PS Old Gen",
              "refId": "F"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "Pool User",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "decbytes",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        },
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "${DS_PROMETHEUS}",
          "fill": 1,
          "id": 17,
          "legend": {
            "alignAsTable": true,
            "avg": true,
            "current": false,
            "max": true,
            "min": true,
            "rightSide": true,
            "show": true,
            "total": true,
            "values": true
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 12,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "expr": "jvm_gc_collection_seconds_count{gc=\"PS Scavenge\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 1,
              "legendFormat": "PS Scavenge",
              "refId": "A"
            },
            {
              "expr": "jvm_gc_collection_seconds_sum{gc=\"PS Scavenge\",instance=~\"$instance\"} ",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 1,
              "legendFormat": "PS Scavenge SUM",
              "refId": "B"
            },
            {
              "expr": "jvm_gc_collection_seconds_count{gc=\"PS MarkSweep\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 1,
              "legendFormat": "PS MarkSweep",
              "refId": "C"
            },
            {
              "expr": "jvm_gc_collection_seconds_sum{gc=\"PS MarkSweep\",instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 1,
              "legendFormat": "PS MarkSweep SUM",
              "refId": "D"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "Count Gc",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "bytes",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 363,
      "panels": [
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "${DS_PROMETHEUS}",
          "fill": 1,
          "id": 6,
          "legend": {
            "alignAsTable": true,
            "avg": true,
            "current": true,
            "max": true,
            "min": true,
            "rightSide": true,
            "show": true,
            "total": true,
            "values": true
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 12,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "expr": "jvm_threads_current{instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "當前的線程連接數",
              "refId": "A"
            },
            {
              "expr": "jvm_threads_started_total{instance=~\"$instance\"}",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "legendFormat": "線程數總和",
              "refId": "B"
            },
            {
              "expr": "jvm_threads_daemon{instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "daemon",
              "refId": "C"
            },
            {
              "expr": "jvm_threads_peak{instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "線程峰值",
              "refId": "D"
            },
            {
              "expr": "jvm_threads_deadlocked{instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "deadlocked",
              "refId": "E"
            },
            {
              "expr": "jvm_threads_deadlocked_monitor{instance=~\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "deadlocked_monitor",
              "refId": "F"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "threads",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "none",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        },
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "${DS_PROMETHEUS}",
          "fill": 1,
          "id": 11,
          "legend": {
            "alignAsTable": true,
            "avg": true,
            "current": true,
            "max": true,
            "min": true,
            "rightSide": true,
            "show": true,
            "total": true,
            "values": true
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 12,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "expr": "jvm_classes_loaded{instance=\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "已經加載的類",
              "refId": "A"
            },
            {
              "expr": "jvm_classes_loaded_total{instance=\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "加載類的總和",
              "refId": "B"
            },
            {
              "expr": "jvm_classes_unloaded_total{instance=\"$instance\"}",
              "format": "time_series",
              "intervalFactor": 2,
              "legendFormat": "已卸載的類",
              "refId": "C"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "Classes",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "none",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 250,
      "panels": [
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "${DS_PROMETHEUS}",
          "fill": 1,
          "id": 19,
          "legend": {
            "alignAsTable": true,
            "avg": true,
            "current": false,
            "max": true,
            "min": true,
            "rightSide": false,
            "show": true,
            "total": true,
            "values": true
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [
            {
              "alias": "output",
              "transform": "negative-Y"
            }
          ],
          "spaceLength": 10,
          "span": 12,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "expr": "sum(irate(tomcat_bytesreceived_total{port=\"8080\",protocol=\"http-nio\",instance=~\"$instance\"}[10m]))",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "legendFormat": "input",
              "refId": "A"
            },
            {
              "expr": "sum(irate(tomcat_bytessent_total{port=\"8080\",protocol=\"http-nio\",instance=~\"$instance\"}[10m]))",
              "format": "time_series",
              "hide": false,
              "intervalFactor": 2,
              "legendFormat": "output",
              "refId": "B"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "Network",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "Bps",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "decbytes",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    }
  ],
  "schemaVersion": 14,
  "style": "dark",
  "tags": [
    "JVM",
    "tomcat"
  ],
  "templating": {
    "list": [
      {
        "allValue": "",
        "current": {},
        "datasource": "${DS_PROMETHEUS}",
        "hide": 0,
        "includeAll": true,
        "label": "Instance",
        "multi": false,
        "name": "instance",
        "options": [],
        "query": "label_values(jvm_memory_bytes_max,instance)",
        "refresh": 1,
        "regex": "",
        "sort": 0,
        "tagValuesQuery": "",
        "tags": [],
        "tagsQuery": "",
        "type": "query",
        "useTags": false
      }
    ]
  },
  "time": {
    "from": "now-6h",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ],
    "time_options": [
      "5m",
      "15m",
      "1h",
      "6h",
      "12h",
      "24h",
      "2d",
      "7d",
      "30d"
    ]
  },
  "timezone": "",
  "title": "tomcat",
  "version": 44
}

prometheus+grafana+docker 監控tomcat jvm