1. 程式人生 > >zabbix使用自動發現功能監控伺服器各JVM程序狀

zabbix使用自動發現功能監控伺服器各JVM程序狀

前言

==========

為什麼需要做伺服器jvm自動發現的監控呢?這個事情主要有兩點原因:

    1.zabbix預設監控jvm狀態是使用jmx中轉進行監控的,監控效率比較低下

    2.zabbix使用jmx監控jvm的時候由於一個主機上的鍵值不能重複,也就導致了一臺主機上只能監控一個jvm例項

    以上兩點原因導致zabbix通過jmx監控jvm的實現不是很理想,加上最近老大要求收集伺服器上面跑的所有java應用的資訊,於是自己琢磨了下,還是自己動手,豐衣足食。利用了週末的時間,通過使用shell指令碼+java工具jstat+zabbix實現監控主機上多jvm例項的功能。

第一章:概念的理解

 首先,既然要監控jvm狀態,那就必須要了解jvm裡面的資訊,樓主通過搜尋資料加自動腦補,把網上的資料取其精華,去其糟粕,整理了一下。JVM中的記憶體分類分為堆記憶體和非堆記憶體,堆記憶體是給實際應用使用的,非堆記憶體是給jvm容器使用的。我們主要關心的是堆記憶體這塊。在堆記憶體裡面,給記憶體分為如下幾塊:

 1.Young代(年輕代)

 2.Old代(老年代)

 3.Perm代(永久代)(關於這一點,在JDK7和JDK8中情況不一樣,將在後面進行分析)

 其中,年輕代裡面又分成了三塊,如下:

 1.Eden代(伊甸園代)

 2.survivor0代(0號倖存區)

 3.survivor1代(1號倖存區)

 至於更詳細的關於JVM堆記憶體的資訊,各位可以自行百度或者google,我這裡就不贅述了,畢竟我也是個半桶水,自己找了點資料外加腦補到的一些東西,不敢在關公門前耍大刀了。

 當然,還得科普一個東西,那就是GC,所謂的GC就是JVM在執行的時候會有一個垃圾回收機制,這個垃圾回收機制是什麼情況呢?就是在程式執行的時候會產生很多已經不使用的空間,但還是被佔用了的情況,這樣會造成很多不必要的浪費,於是JVM就有一個垃圾回收機制,針對程式中已經不使用的記憶體資源,會進行回收釋放,這個過程就叫做GC。當然,關於GC還有很多內容我這裡也沒有詳述,理由同上條。各位看官只需要知道GC是JVM監控裡面的一個很重要的引數就行了。

 第一章,關於JVM中概念的理解結束了,預知後事如何,請聽下回分解。

第二章:JAVA工具的選用

 java工具有很多,關於jvm監控的工具主要有如下幾個:

 + jstat

 + jmap

 + jstack

 其中jmap --heap pid可以抓出挺多的關於某個jvm的執行引數,但是老大提醒我最好不要使用jmap進行jvm監控,具體沒有說明原因。於是本著打破砂鍋問到底的精神,我又去搜了一把,發現瞭如下內容:

 jmap最主要的危險操作是下面這三種: 

1. jmap -dump 

這個命令執行,JVM會將整個heap的資訊dump寫入到一個檔案,heap如果比較大的話,就會導致這個過程比較耗時,並且執行的過程中為了保證dump的資訊是可靠的,所以會暫停應用。

2. jmap -permstat 

這個命令執行,JVM會去統計perm區的狀況,這整個過程也會比較的耗時,並且同樣也會暫停應用。

3. jmap -histo:live 

這個命令執行,JVM會先觸發gc,然後再統計資訊。

上面的這三個操作都將對應用的執行產生影響,所以建議如果不是很有必要的話,不要去執行。

所以,從上面三點來看,jmap命令對jvm狀態影響還是比較大的,而且執行jmap --heap的時間也比較長,效率較低,予以排除。

接下來是jstack,這個命令可以深入到JVM裡面對JVM執行問題進行排查,據說還可以統計JVM裡面的執行緒數量。但是這個命令執行效率也比較低,被排除掉了。

於是剩下的只有一個jstat命令了。下面來詳細的講解該命令的使用了,咳咳,各位快點打起點精神來,這可是重頭戲來了。

首先,列出jstat命令的一些使用案例吧:

<span style="color:#333333"><span style="color:black"><code class="language-bash"><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span>
1.jstat -gc pid
            可以顯示gc的資訊,檢視gc的次數,及時間。
            其中最後五項,分別是young gc的次數,young gc的時間,full gc的次數,full gc的時間,gc的總時間。
S0C    S1C    S0U    S1U      EC       EU        OC         OU       PC     PU    YGC     YGCT    FGC    FGCT     GCT   
9792.0 10048.0  0.0   5143.2 242048.0 220095.4  323200.0   211509.3  186368.0 114451.6    317    4.850   4      0.971    5.821
 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT   
1024.0 1024.0  0.0   320.0  11776.0  11604.6   260608.0   149759.6  39344.0 38142.6 4528.0 4303.1   5473   24.010   2      0.128   24.138
2.jstat -gccapacity pid
            可以顯示,VM記憶體中三代(young,old,perm)物件的使用和佔用大小,
            如 PGCMN顯示的是最小perm的記憶體使用量,PGCMX顯示的是perm的記憶體最大使用量,
            PGC是當前新生成的perm記憶體佔用量,PC是但前perm記憶體佔用量。
            其他的可以根據這個類推, OC是old內純的佔用量。
 NGCMN    NGCMX     NGC     S0C   S1C       EC      OGCMN      OGCMX       OGC         OC      PGCMN    PGCMX     PGC       PC     YGC    FGC 
 87360.0 262144.0 262144.0 9792.0 10048.0 242048.0   174784.0   786432.0   323200.0   323200.0 131072.0 262144.0 186368.0 186368.0    317     4
 NGCMN    NGCMX     NGC     S0C   S1C       EC      OGCMN      OGCMX       OGC         OC       MCMN     MCMX      MC     CCSMN    CCSMX     CCSC    YGC    FGC 
 1536.0 174592.0  13312.0  512.0  512.0  11776.0   260608.0   349696.0   260608.0   260608.0      0.0 1083392.0  39344.0      0.0 1048576.0   4528.0   5474     2
3.jstat -gcutil pid
            統計gc資訊統計。
  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT   
  0.00  51.19  83.29  65.44  61.41    317    4.850     4    0.971    5.821
  
  S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT   
 68.75   0.00  46.74  57.47  96.95  95.03   5474   24.014     2    0.128   24.143
4.jstat -gcnew pid
           年輕代物件的資訊。
 S0C    S1C    S0U    S1U   TT MTT  DSS      EC       EU     YGC     YGCT  
9792.0 10048.0    0.0 5143.2  3  15 9792.0 242048.0 198653.2    317    4.850
 S0C    S1C    S0U    S1U   TT MTT  DSS      EC       EU     YGC     YGCT  
 512.0  512.0  352.0    0.0 15  15  512.0  11776.0   8446.4   5474   24.014
5.jstat -gcnewcapacity pid
           年輕代物件的資訊及其佔用量。
NGCMN      NGCMX       NGC      S0CMX     S0C     S1CMX     S1C       ECMX        EC      YGC   FGC 
   87360.0   262144.0   262144.0  87360.0   9792.0  87360.0  10048.0   262016.0   242048.0   317     4
NGCMN      NGCMX       NGC      S0CMX     S0C     S1CMX     S1C       ECMX        EC      YGC   FGC 
   1536.0   174592.0    13312.0  57856.0    512.0  57856.0    512.0   173568.0    11776.0  5475     2
6.jstat -gcold pid
          old代物件的資訊。
   PC       PU        OC          OU       YGC    FGC    FGCT     GCT   
186368.0 114451.6    323200.0    211509.3    317     4    0.971    5.821
   MC       MU      CCSC     CCSU       OC          OU       YGC    FGC    FGCT     GCT   
 39344.0  38142.6   4528.0   4303.1    260608.0    149783.6   5475     2    0.128   24.148
7.jstat -gcoldcapacity pid
          old代物件的資訊及其佔用量。
   OGCMN       OGCMX        OGC         OC       YGC   FGC    FGCT     GCT   
   174784.0    786432.0    323200.0    323200.0   317     4    0.971    5.821
   OGCMN       OGCMX        OGC         OC       YGC   FGC    FGCT     GCT   
   260608.0    349696.0    260608.0    260608.0  5475     2    0.128   24.148
   
   
8.jstat -gcpermcapacity pid
          perm物件的資訊及其佔用量。
  PGCMN      PGCMX       PGC         PC      YGC   FGC    FGCT     GCT   
  131072.0   262144.0   186368.0   186368.0   317     4    0.971    5.821
沒有
9.jstat -class pid
          顯示載入class的數量,及所佔空間等資訊。
Loaded  Bytes  Unloaded  Bytes     Time   
 25315 45671.7     5976  7754.1      15.19
Loaded  Bytes  Unloaded  Bytes     Time   
  6472 11893.0        0     0.0       5.97
10.jstat -compiler pid
          顯示VM實時編譯的數量等資訊。
Compiled Failed Invalid   Time   FailedType FailedMethod
    4219      3       0    63.36          1 org/aspectj/weaver/ResolvedType addAndRecurse
Compiled Failed Invalid   Time   FailedType FailedMethod
   11364      1       0   107.53          1 sun/nio/cs/UTF_8<span style="color:#ee9900">$Decoder</span> decode
11.stat -printcompilation pid
          當前VM執行的資訊。
Compiled  Size  Type Method
    4219   2232    1 net/spy/memcached/protocol/ascii/BaseGetOpImpl initialize
Compiled  Size  Type Method
   11364    212    1 com/alibaba/rocketmq/client/impl/consumer/RebalanceService run
   <span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span></code></span></span>

   可以看出上面我列出的命令執行結果為什麼有兩行呢,這是因為是用不同的jdk版本執行的。

   上面是JDK7執行結果,下面是JDK8執行結果,這兩個版本之間輸出的結果是有差距的,下面,就來分析為什麼會產生這種差異。

JDK7和JDK8中JVM堆記憶體劃分差異

   如果記性好的童鞋們應該還能記得我上面在介紹JVM堆記憶體分類的時候括號裡寫的那個東東吧,沒錯,就是這個東西導致的。在JDK7中的Perm代(永久代)在JDK8中被廢除了,取而代之的是Metadata代(元資料代),據說這個元資料代相對於永久代進行了優化,如果不設定最大值的話,預設會按需增長, 不會造成像Perm代中記憶體佔滿後會爆出記憶體溢位的錯誤,元資料代也可以設定最大值,這樣的話,當記憶體區域被消耗完的時候將會和Perm代一樣爆出記憶體溢位的錯誤。(PS:原諒我的班門弄斧,只能解釋到這一個層面了。)

好了,解釋清楚了JDK7和JDK8的差異以後,接下來我們來解釋jstat抓到的這些引數了。

<span style="color:#333333"><span style="color:black"><code class="language-bash">jstat命令獲取引數解析
<span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span>
* S0C 年輕代中第一個survivor(倖存區)的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$1*1024</span>}'</span>
* S0U 年輕代中第一個survivor(倖存區)目前已使用空間 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$3*1024</span>}'</span>
* S0 年輕代中第一個survivor(倖存區)已使用的佔當前容量百分比jstat -gcutil <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$1</span>}'</span>
* S0CMX 年輕代中第一個survivor(倖存區)的最大容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnewcapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$4*1024</span>}'</span>
* 
* S1C 年輕代中第二個survivor(倖存區)的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$2*1024</span>}'</span>
* S1U 年輕代中第二個survivor(倖存區)目前已使用空間 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$4*1024</span>}'</span>
* S1 年輕代中第二個survivor(倖存區)已使用的佔當前容量百分比jstat -gcutil <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$2</span>}'</span>
* S1CMX  年輕代中第二個survivor(倖存區)的最大容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnewcapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$6*1024</span>}'</span>
* DSS 當前需要survivor(倖存區)的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>(Eden區已滿)jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$7*1024</span>}'</span>
* 
* EC 年輕代中Eden(伊甸園)的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$8*1024</span>}'</span>
* EU 年輕代中Eden(伊甸園)目前已使用空間 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$9*1024</span>}'</span>
* ECMX 年輕代中Eden(伊甸園)的最大容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcnewcapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$8*1024</span>}'</span>
* E 年輕代中Eden(伊甸園)已使用的佔當前容量百分比jstat -gcutil <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$3</span>}'</span>
* 
* NGCMN 年輕代<span style="color:#999999">(</span>young<span style="color:#999999">)</span>中初始化<span style="color:#999999">(</span>最小<span style="color:#999999">)</span>的大小 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$1*1024</span>}'</span>
* NGCMX 年輕代<span style="color:#999999">(</span>young<span style="color:#999999">)</span>的最大容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$2*1024</span>}'</span>
* NGC 年輕代<span style="color:#999999">(</span>young<span style="color:#999999">)</span>中當前的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$3*1024</span>}'</span>
* 
* OC Old代的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcold <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$3*1024</span>}'</span>
* OU Old代目前已使用空間 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gcold <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$4*1024</span>}'</span>
* OGCMX old代的最大容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$8*1024</span>}'</span>
* OGCMN old代中初始化<span style="color:#999999">(</span>最小<span style="color:#999999">)</span>的大小 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$7*1024</span>}'</span>
* O old代已使用的佔當前容量百分比jstat -gcutil <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$4</span>}'</span>
* OGC old代當前新生成的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$9*1024</span>}'</span>
* 
* PC Perm<span style="color:#999999">(</span>持久代<span style="color:#999999">)</span>的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$14*1024</span>}'</span>
* PU Perm<span style="color:#999999">(</span>持久代<span style="color:#999999">)</span>目前已使用空間 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gc <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$10*1024</span>}'</span>
* PGCMX perm代的最大容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$12*1024</span>}'</span>
* PGCMN perm代中初始化<span style="color:#999999">(</span>最小<span style="color:#999999">)</span>的大小 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$11*1024</span>}'</span>
* P perm代已使用的佔當前容量百分比 jstat -gcutil <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$5*1024</span>}'</span>
* PGC perm代當前新生成的容量 <span style="color:#999999">(</span>位元組<span style="color:#999999">)</span>jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$13*1024</span>}'</span>
* 
* YGC 從應用程式啟動到取樣時年輕代中gc次數jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$15</span>}'</span>
* YGCT 從應用程式啟動到取樣時年輕代中gc所用時間<span style="color:#999999">(</span>s<span style="color:#999999">)</span>jstat -gcutil <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$7</span>}'</span>
* FGC從應用程式啟動到取樣時old代<span style="color:#999999">(</span>全gc<span style="color:#999999">)</span>gc次數jstat -gccapacity <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$16</span>}'</span>
* FGCT 從應用程式啟動到取樣時old代<span style="color:#999999">(</span>全gc<span style="color:#999999">)</span>gc所用時間<span style="color:#999999">(</span>s<span style="color:#999999">)</span>jstat -gcutil <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$9</span>}'</span>
* GCT 從應用程式啟動到取樣時gc用的總時間<span style="color:#999999">(</span>s<span style="color:#999999">)</span>jstat -gcutil <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$10</span>}'</span>
* 
* TT  持有次數限制jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$5</span>}'</span>
* MTT   最大持有次數限制jstat -gcnew <span style="color:#ee9900">$pid</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -1<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$6</span>}'</span>
*
* Loadedjvm載入class數量
* Unloadedjvm未載入class數量
*
* M元資料區使用比例
* MC當前元資料空間大小
* MU元資料空間使用大小
* MCMN最小元資料容量 
* MCMX最大元資料容量
* 
* CCS壓縮使用比例
* CCSC當前壓縮類空間大小
* CCSU壓縮類空間使用大小
* CCSMN最小壓縮類空間大小
* CCSMX最大壓縮類空間大小
<span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span><span style="color:#9a6e3a">==</span></code></span></span>

好了,上面就是我找到的一些對jstat獲取的資料意思的統計,各位看官可以做個參考。

好了,這一章的內容到此基本結束,前面的東西都是一些理論類的東西,沒有實際的操作。俗話說,光說不練假把式。接下來,我們將開啟下一章的旅程,指令碼+jstat的使用。

第三章:指令碼+jstat獲取資料

首先,我們來看一下該章節介紹的幾個指令碼吧:

1.jvm_list.sh 獲取該機器上所有執行的JVM的程序對應的程式根目錄以及程式名稱

2.get_jvmlist.sh 將獲取的該機器上的所有程序對應的程式名稱序列化成json格式併發送給zabbix伺服器

3.get_jvmstatus.sh 通過獲取的程式根目錄獲取到對應的程式程序,再通過jstat抓取資料寫入到檔案中快取

4.set_jvmstatus.sh zabbix通過呼叫該指令碼獲取快取檔案中的關於某個JVM程序的狀態資訊

好了,簡單介紹了上面幾個指令碼的功能,下面我們列出這幾個指令碼的實際內容:

<span style="color:#333333"><span style="color:black"><code class="language-bash">    <span style="color:slategray">#cat jvm_list.sh </span>
    <span style="color:slategray">#!/bin/bash</span>
    
    packagePath<span style="color:#9a6e3a">=</span>/usr/local/etc/scripts/package_path.txt
    <span style="color:#0077aa">echo</span> -n <span style="color:#9a6e3a">></span><span style="color:#ee9900">$packagePath</span>
    
    <span style="color:#0077aa">for</span> i <span style="color:#0077aa">in</span> <span style="color:#ee9900"><span style="color:#ee9900">`</span><span style="color:#dd4a68">ps</span> -fC java<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">tail</span> -n +2<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">grep</span> -v <span style="color:#669900">'flume'</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$2</span>}'</span><span style="color:#ee9900">`</span></span><span style="color:#999999">;</span>
    <span style="color:#0077aa">do</span>
            pgrootpath<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">`</span><span style="color:#dd4a68">ls</span> -l /proc/$i/cwd<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$NF</span>}'</span><span style="color:#ee9900">`</span></span>
            <span style="color:#0077aa">if</span> <span style="color:#999999">[</span><span style="color:#999999">[</span> -r <span style="color:#ee9900">$pgrootpath</span>/appconfig <span style="color:#999999">]</span><span style="color:#999999">]</span> <span style="color:#9a6e3a">&&</span> <span style="color:#999999">[</span>  <span style="color:#ee9900"><span style="color:#ee9900">`</span><span style="color:#dd4a68">grep</span> ^packagename<span style="color:#9a6e3a">=</span> $pgrootpath/appconfig<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">wc</span> -l<span style="color:#ee9900">`</span></span><span style="color:#9a6e3a">==</span>1 <span style="color:#999999">]</span><span style="color:#999999">;</span><span style="color:#0077aa">then</span>
                            packagename<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#dd4a68">grep</span> ^packagename<span style="color:#9a6e3a">=</span> $pgrootpath/appconfig 2<span style="color:#9a6e3a">></span>/dev/null<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> -F<span style="color:#669900">'"'</span> <span style="color:#669900">'{print <span style="color:#ee9900">$2</span>}'</span><span style="color:#ee9900">)</span></span>
            <span style="color:#0077aa">elif</span> <span style="color:#999999">[</span><span style="color:#999999">[</span> -r <span style="color:#ee9900">$pgrootpath</span>/webconfig <span style="color:#999999">]</span><span style="color:#999999">]</span> <span style="color:#9a6e3a">&&</span> <span style="color:#999999">[</span>  <span style="color:#ee9900"><span style="color:#ee9900">`</span><span style="color:#dd4a68">grep</span> ^packagename<span style="color:#9a6e3a">=</span> $pgrootpath/webconfig<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">wc</span> -l<span style="color:#ee9900">`</span></span><span style="color:#9a6e3a">==</span>1 <span style="color:#999999">]</span><span style="color:#999999">;</span><span style="color:#0077aa">then</span>
                            packagename<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#dd4a68">grep</span> ^packagename<span style="color:#9a6e3a">=</span> $pgrootpath/webconfig 2<span style="color:#9a6e3a">></span>/dev/null<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> -F<span style="color:#669900">'"'</span> <span style="color:#669900">'{print <span style="color:#ee9900">$2</span>}'</span><span style="color:#ee9900">)</span></span>
            <span style="color:#0077aa">else</span>
                    packagename<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#dd4a68">basename</span> $pgrootpath<span style="color:#ee9900">)</span></span>-1.0.0-bin.tar.gz
            <span style="color:#0077aa">fi</span>
    
            <span style="color:#0077aa">echo</span> <span style="color:#669900">"<span style="color:#ee9900">$packagename</span> <span style="color:#ee9900">$pgrootpath</span>"</span> <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$packagePath</span>
    <span style="color:#0077aa">done</span></code></span></span>

該指令碼的目的是先通過使用ps -fC java命令獲取該機器上面除了flume程序外的所有其他java程序(我這邊使用的是flume來收集業務日誌的。)

然後,通過獲取到的PID使用ll /proc/pid/cwd命令獲取該程序的程式根目錄,後面那些判斷是獲取該程序對應的包名(這一步各位可以根據自己公司的情況自行修改,我這邊取包名的方式並不能夠匹配各位公司的設定,在下愛莫能助了。)

最後是將獲取到的程式根目錄和包名存放在變數packagePath對應的檔案中。

<span style="color:#333333"><span style="color:black"><code class="language-bash">    <span style="color:slategray">#cat get_jvmlist.sh </span>
    <span style="color:slategray">#!/bin/bash</span>
    
    TABLESPACE<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">`</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$1</span>}'</span> /usr/local/etc/scripts/package_path.txt<span style="color:#ee9900">`</span></span>
    COUNT<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">`</span><span style="color:#0077aa">echo</span> <span style="color:#669900">"<span style="color:#ee9900">$TABLESPACE</span>"</span> <span style="color:#9a6e3a">|</span><span style="color:#dd4a68">wc</span> -l<span style="color:#ee9900">`</span></span>
    INDEX<span style="color:#9a6e3a">=</span>0
    <span style="color:#0077aa">echo</span> <span style="color:#669900">'{"data":['</span>
    <span style="color:#0077aa">echo</span> <span style="color:#669900">"<span style="color:#ee9900">$TABLESPACE</span>"</span> <span style="color:#9a6e3a">|</span> <span style="color:#0077aa">while</span> <span style="color:#dd4a68">read</span> LINE<span style="color:#999999">;</span> <span style="color:#0077aa">do</span>
        <span style="color:#0077aa">echo</span> -n <span style="color:#669900">'{"{#TABLENAME}":"'</span><span style="color:#ee9900">$LINE</span><span style="color:#669900">'"}'</span>
        INDEX<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">`</span><span style="color:#dd4a68">expr</span> $INDEX + 1<span style="color:#ee9900">`</span></span>
        <span style="color:#0077aa">if</span> <span style="color:#999999">[</span> <span style="color:#ee9900">$INDEX</span> -lt <span style="color:#ee9900">$COUNT</span> <span style="color:#999999">]</span><span style="color:#999999">;</span> <span style="color:#0077aa">then</span>
            <span style="color:#0077aa">echo</span> <span style="color:#669900">','</span>
        <span style="color:#0077aa">fi</span>
    <span style="color:#0077aa">done</span>
    <span style="color:#0077aa">echo</span> <span style="color:#669900">']}'</span></code></span></span>

這個指令碼的作用就是通過讀取檔案裡面的包名,然後將包名進行json序列化輸出,沒什麼好講的,套路套一個迴圈指令碼就行。

接下來就是重要的指令碼了,呼叫jstat獲取JVM狀態,並快取到檔案中。

<span style="color:#333333"><span style="color:black"><code class="language-bash">    <span style="color:slategray">#cat get_jvmstatus.sh </span>
    <span style="color:slategray">#!/bin/bash</span>
    
    MAINCLASS<span style="color:#9a6e3a">=</span><span style="color:#669900">"*Main.class"</span>
    scriptPath<span style="color:#9a6e3a">=</span>/usr/local/etc/scripts
    
    <span style="color:#dd4a68">cat</span> <span style="color:#ee9900">$scriptPath</span>/package_path.txt<span style="color:#9a6e3a">|</span><span style="color:#0077aa">while</span> <span style="color:#dd4a68">read</span> line
    <span style="color:#0077aa">do</span>
    packageName<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#0077aa">echo</span> $line<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$1</span>}'</span><span style="color:#ee9900">)</span></span>
    pgRootPath<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#0077aa">echo</span> $line<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$2</span>}'</span><span style="color:#ee9900">)</span></span>
    <span style="color:#0077aa">if</span> <span style="color:#999999">[</span><span style="color:#999999">[</span> -d <span style="color:#ee9900">$pgRootPath</span>/tomcat <span style="color:#999999">]</span><span style="color:#999999">]</span><span style="color:#999999">;</span><span style="color:#0077aa">then</span>
    pid<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#dd4a68">cat</span> $pgRootPath/tomcat/tomcat.pid<span style="color:#ee9900">)</span></span>
    <span style="color:#0077aa">else</span>
    mainPath<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#dd4a68">find</span> $pgRootPath -name $MAINCLASS<span style="color:#ee9900">)</span></span>
    appName<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#0077aa">echo</span> $<span style="color:#999999">{</span>mainPath##*classes/<span style="color:#999999">}</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">sed</span> <span style="color:#669900">'s#/#.#g'</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">sed</span> <span style="color:#669900">'s#.class##g'</span><span style="color:#ee9900">)</span></span>
    pid<span style="color:#9a6e3a">=</span><span style="color:#ee9900"><span style="color:#ee9900">$(</span><span style="color:#dd4a68">ps</span> -fC java<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">grep</span> <span style="color:#669900">"<span style="color:#ee9900">$appName</span>"</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$2</span>}'</span><span style="color:#ee9900">)</span></span>
    <span style="color:#0077aa">fi</span>
    javaHome<span style="color:#9a6e3a">=</span>/usr/local/java/jdk1.8.0
    <span style="color:slategray">#javaHome=/usr/local/java/latest</span>
    <span style="color:slategray">#if [[ -r $pgRootPath/appconfig ]] && [  `grep ^JAVA_HOME= $pgRootPath/appconfig|wc -l` == 1 ] && [ `grep ^JAVA_HOME= $pgRootPath/appconfig|grep 8|wc -l` == 1 ];then</span>
                            <span style="color:slategray">#javaHome=$(grep ^JAVA_HOME= $pgRootPath/appconfig 2>/dev/null|awk -F'=' '{print $2}')</span>
    <span style="color:slategray">#javaHome=/usr/local/java/jdk1.8.0</span>
            <span style="color:slategray">#else</span>
            <span style="color:slategray">#        if [[ -r $pgRootPath/webconfig ]] && [ `grep ^'export JAVA_HOME=' $pgRootPath/webconfig|wc -l` == 1 ] && [ `grep ^'export JAVA_HOME=' $pgRootPath/webconfig|grep 8|wc -l` == 1 ];then</span>
            <span style="color:slategray">#                #javaHome=$(grep ^'export JAVA_HOME=' $pgRootPath/webconfig 2>/dev/null|awk -F'"' '{print $2}')</span>
            <span style="color:slategray">#        javaHome=/usr/local/java/jdk1.8.0</span>
    <span style="color:slategray">#fi</span>
    <span style="color:slategray">#fi</span>
    <span style="color:slategray">#echo --------------------------------$pgRootPath</span>
    <span style="color:slategray">#echo $javaHome</span>
    <span style="color:#0077aa">echo</span> -------------------------------<span style="color:#ee9900">$pid</span>
    <span style="color:#dd4a68">sleep</span> 5
    <span style="color:slategray">#echo -n >$scriptPath/package/$packageName</span>
    <span style="color:slategray">#$javaHome/bin/jstat -gccapacity $pid > ./package/$packageName 2>/dev/null</span>
    <span style="color:slategray">#$javaHome/bin/jmap -heap $pid>>./package/$packageName 2>/dev/null</span>
    <span style="color:#0077aa">echo</span> gcnew <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#ee9900">$javaHome</span>/bin/jstat -gcnew <span style="color:#ee9900">$pid</span> <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#0077aa">echo</span> gcutil <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#ee9900">$javaHome</span>/bin/jstat -gcutil <span style="color:#ee9900">$pid</span> <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#0077aa">echo</span> gcnewcapacity <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#ee9900">$javaHome</span>/bin/jstat -gcnewcapacity <span style="color:#ee9900">$pid</span> <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
            <span style="color:#0077aa">echo</span> gccapacity <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#ee9900">$javaHome</span>/bin/jstat -gccapacity <span style="color:#ee9900">$pid</span> <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
            <span style="color:slategray">#echo gcold >> $scriptPath/package/$packageName 2>/dev/null</span>
    <span style="color:slategray">#$javaHome/bin/jstat -gcold $pid >> $scriptPath/package/$packageName 2>/dev/null</span>
            <span style="color:#0077aa">echo</span> gc <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#ee9900">$javaHome</span>/bin/jstat -gc <span style="color:#ee9900">$pid</span> <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
            <span style="color:#0077aa">echo</span> class <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#ee9900">$javaHome</span>/bin/jstat -class <span style="color:#ee9900">$pid</span> <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#0077aa">echo</span> cpu <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#0077aa">echo</span> -e <span style="color:#669900">"CPU\n<span style="color:#ee9900"><span style="color:#ee9900">$(</span> <span style="color:#dd4a68">ps</span> aux<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">grep</span> $pid<span style="color:#9a6e3a">|</span><span style="color:#dd4a68">grep</span> -v <span style="color:#dd4a68">grep</span><span style="color:#9a6e3a">|</span><span style="color:#dd4a68">awk</span> <span style="color:#669900">'{print <span style="color:#ee9900">$3</span>}'</span><span style="color:#ee9900">)</span></span>"</span> <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#0077aa">echo</span> mem <span style="color:#9a6e3a">>></span> <span style="color:#ee9900">$scriptPath</span>/package/<span style="color:#ee9900">$packageName</span> 2<span style="color:#9a6e3a">></span>/dev/null
    <span style="color:#0077aa">echo</span> -e <span style="color:#669900">"MEM\n<span style="color:#ee9900"><span style="c