使用PostgreSQL分析作業系統命令及檔案
摘要：
版權宣告：本文為Buddy Yuan原創文章，未經允許不得轉載。原文地址：使用PostgreSQL分析作業系統命令及檔案
通過在Linux安裝了sysstat，可以實現自動採集效能資料...
版權宣告：本文為Buddy Yuan原創文章，未經允許不得轉載。原文地址：ofollow,noindex" target="_blank">使用PostgreSQL分析作業系統命令及檔案
通過在Linux安裝了sysstat，可以實現自動採集效能資料存放在/var/log/sa中。要開啟自動採集，需要開啟syssata服務,在centos 7當中使用下列命令：
systemctl enable sysstat systemctl start sysstat
當然自動採集是10分鐘一次，為了實驗方便，我修改了配置檔案調整成1分鐘1次。
[root@db sa]# more /etc/cron.d/sysstat # Run system activity accounting tool every 10 minutes */1 * * * * root /usr/lib64/sa/sa1 1 1 # 0 * * * * root /usr/lib64/sa/sa1 600 6 & # Generate a daily summary of process accounting at 23:53 53 23 * * * root /usr/lib64/sa/sa2 -A
接下來我們執行sar -f命令檢視自動生成的檔案.
[root@db sa]# sar -f sa22 | head -20 Linux 3.10.0-693.21.1.el7.x86_64 (db.gowithme.tw) 2018年10月22日 _x86_64_(1 CPU) 21時48分59秒LINUX RESTART 21時59分54秒LINUX RESTART 22時00分01秒CPU%user%nice%system%iowait%steal%idle 22時01分01秒all0.020.000.200.030.0099.75 22時02分01秒all0.020.000.120.020.0099.85 22時03分01秒all0.000.000.070.020.0099.92 22時04分01秒all0.000.000.070.000.0099.93 22時05分01秒all0.020.000.080.000.0099.90 22時06分01秒all0.000.000.100.020.0099.88 22時07分02秒all0.020.000.070.000.0099.92 22時08分01秒all0.000.000.070.020.0099.91 22時09分01秒all0.020.000.070.020.0099.90 22時10分01秒all0.000.000.070.000.0099.93 22時11分01秒all0.020.000.070.000.0099.92 22時12分01秒all0.000.000.070.020.0099.92 22時13分01秒all0.000.000.080.000.0099.92
接下來我們就可以使用postgresql將OS命令的結果儲存到變數中.
-bash-4.2$ psql psql (10.4) Type "help" for help. 然後查詢變數，就可以檢視到os的資料. postgres=# \set varls `ls -la` postgres=# select :'varls'; ?column? ---------------------------------------------------------------------- total 20+ dr-xr-xr-x.18 rootroot239 Oct 22 20:33 .+ dr-xr-xr-x.18 rootroot239 Oct 22 20:33 ..+ lrwxrwxrwx.1 rootroot7 Mar 312018 bin -> usr/bin+ dr-xr-xr-x.5 rootroot4096 Mar 312018 boot+ drwxr-xr-x.18 rootroot3040 Oct 22 21:33 dev+ drwxr-xr-x.76 rootroot8192 Oct 22 21:45 etc+ drwxr-xr-x.3 rootroot21 Mar 312018 home+ lrwxrwxrwx.1 rootroot7 Mar 312018 lib -> usr/lib+ lrwxrwxrwx.1 rootroot9 Mar 312018 lib64 -> usr/lib64+ drwxr-xr-x.2 rootroot6 Nov52016 media+ drwxr-xr-x.2 rootroot6 Nov52016 mnt+ drwxr-xr-x.3 rootroot39 Mar 312018 opt+ dr-xr-xr-x. 113 rootroot0 Oct 22 21:33 proc+ dr-xr-x---.3 rootroot163 Mar 312018 root+ drwxr-xr-x.23 rootroot700 Oct 22 21:45 run+ lrwxrwxrwx.1 rootroot8 Mar 312018 sbin -> usr/sbin+ drwxr-xr-x.2 rootroot6 Nov52016 srv+ dr-xr-xr-x.13 rootroot0 Oct 22 21:33 sys+ drwxrwxrwt.7 rootroot140 Oct 22 21:45 tmp+ drwxr-xr-x.16 rootroot199 Jun 28 10:50 usr+ drwxr-xr-x.1 vagrant vagrant 4096 Oct 22 21:23 vagrant+ drwxr-xr-x.19 rootroot267 Mar 312018 var (1 row)
接下來我們在把sar的結果放在var_iostat變數裡面.
postgres=# \set var_iostat `sar -f /var/log/sa/sa22` postgres=# select :'var_iostat'; ?column? ------------------------------------------------------------------------------------------------- Linux 3.10.0-693.21.1.el7.x86_64 (db.gowithme.tw)10/22/2018_x86_64_(1 CPU)+ + 09:48:59 PMLINUX RESTART+ + 09:59:54 PMLINUX RESTART+ + 10:00:01 PMCPU%user%nice%system%iowait%steal%idle+ 10:01:01 PMall0.020.000.200.030.0099.75+ 10:02:01 PMall0.020.000.120.020.0099.85+ 10:03:01 PMall0.000.000.070.020.0099.92+ 10:04:01 PMall0.000.000.070.000.0099.93+ 10:05:01 PMall0.020.000.080.000.0099.90+ 10:06:01 PMall0.000.000.100.020.0099.88+ 10:07:02 PMall0.020.000.070.000.0099.92+ 10:08:01 PMall0.000.000.070.020.0099.91+ 10:09:01 PMall0.020.000.070.020.0099.90+ 10:10:01 PMall0.000.000.070.000.0099.93+ 10:11:01 PMall0.020.000.070.000.0099.92+ 10:12:01 PMall0.000.000.070.020.0099.92+
由於我們只對iowait這一列感興趣，我們可以使用awk進行過濾。
postgres=# \set var_iostat `sar -f /var/log/sa/sa22 | egrep -v "^$|Average|Linux|LINUX|CPU" | awk -F " " '{print $7}'`
postgres=# select :'var_iostat';
?column?
----------
0.03+
0.02+
0.02+
0.00+
0.00+
0.02+
0.00+
0.02+
0.02+
0.00+
0.00+
0.02+
0.00+
0.02+
如果我們把這個插入到表中，結果會是一個varchar列，而不是多列。這對於我們分析資料不是很好，接下來我們需要做一下行列轉換，先把這麼多行轉換成一行資料，並用,分割開。
postgres=# \set var_iostat `sar -f /var/log/sa/sa22 | egrep -v "^$|Average|Linux|LINUX|CPU" | awk -F " " '{print $7}' | sed ':a;N;$!ba;s/\n/,/g'`
postgres=# select :'var_iostat';
?column?
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------
0.03,0.02,0.02,0.00,0.00,0.02,0.00,0.02,0.02,0.00,0.00,0.02,0.00,0.02,0.00,0.00,0.00,0.00,0.02,0.00,0.00,0.02,0.00,0.02,0.02,0.00,0.02,0.00,0.02,0.02,0.00,0.00,0.02,0.00,0.00,0.00,0.00,0.00,0.02,0.00,0.00,0.00,0.02,0.00,0.00,0.02,0.00,0.02,0.02,0.00,0.0
0,0.00,0.02,0.00,0.02,0.00,0.07,0.00,0.02,0.00,0.00,0.02,0.00,0.02
(1 row)
現在我們把上述這個變數用逗號分割，然後轉換成陣列。
postgres=# select string_to_array(:'var_iostat',',')::numeric[];
string_to_array
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------
{0.03,0.02,0.02,0.00,0.00,0.02,0.00,0.02,0.02,0.00,0.00,0.02,0.00,0.02,0.00,0.00,0.00,0.00,0.02,0.00,0.00,0.02,0.00,0.02,0.02,0.00,0.02,0.00,0.02,0.02,0.00,0.00,0.02,0.00,0.00,0.00,0.00,0.00,0.02,0.00,0.00,0.00,0.02,0.00,0.00,0.02,0.00,0.02,0.02,0.00,0.
00,0.00,0.02,0.00,0.02,0.00,0.07,0.00,0.02,0.00,0.00,0.02,0.00,0.02}
(1 row)
接下來儲存到表裡面，我們就可以查詢出當前使用率最大值和最小值做一些統計.
postgres=# create table tab_iowaits (varday date, iowaits numeric[]); postgres=# insert into tab_iowaits values (current_date, string_to_array(:'var_iostat',',')::numeric[]); postgres=# with tab as ( select unnest(iowaits) vals, varday from tab_iowaits where varday = current_date ) select max(vals) from tab; max ------ 0.07 (1 row) postgres=# with tab as ( select unnest(iowaits) vals, varday from tab_iowaits where varday = current_date ) select min(vals) from tab; min ------ 0.00 (1 row)