shell （awk）數組使用案例
1.取ifconfig eth0的IP地址
[liu@weblogic ~]$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:0C:29:68:47:71
inet addr:192.168.48.144 Bcast:192.168.48.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe68:4771/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:48161 errors:0 dropped:0 overruns:0 frame:0
TX packets:25022 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:36048837 (34.3 MiB) TX bytes:2202106 (2.1 MiB)
[liu@weblogic ~]$ ifconfig eth0|awk -F ‘[ :]+‘ ‘NR==2{print NR,$4}‘
2 192.168.48.144
2.統計apache日誌單IP訪問請求數排名
[liu@weblogic ~]$ cat 2.txt
10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
10.0.0.5 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
10.0.0.6 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
[liu@weblogic ~]$ awk ‘{print $1}‘ 2.txt |sort|uniq -c
6 10.0.0.3
1 10.0.0.5
1 10.0.0.6
[liu@weblogic ~]$ awk ‘{array[$1]++}END{for(k in array){print k,array[k];}}‘ 2.txt
10.0.0.3 6
10.0.0.5 1
10.0.0.6 1
3.統計域名訪問量
[liu@weblogic ~]$ cat 1.txt
http://www.baidu.com/index.html
http://www.163.com/1.html
http://www.cnblogs.com/index.html
http://www.baidu.com/2.html
http://www.163.com/index.html
http://www.qq.com/index.html
http://www.baidu.com/3.html
http://www.163.com/2.html
http://www.baidu.com/2.html
[liu@weblogic ~]$ awk ‘{split($0,array,"/+");key=array[2];count[key]++}END{for(kk in count) {print kk,count[kk]}}‘ 1.txt
www.qq.com 1
www.cnblogs.com 1
www.baidu.com 4
www.163.com 3
[liu@weblogic ~]$ awk -F "[ /]+" ‘{array[$2]++}END{for(i in array){print i,array[i]}}‘ 1.txt
www.qq.com 1
www.cnblogs.com 1
www.baidu.com 4
www.163.com 3
4.計算每個人的總工資和平均工資
[liu@weblogic ~]$ awk ‘{array[$2]+=$3;count[$2]++}END{for(key in count){print key,array[key]"k",array[key]/count[key]}}‘ 3.txt
kuqi 20k 10
jeacen 14k 7
yingsui 27k 13.5
xiaofen 16k 8
wodi 23k 11.5
yideng 21k 10.5
5.對本地IP和遠程IP去重並統計重復數
[liu@weblogic ~]$ cat 4.txt
Proto Recv-Q Send-Q Local Addree Foreign Addree State
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 115.29.49.213:80 117.136.27.254:13779 SYN_RECV
tcp 0 0 115.29.49.213:80 113.97.117.157:1847 SYN_RECV
tcp 0 0 115.29.49.213:80 117.136.40.20:19594 SYN_RECV
tcp 0 0 115.29.49.213:80 117.136.40.20:19595 SYN_RECV
tcp 0 0 115.29.49.213:80 121.236.219.69:45363 SYN_RECV
tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
unix 3 [] SYREAM CONNECTED 11183664 /TMP/MYSQL.SOCK
unix 3 [] SYREAM CONNECTED 11183646 /TMP/MYSQL.SOCK
unix 3 [] SYREAM CONNECTED 11183665 /TMP/MYSQL.SOCK
unix 3 [] SYREAM CONNECTED 11183668 /TMP/MYSQL.SOCK
unix 3 [] SYREAM CONNECTED 11183654 /TMP/MYSQL.SOCK
unix 3 [] SYREAM CONNECTED 11183655 /TMP/MYSQL.SOCK
unix 3 [] SYREAM CONNECTED 11183668 /TMP/MYSQL.SOCK
unix 3 [] SYREAM CONNECTED 11183676 /TMP/MYSQL.SOCK
unix 3 [] SYREAM CONNECTED 11183672 /TMP/MYSQL.SOCK
[liu@weblogic ~]$ awk -F ‘[ :]+‘ ‘/^tcp/{array[$4" "$6]++}END{for(key in array){print key,array[key]}}‘ 4.txt
115.29.49.213 117.136.40.20 2
0.0.0.0 0.0.0.0 3
115.29.49.213 113.97.117.157 1
115.29.49.213 121.236.219.69 1
115.29.49.213 117.136.27.254 1
6.統計源IP端口及目的IP同時去重
[liu@weblogic ~]$ cat 5.txt
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.43:54963->203.81.19.92:80 on ppp6) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (UDP 10.0.0.19:1441->121.14.96.233:80 on ppp0) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (UDP 172.16.1.103:57318->211.147.6.3:80 on ppp2) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 172.16.1.203:4372->61.135.163.86:80 on ixp7) [0,0]
Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
[liu@weblogic ~]$ vim 5.sh
{
split($9,array,":|->")
sip=array[1]
sport=array[2]
mip=array[3]
if (!((sip,sport,mip) in tee)) {
tee[sip,sport,mip] = 1
}
}
END{
for (key in tee)
print key
}
[liu@weblogic ~]$ awk -f 5.sh 5.txt
172.16.1.103 57318 211.147.6.3
10.0.0.19 1441 121.14.96.233
172.16.1.203 4372 61.135.163.86
10.0.0.43 54963 203.81.19.92
10.0.0.131 1227 210.192.121.172
優化效果如下
在5.sh開頭添加BEGIN { printf("%-16s %-6s %-16s\n","SIP","SPORT","MIP")}
[liu@weblogic ~]$ awk -f 5.sh 5.txt
SIP SPORT MIP
172.16.1.10357318211.147.6.3
10.0.0.191441121.14.96.233
172.16.1.203437261.135.163.86
10.0.0.4354963203.81.19.92
10.0.0.1311227210.192.121.172
shell （awk）數組使用案例