02-shell文字處理三劍客之sed
阿新 • • 發佈：2018-12-05
sed 是流編輯器，但是它不會修改原始檔。
sed (流文字編輯器)
用法：sed OPTIONS… [SCRIPT] [INPUTFILE…]
-r: 使用擴充套件的正則表示式;
-n, –quiet, –silent: 不輸出模式空間的內容；
-i：直接編輯原檔案；
-e: -e ” -e ” -e ”
‘CMD1;CMD2;CMD3’
-f /path/to/sed_script：從指定的檔案讀取處理指令碼
1 模式空間是什麼？
模式空間其實就是記憶體中一個空間，歸sed所有s。ed讀取流文字檔案的時候，每次讀取一行，然後載入記憶體中，這個記憶體就成為模式空間。那麼我們編輯的內容是屬於模式空間中的內容，而不是原始檔內容。
[[email protected] shelltest]# sed '1,10d' /etc/fstab
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 sed ‘1,10d’ /etc/fstab 這一句語句的執行邏輯是，sed把內容讀到模式空間中，判斷是不是1——10的內容，是的話執行刪除命令d,所以刪除了1——10行內容，11行以後的內容就顯示出來了。
sed的工作流程可以總結為：
第一步：提取一行內容到模式空間
第二步：針對模式空間中的內容執行編輯命令。
第三步：將模式空間中的內容進行輸出。
第四步：清空模式空間。
2 地址定界：
#（行號）: 指定行
$: 最後一行
/regexp/：任何能夠被regexp匹配到的行
\%regexp%: 任何能夠被regexp匹配到的行；換用%為邊界符號；
/regexp/I
\%regexp%I：匹配時忽略字元大小寫；
addr1, addr2: 指定範圍內的所有行：
1·,/regexp/
#（行號）,#（行號）
/regexp/,/regexp/
#(行號),+#
first~step：指定起始位置，以及步長 1~3
[[email protected] shelltest]# sed '$d' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
[[email protected] shelltest]# 以上功能是刪除了最後一行。
[[email protected] shelltest]# sed '/dev/d' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[[email protected] shelltest]# sed '\%/dev/mapper%d' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[[email protected] shelltest]# sed '1+2d' /etc/fstab
sed: -e expression #1, char 2: unknown command: `+'
[[email protected] shelltest]# sed '1,+2d' /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[[email protected] shelltest]# sed '1~2d' /etc/fstab
#
# Created by anaconda on Tue Jan 26 09:32:28 2016
# Accessible filesystems, by reference, are maintained under '/dev/disk'
#
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
sysfs /sys sysfs defaults 0 0
[[email protected] shelltest]#基本正則表示式元字元：
字元：. [] [^]
次數：*, \+, \?, \{i,j\}
分組：\(\)
錨定：^, $, \<, \>
或者：|
後向引用：\1, \2, ...
編輯命令：
注意：命令之前可以使用!表示取反操作；
=: 顯示行號
# sed -n '\%/dev/mapper%=' /etc/fstab
d: 刪除匹配到的行；
p: 列印模式空間中的內容；
通常與-n選項一同使用；
a \text: append，
a \text1\ntext2\ntext3: 附加多行文字
i \text：insert,
i \text1\ntext2: 插入多行文字
c \text: change，把匹配到的行替換為給定的文字
s///: 替換命令
g: 全域性替換
i: 不區分大小寫
[root@hadoop1 shelltest]# sed -n '\%/dev/%=' /etc/fstab
6
12
13
[root@hadoop1 shelltest]# sed -n '5,7p' /etc/fstab
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
[root@hadoop1 shelltest]# 追加：
[[email protected] shelltest]# sed '1a \xielaoshi' /etc/fstab
xielaoshi
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[[email protected] shelltest]# 還可以同時執行多條命令：
[[email protected] shelltest]# sed '1d;2p;3=' /etc/fstab
#
#
3
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[[email protected] shelltest]# 替換：
[[email protected] shelltest]# sed '3c \bbbb' /etc/fstab
#
bbbb
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[[email protected] shelltest]#[[email protected] shelltest]# sed 's/dev/xielaoshi/' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/xielaoshi/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /xielaoshi/shm tmpfs defaults 0 0
xielaoshipts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[[email protected] shelltest]#全域性替換：
[[email protected] shelltest]# sed 's/dev/xielaoshi/g' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/xielaoshi/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /xielaoshi/shm tmpfs defaults 0 0
xielaoshipts /xielaoshi/pts xielaoshipts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[[email protected] shelltest]#3 練習
練習1：刪除/boot/grub/grub.conf檔案中所以空白開頭的行行首的空白；
[[email protected] shelltest]# cat /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/sda2
# initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-431.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-431.el6.x86_64.img
[[email protected] shelltest]# sed 's/^[[:space:]]\+//g' /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/sda2
# initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-431.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-431.el6.x86_64.img
[[email protected] shelltest]# 練習2：刪除/etc/fstab檔案中所有以#開頭，後跟至少一個空白字元的行的行首的#和空白字元；
[root@hadoop1 shelltest]# sed 's/^#[[:space:]]\+//g' /etc/fstab
#
/etc/fstab
Created by anaconda on Tue Jan 26 09:32:28 2016
#
Accessible filesystems, by reference, are maintained under '/dev/disk'
See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# 練習3：echo一個檔案路徑給sed命令，取出其基名；取出其目錄名；
[root@hadoop1 shelltest]# echo "/etc/sysconfig/" | sed '[email protected][^/]\{1,\}/\[email protected]@g'
/etc/
[root@hadoop1 shelltest]# 練習4：替換/etc/inittab檔案“id:數字:initdefault:”一行中的那個數字為6；
[[email protected] shelltest]# cat /etc/inittab
# inittab is only used by upstart for the default runlevel.
#
# ADDING OTHER CONFIGURATION HERE WILL HAVE NO EFFECT ON YOUR SYSTEM.
#
# System initialization is started by /etc/init/rcS.conf
#
# Individual runlevels are started by /etc/init/rc.conf
#
# Ctrl-Alt-Delete is handled by /etc/init/control-alt-delete.conf
#
# Terminal gettys are handled by /etc/init/tty.conf and /etc/init/serial.conf,
# with configuration in /etc/sysconfig/init.
#
# For information on how to write upstart event handlers, or how
# upstart works, see init(5), init(8), and initctl(8).
#
# Default runlevel. The runlevels used are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:5:initdefault:
[[email protected] shelltest]# sed '[email protected]:[0-9]:initdefault:@id:6:[email protected]' /etc/inittab
# inittab is only used by upstart for the default runlevel.
#
# ADDING OTHER CONFIGURATION HERE WILL HAVE NO EFFECT ON YOUR SYSTEM.
#
# System initialization is started by /etc/init/rcS.conf
#
# Individual runlevels are started by /etc/init/rc.conf
#
# Ctrl-Alt-Delete is handled by /etc/init/control-alt-delete.conf
#
# Terminal gettys are handled by /etc/init/tty.conf and /etc/init/serial.conf,
# with configuration in /etc/sysconfig/init.
#
# For information on how to write upstart event handlers, or how
# upstart works, see init(5), init(8), and initctl(8).
#
# Default runlevel. The runlevels used are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:6:initdefault
[[email protected] shelltest]# 4 高階編輯命令：
h: 用模式空間中的內容覆蓋保持空間中的內容；
H：把模式空間中的內容追求至保持空間；
g: get, 從保持空間取回資料至模式空間，覆蓋模式空間中原有的內容；
G：GET, 從保持空間取回資料至模式空間，追加於模式空間中現有的內容後面；
x: 把模式空間和保持空間中的內容進行交換；
n: 讀取匹配到的行下一行至模式空間；
N: 追加讀匹配到的行下一行至模式空間；
d: 刪除模式空間中的內容；
D：刪除多行模式空間中首行；
sed ‘1!G;h;$!d’ FILE
[root@hadoop1 shelltest]# sed '1!G;h;$!d' /etc/issue
Kernel \r on an \m
CentOS release 6.5 (Final)
[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]# 顯示檔案的偶數行：sed -n 'n;p' FILE; sed -n '2~2p' FILE
1
2
3
4
顯示檔案的奇數行：sed -n 'p;n' FILE; sed -n '1~2;p' FILE
練習：用sed將檔案的奇數行與偶數行合併為一行，中間用“&&”隔開；
[root@hadoop1 shelltest]# sed '$!N;[email protected]\[email protected]\&\&@g' /etc/issue
CentOS release 6.5 (Final)&&Kernel \r on an \m
[root@hadoop1 shelltest]# 5 課後練習：說明以下sed命令的意義：
(1) sed ‘
[root@hadoop1 shelltest]# sed '$!N;$!D' /etc/issue
Kernel \r on an \m
[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]#(2) sed ‘$!d’ /etc/issue
刪除非尾行。
(3) sed ‘G’ /etc/issue
[root@hadoop1 shelltest]# sed 'G' /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]# 因為保持空間裡面都是空的，所以追加到模式空間後面全是空行。
(4) sed ‘/^$/d;G’ /etc/issue
[root@hadoop1 shelltest]# sed '/^$/d;G' /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]# (5) sed ‘n;d’ /etc/issue
[root@hadoop1 shelltest]# sed 'n;d' /etc/issue
CentOS release 6.5 (Final)
[root@hadoop1 shelltest]# (6) sed -n ‘1!G;h;$p’ /etc/issue
[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]# 6 參考資料
模式空間和保持空間的理解：
什麼是模式空間和保持空間
O(∩∩)O嗯!O(∩∩)O嗯!O(∩_∩)O嗯!今天就寫到這裡，休息一下。如果你看到此文，想進一步學習或者和我溝通，加我微信公眾號：名字：五十年後 。
