iscsi共享儲存的簡單配置和應用
摘要：
1、環境介紹
SCSI（Small Computer System Interface）是塊資料傳輸協議，在儲存行業廣泛應用，是儲存裝置最基本的標準協議。從根本上說，iSCSI協議是一種利用IP網路來傳輸潛伏時間短的SCSI資料塊的方法，ISCSI使用 乙太網 協議傳送SCSI命令、響應...
1、環境介紹
SCSI（Small Computer System Interface）是塊資料傳輸協議，在儲存行業廣泛應用，是儲存裝置最基本的標準協議。從根本上說，iSCSI協議是一種利用IP網路來傳輸潛伏時間短的SCSI資料塊的方法，ISCSI使用 ofollow,noindex">乙太網 協議傳送SCSI命令、響應和資料。ISCSI可以用我們已經熟悉和每天都在使用的乙太網來構建IP儲存區域網。通過這種方法，ISCSI克服了直接連線儲存的侷限性，使我們可以跨不同 伺服器 共享儲存資源，並可以在不停機狀態下擴充儲存容量。
所需軟體與軟體結構
CentOS 將 tgt 的軟體名稱定義為 scsi-target-utils ，因此你得要使用 yum 去安裝他才行。至於用來作為 initiator 的軟體則是使用 linux-iscsi 的專案，該專案所提供的軟體名稱則為 iscsi-initiator-utils 。所以，總的來說，你需要的軟體有：
scsi-target-utils：用來將 Linux 系統模擬成為 iSCSI target 的功能；
iscsi-initiator-utils：掛載來自 target 的磁碟到 Linux 本機上。
那麼 scsi-target-utils 主要提供哪些檔案呢？基本上有底下幾個比較重要需要注意的：
/etc/tgt/targets.conf：主要配置檔案，設定要分享的磁碟格式與哪幾顆；
/usr/sbin/tgt-admin：線上查詢、刪除 target 等功能的設定工具；
/usr/sbin/tgt-setup-lun：建立 target 以及設定分享的磁碟與可使用的
客戶端等工具軟體。
/usr/sbin/tgtadm：手動直接管理的管理員工具 (可使用配置檔案取代)；
/usr/sbin/tgtd：主要提供 iSCSI target 服務的主程式；
/usr/sbin/tgtimg：建置預計分享的映像檔案裝置的工具 (以映像檔案模擬磁碟)；
這次的實驗結構
image.png
（sdx1、sdy1是物理的磁碟通過lun連線到target虛擬的共享塊，在客戶端掛載這個塊。這裡我們新增一塊硬碟sdb，建立sdb1（400m）和sdb2（500m））
2、server端配置
新增一塊磁碟後
複查一下
[root@100 ~]# lsblk NAMEMAJ:MIN RMSIZE RO TYPE MOUNTPOINT sda8:0060G0 disk ├─sda18:10500M0 part /boot └─sda28:20 59.5G0 part ├─rhel-swap 253:002G0 lvm[SWAP] ├─rhel-root 253:10 38.6G0 lvm/ └─rhel-home 253:20 18.9G0 lvm/home sdb8:16010G0 disk ├─sdb18:170400M0 part └─sdb28:180500M0 part sr011:013.6G0 rom/dvd
安裝target並加入開機自啟動
yum install -y target*
systemctl start target
systemctl enable target
配置target的ctl將sdb1和sdb2共享出去
（一）、block關聯磁碟
[root@100 ~]# targetcli targetcli shell version 2.1.fb37 Copyright 2011-2013 by Datera, Inc and others. For help on commands, type 'help'. /> /backstores/block create block1 /dev/sdb /dev/sdb/dev/sdb1/dev/sdb2 ...................................dev /> /backstores/block create block1 /dev/sdb1 Created block storage object block1 using /dev/sdb1. /> /backstores/block create block2 /dev/sdb2 Created block storage object block2 using /dev/sdb2. /> ls o- / ..................................................................... [...] o- backstores .......................................................... [...] | o- block .............................................. [Storage Objects: 2] | | o- block1 .................. [/dev/sdb1 (400.0MiB) write-thru deactivated]#新建的block | | o- block2 .................. [/dev/sdb2 (500.0MiB) write-thru deactivated]#新建的block | o- fileio ............................................. [Storage Objects: 0] | o- pscsi .............................................. [Storage Objects: 0] | o- ramdisk ............................................ [Storage Objects: 0] o- iscsi ........................................................ [Targets: 0] o- loopback ..................................................... [Targets: 0] />
（二）、建立target
/> /iscsi create iqn.2018-10.20.com:disk Created target iqn.2018-10.20.com:disk. Created TPG 1. Global pref auto_add_default_portal=true Created default portal listening on all IPs (0.0.0.0), port 3260. /> ls o- / ..................................................................... [...] o- backstores .......................................................... [...] | o- block .............................................. [Storage Objects: 2] | | o- block1 .................. [/dev/sdb1 (400.0MiB) write-thru deactivated] | | o- block2 .................. [/dev/sdb2 (500.0MiB) write-thru deactivated] | o- fileio ............................................. [Storage Objects: 0] | o- pscsi .............................................. [Storage Objects: 0] | o- ramdisk ............................................ [Storage Objects: 0] o- iscsi ........................................................ [Targets: 1] #這裡是建立tartet的目錄 | o- iqn.2018-10.20.com:disk ....................................... [TPGs: 1] |o- tpg1 ........................................... [no-gen-acls, no-auth] |o- acls ...................................................... [ACLs: 0] |o- luns ...................................................... [LUNs: 0] |o- portals ................................................ [Portals: 1] |o- 0.0.0.0:3260 ................................................. [OK] o- loopback ..................................................... [Targets: 0]
（三）、建立lun關聯block和target
/> ls o- / ..................................................................... [...] o- backstores .......................................................... [...] | o- block .............................................. [Storage Objects: 2] | | o- block1 .................. [/dev/sdb1 (400.0MiB) write-thru deactivated] | | o- block2 .................. [/dev/sdb2 (500.0MiB) write-thru deactivated] | o- fileio ............................................. [Storage Objects: 0] | o- pscsi .............................................. [Storage Objects: 0] | o- ramdisk ............................................ [Storage Objects: 0] o- iscsi ........................................................ [Targets: 1] | o- iqn.2018-10.20.com:disk ....................................... [TPGs: 1] |o- tpg1 ........................................... [no-gen-acls, no-auth] |o- acls ...................................................... [ACLs: 0] |o- luns ...................................................... [LUNs: 0] |o- portals ................................................ [Portals: 1] |o- 0.0.0.0:3260 ................................................. [OK] o- loopback ..................................................... [Targets: 0] /> iscsi/iqn.2018-10.20.com:disk/tpg1/acls create iqn.2018-10.20.com:xx Created Node ACL for iqn.2018-10.20.com:xx /> ls o- / ..................................................................... [...] o- backstores .......................................................... [...] | o- block .............................................. [Storage Objects: 2] | | o- block1 .................. [/dev/sdb1 (400.0MiB) write-thru deactivated] | | o- block2 .................. [/dev/sdb2 (500.0MiB) write-thru deactivated] | o- fileio ............................................. [Storage Objects: 0] | o- pscsi .............................................. [Storage Objects: 0] | o- ramdisk ............................................ [Storage Objects: 0] o- iscsi ........................................................ [Targets: 1] | o- iqn.2018-10.20.com:disk ....................................... [TPGs: 1] |o- tpg1 ........................................... [no-gen-acls, no-auth] |o- acls ...................................................... [ACLs: 1] || o- iqn.2018-10.20.com:xx ............................ [Mapped LUNs: 0] |o- luns ...................................................... [LUNs: 0] |o- portals ................................................ [Portals: 1] |o- 0.0.0.0:3260 ................................................. [OK] o- loopback ..................................................... [Targets: 0] /> iscsi/iqn.2018-10.20.com:disk/tpg1/luns create / /backstores/block/block1/lib//sbin/ /backstores/block/block2/lib64//srv/ /bin//media//sys/ /boot//mnt//tmp/ /dev//opt//usr/ /dvd//proc//var/ /etc//root/ /home//run/ ..................................................storage_object /> iscsi/iqn.2018-10.20.com:disk/tpg1/luns create /backstores/block/block /backstores/block/block1/backstores/block/block2 ..................................................storage_object /> iscsi/iqn.2018-10.20.com:disk/tpg1/luns create /backstores/block/block1 Created LUN 0. Created LUN 0->0 mapping in node ACL iqn.2018-10.20.com:xx /> iscsi/iqn.2018-10.20.com:disk/tpg1/luns create /backstores/block/block2 Created LUN 1. Created LUN 1->1 mapping in node ACL iqn.2018-10.20.com:xx /> ls o- / ..................................................................... [...] o- backstores .......................................................... [...] | o- block .............................................. [Storage Objects: 2] | | o- block1 .................... [/dev/sdb1 (400.0MiB) write-thru activated] | | o- block2 .................... [/dev/sdb2 (500.0MiB) write-thru activated] | o- fileio ............................................. [Storage Objects: 0] | o- pscsi .............................................. [Storage Objects: 0] | o- ramdisk ............................................ [Storage Objects: 0] o- iscsi ........................................................ [Targets: 1] | o- iqn.2018-10.20.com:disk ....................................... [TPGs: 1] |o- tpg1 ........................................... [no-gen-acls, no-auth] |o- acls ...................................................... [ACLs: 1] #添加了一條認證 || o- iqn.2018-10.20.com:xx ............................ [Mapped LUNs: 2] ||o- mapped_lun0 ............................ [lun0 block/block1 (rw)] #關聯lun ||o- mapped_lun1 ............................ [lun1 block/block2 (rw)] #關聯lun |o- luns ...................................................... [LUNs: 2] || o- lun0 ................................... [block/block1 (/dev/sdb1)] || o- lun1 ................................... [block/block2 (/dev/sdb2)] |o- portals ................................................ [Portals: 1] |o- 0.0.0.0:3260 ................................................. [OK] o- loopback ..................................................... [Targets: 0] /> exit Global pref auto_save_on_exit=true Last 10 configs saved in /etc/target/backup. Configuration saved to /etc/target/saveconfig.json
至此伺服器端就配置完了，詳細的配置可以到/etc/target/saveconfig.json修改
3、client端配置
（一）、安裝iscsi*
軟體包 iscsi-initiator-utils-6.2.0.873-29.el7.x86_64
軟體包 iscsi-initiator-utils-iscsiuio-6.2.0.873-29.el7.x86_64
[root@200 ~]# yum install -y iscsi*
（二）、加入開機自啟
[root@200 ~]# systemctl restart iscsid.service
[root@200 ~]# systemctl enable iscsid.service
（三）、做一個發現的操作，發現伺服器共享的target名字是什麼
[root@200 ~]# iscsiadm -m discovery -t st -p 192.168.137.100 192.168.137.100:3260,1 iqn.2018-10.20.com:disk m模式mode-discovery, discoverydb. t型別typemustbesendtargets (or abbreviated as st), slp, isns or fw. Currently only sendtargets, fw, and iSNS issupported,see the DISCOVERY TYPES section. p是IP：port
（四）、檢視發現的條目
[root@200 ~]# iscsiadm -m discovery -p 192.168.137.100 # BEGIN RECORD 6.2.0.873-28 discovery.startup = manual discovery.type = sendtargets #-t st discovery.sendtargets.address = 192.168.137.100 #target服務端 discovery.sendtargets.port = 3260 #埠 discovery.sendtargets.auth.authmethod = None #認證沒有 discovery.sendtargets.auth.username = <empty> discovery.sendtargets.auth.password = <empty> discovery.sendtargets.auth.username_in = <empty> discovery.sendtargets.auth.password_in = <empty> discovery.sendtargets.timeo.login_timeout = 15 #登入超時時間 discovery.sendtargets.use_discoveryd = No #沒有使用這個發現 discovery.sendtargets.discoveryd_poll_inval = 30 discovery.sendtargets.reopen_max = 5 discovery.sendtargets.timeo.auth_timeout = 45 discovery.sendtargets.timeo.active_timeout = 30 discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768 # END RECORD
（五）、修改客戶端的acl並登入
[root@200 ~]# iscsiadm -m node -T iqn.2018-10.20.com:disk -p 192.168.137.100 -l Logging in to [iface: default, target: iqn.2018-10.20.com:disk, portal: 192.168.137.100,3260] (multiple) Login to [iface: default, target: iqn.2018-10.20.com:disk, portal: 192.168.137.100,3260] successful. [root@200 ~]# lsblk NAMEMAJ:MIN RMSIZE RO TYPE MOUNTPOINT sda8:0060G0 disk ├─sda18:10500M0 part /boot └─sda28:20 59.5G0 part ├─rhel-swap 253:002G0 lvm[SWAP] ├─rhel-root 253:10 38.6G0 lvm/ └─rhel-home 253:20 18.9G0 lvm/home sdb8:160400M0 disk sdc8:320500M0 disk sr011:013.6G0 rom/dvd [root@200 ~]# cat /etc/iscsi/initiatorname.iscsi #InitiatorName=iqn.1994-05.com.redhat:949f18f9ce84 InitiatorName=iqn.2018-10.20.com:xx ##剛才設定的acl條目
注：這個自動載入過來的磁碟重啟之後是自動載入過來的
4、客戶端掛載iscsi的盤
格式化sdb和sdc
[root@200 ~]# mkfs.xfs /dev/sdb meta-data=/dev/sdbisize=256agcount=8, agsize=12800 blks =sectsz=512attr=2, projid32bit=1 =crc=0finobt=0 data=bsize=4096blocks=102400, imaxpct=25 =sunit=0swidth=0 blks naming=version 2bsize=4096ascii-ci=0 ftype=0 log=internal logbsize=4096blocks=853, version=2 =sectsz=512sunit=0 blks, lazy-count=1 realtime =noneextsz=4096blocks=0, rtextents=0 [root@200 ~]# mkfs.xfs /dev/sdc meta-data=/dev/sdcisize=256agcount=8, agsize=16000 blks =sectsz=512attr=2, projid32bit=1 =crc=0finobt=0 data=bsize=4096blocks=128000, imaxpct=25 =sunit=0swidth=0 blks naming=version 2bsize=4096ascii-ci=0 ftype=0 log=internal logbsize=4096blocks=853, version=2 =sectsz=512sunit=0 blks, lazy-count=1 realtime =noneextsz=4096blocks=0, rtextents=0
建立掛載點並掛載
[root@200 ~]# mkdir /sdb
[root@200 ~]# mkdir /sdc
注：如果需要實時同步的話需要使用gfs叢集檔案系統
[root@200 ~]# cat /etc/fstab |tail -n 4 /dev/mapper/rhel-swapswapswapdefaults0 0 /dev/cdrom /dvd iso9660 defaults 0 0 /dev/sdb /sdb xfs _netdev 0 0 #新增的條目 /dev/sdc /sdc xfs _netdev 0 0 #新增的條目 [root@200 ~]# # mount -a [root@200 ~]# mount |tail -n 4 gvfsd-fuse on /run/user/0/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0) fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime) /dev/sdb on /sdb type xfs (rw,relatime,seclabel,attr2,inode64,noquota,_netdev) /dev/sdc on /sdc type xfs (rw,relatime,seclabel,attr2,inode64,noquota,_netdev) [root@200 ~]#
5、總結
（一）、增加iscsi儲存
(1)發現iscsi儲存:iscsiadm
-m discovery -t st -p ISCSI_IP
(2)檢視iscsi發現記錄:iscsiadm
-m node
(3)登入iscsi儲存:iscsiadm
-m node -T LUN_NAME -p ISCSI_IP -l
(4)開機自動:
iscsiadm -m node –T LUN_NAME -p ISCSI_IP --op
update -n node.startup -v
automatic
（二）、刪除iscsi儲存
(1)登出iscsi儲存
iscsiadm -m node -T LUN_NAME -p ISCSI_IP -u
(2)對出iscsi所有登入
iscsiadm -m node --logoutall=all
(3)刪除iscsi發現記錄:iscsiadm
-m node -o delete -T LUN_NAME -p ISCSI_IP