基於Centos7.4搭建Ceph
本文使用ceph-deploy工具，能快速搭建出一個ceph集群。
一、環境準備
修改主機名
[[email protected] ~]# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
| IP | 主機名 | 角色 |
10.10.10.20 | admin-node | ceph-deploy |
| 10.10.10.21 | node1 | mon |
| 10.10.10.22 | node2 | osd |
| 10.10.10.23 | node3 | osd |
設置DNS解析（我們這裏修改/etc/hosts文件）
每個節點都要配置
[[email protected] ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.10.10.20 admin-node
10.10.10.21 node1
10.10.10.22 node2
10.10.10.23 node3
配置yum源
每個節點都要配置
[[email protected] ~]# mv /etc/yum.repos.d{,.bak}
[[email protected] ~]# mkdir /etc/yum.repos.d
[[email protected] ~]# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
[[email protected] ceph]# cat /etc/yum.repos.d/ceph.repo
[Ceph]
name=Ceph
baseurl=http://mirrors.aliyun.com/ceph/rpm-jewel/el7/noarch/
enabled=1
gpgcheck=0
關閉防火墻和Selinux
每個節點都要配置
[[email protected] ~]# systemctl stop firewalld.service
[[email protected] ~]# systemctl disable firewalld.service
[[email protected] ~]# setenforce 0
[[email protected] ~]# sed -i ‘s/SELINUX=enforcing/SELINUX=disabled/‘ /etc/selinux/config
設置節點之間面秘鑰登入
每個節點都要配置
[[email protected] ~]# ssh-keygen
[[email protected] ~]# ssh-copy-id 10.10.10.21
[[email protected] ~]# ssh-copy-id 10.10.10.22
[[email protected] ~]# ssh-copy-id 10.10.10.23
使用chrony同步時間
每個節點都要配置
[[email protected] ~]# yum install chrony -y
[[email protected] ~]# systemctl restart chronyd
[[email protected] ~]# systemctl enable chronyd
[[email protected] ~]# chronyc source -v （查看時間是否同步，*表示同步完成）
二、安裝ceph-luminous
安裝ceph-deploy
只在admin-node節點安裝
[[email protected] ~]# yum install ceph-deploy -y
在管理節點上創建一個目錄，用於保存 ceph-deploy 生成的配置文件和密鑰對
只在admin-node節點安裝
[[email protected] ~]# mkdir /etc/ceph
[[email protected] ~]# cd /etc/ceph/
清除配置（若想從新安裝可以執行以下命令）
只在admin-node節點安裝
[[email protected] ceph]# ceph-deploy purgedata node1 node2 node3
[[email protected] ceph]# ceph-deploy forgetkeys
創建集群
只在admin-node節點安裝
[[email protected] ceph]# ceph-deploy new node1
修改ceph的配置，將副本數改為2
只在admin-node節點安裝
[[email protected] ceph]# vi ceph.conf
[global]
fsid = 183e441b-c8cd-40fa-9b1a-0387cb8e8735
mon_initial_members = node1
mon_host = 10.10.10.21
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true
osd pool default size = 2
安裝ceph
只在admin-node節點安裝
[[email protected] ceph]# ceph-deploy install admin-node node1 node2 node3
[node3][DEBUG ] Configure Yum priorities to include obsoletes
[node3][WARNIN] check_obsoletes has been enabled for Yum priorities plugin
[node3][INFO ] Running command: rpm --import https://download.ceph.com/keys/release.asc
[node3][INFO ] Running command: rpm -Uvh --replacepkgs https://download.ceph.com/rpm-jewel/el7/noarch/ceph-release-1-0.el7.noarch.rpm
[node3][DEBUG ] 獲取https://download.ceph.com/rpm-jewel/el7/noarch/ceph-release-1-0.el7.noarch.rpm
[node3][WARNIN] 警告：/etc/yum.repos.d/ceph.repo 已建立為 /etc/yum.repos.d/ceph.repo.rpmnew
[node3][DEBUG ] 準備中... ########################################
[node3][DEBUG ] 正在升級/安裝...
[node3][DEBUG ] ceph-release-1-1.el7 ########################################
[node3][WARNIN] ensuring that /etc/yum.repos.d/ceph.repo contains a high priority
[ceph_deploy][ERROR ] RuntimeError: NoSectionError: No section: ‘ceph-noarch‘
這個地方報錯了，安裝了一個高版本的ceph-release
解決方法：yum remove ceph-release
每個節點刪除ceph-release後再次重新執行上一次的命令
配置初始 monitor(s)、並收集所有密鑰
只在admin-node節點安裝
[[email protected] ceph]# ceph-deploy mon create-initial
[[email protected] ceph]# ls
ceph.bootstrap-mds.keyring ceph.bootstrap-rgw.keyring ceph-deploy-ceph.log
ceph.bootstrap-mgr.keyring ceph.client.admin.keyring ceph.mon.keyring
ceph.bootstrap-osd.keyring ceph.conf rbdmap
[[email protected] ceph]# ceph -s （查看集群狀態）
cluster 8d395c8f-6ac5-4bca-bbb9-2e0120159ed9
health HEALTH_ERR
no osds
monmap e1: 1 mons at {node1=10.10.10.21:6789/0}
election epoch 3, quorum 0 node1
osdmap e1: 0 osds: 0 up, 0 in
flags sortbitwise,require_jewel_osds
pgmap v2: 64 pgs, 1 pools, 0 bytes data, 0 objects
0 kB used, 0 kB / 0 kB avail
64 creating
創建OSD
[[email protected] ~]# lsblk （node2，node3做osd）
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
fd0 2:0 1 4K 0 disk
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 19G 0 part
├─cl-root 253:0 0 17G 0 lvm /
└─cl-swap 253:1 0 2G 0 lvm [SWAP]
sdb 8:16 0 50G 0 disk /var/local/osd0
sdc 8:32 0 5G 0 disk
sr0 11:0 1 4.1G 0 rom
[[email protected] ~]# mkfs.xfs /dev/sdb
[[email protected] ~]# mkdir /var/local/osd0
[[email protected] ~]# mount /dev/sdb /var/local/osd0
[[email protected] ~]# chown ceph:ceph /var/local/osd0
[[email protected] ~]# mkdir /var/local/osd1
[[email protected] ~]# mkfs.xfs /dev/sdb
[[email protected] ~]# mount /dev/sdb /var/local/osd1/
[[email protected] ~]# chown ceph:ceph /var/local/osd1
[[email protected] ceph]# ceph-deploy osd prepare node2:/var/local/osd0 node3:/var/local/osd1 （在admin-node節點執行）
將admin-node上的密鑰和配合文件拷貝到各個節點
只在admin-node節點安裝
[[email protected] ceph]# ceph-deploy admin admin-node node1 node2 node3
確保對 ceph.client.admin.keyring 有正確的操作權限
只在OSD節點執行
[[email protected] ~]# chmod +r /etc/ceph/ceph.client.admin.keyring
管理節點執行 ceph-deploy 來準備 OSD
[[email protected] ceph]# ceph-deploy osd prepare node2:/var/local/osd0 node3:/var/local/osd1
激活 OSD
[[email protected] ceph]# ceph-deploy osd activate node2:/var/local/osd0 node3:/var/local/osd1
檢查集群的健康狀況
[[email protected] ceph]# ceph health
HEALTH_OK
[[email protected] ceph]# ceph health
HEALTH_OK
[[email protected] ceph]# ceph -s
cluster 69f64f6d-f084-4b5e-8ba8-7ba3cec9d927
health HEALTH_OK
monmap e1: 1 mons at {node1=10.10.10.21:6789/0}
election epoch 3, quorum 0 node1
osdmap e14: 3 osds: 3 up, 3 in
flags sortbitwise,require_jewel_osds
pgmap v29: 64 pgs, 1 pools, 0 bytes data, 0 objects
15459 MB used, 45950 MB / 61410 MB avail
64 active+clean
本文出自 “若不奮鬥，何以稱王” 博客，請務必保留此出處http://wangzc.blog.51cto.com/12875919/1966109
基於Centos7.4搭建Ceph