Corosync+Pacemaker+NFS+Mysql高可用叢集部署
Corosync+Pacemaker+NFS+Mysql高可用叢集部署（使用資源管理工具crmsh配置）
框架：crmsh(Corosync+pacemaker)+nfs+mysql
叢集節點1:192.168.88.132 cen7.field.com
叢集節點2:192.168.88.133 node2.field.com
叢集節點3:192.168.88.135 node3.field.com
vip: 192.168.88.188 資源代理：ocf:heartbeat:IPaddr
nfs伺服器：node3.field.com 資源代理：ocf:heartbeat:Filesystem
mysql伺服器：cen7.field.com node1.field.com 資源代理：lsb:mysqld
配置叢集的前提
（1）、時間同步；
（2）、基於當前正在使用的主機名互相訪問；
（3）、是否會用到仲裁裝置；
一、配置NFS伺服器：共享目錄作為mysql的datadir
該步驟中：主要通過建立邏輯卷並設定開機自動掛載到nfs共享目錄，用於存放mysql資料庫檔案，其所有者必須是mysql使用者和mysql組。
1、建立邏輯卷
1）、fdisk建立物理卷分割槽
[[email protected] ~]# fdisk /dev/sda
歡迎使用 fdisk (util-linux 2.23.2)。
更改將停留在記憶體中，直到您決定將更改寫入磁碟。
使用寫入命令前請三思。
命令(輸入 m 獲取幫助)：p
磁碟 /dev/sda：32.2 GB, 32212254720 位元組，62914560 個扇區
Units = 扇區 of 1 * 512 = 512 bytes
扇區大小(邏輯/物理)：512 位元組 / 512 位元組
I/O 大小(最小/最佳)：512 位元組 / 512 位元組
磁碟標籤型別：dos
磁碟識別符號：0x00057c95
裝置 Boot Start End Blocks Id System
/dev/sda1 * 2048 1075199 536576 83 Linux
/dev/sda2 1075200 3174399 1049600 82 Linux swap / Solaris
/dev/sda3 3174400 36728831 16777216 83 Linux
命令(輸入 m 獲取幫助)：n
Partition type:
p primary (3 primary, 0 extended, 1 free)
e extended
Select (default e): p
已選擇分割槽 4
起始 扇區 (36728832-62914559，預設為 36728832)：
將使用預設值 36728832
Last 扇區, +扇區 or +size{K,M,G} (36728832-62914559，預設為 62914559)：+2G
分割槽 4 已設定為 Linux 型別，大小設為 2 GiB
命令(輸入 m 獲取幫助)：t
分割槽號 (1-4，預設 4)：4
Hex 程式碼(輸入 L 列出所有程式碼)：8e -->說明：用作物理卷的分割槽id為8e
已將分割槽“Linux”的型別更改為“Linux LVM”
命令(輸入 m 獲取幫助)：w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: 裝置或資源忙.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
正在同步磁碟。
2）、使用partprobe讓核心識別新建的分割槽
[[email protected] ~]# partprobe /dev/sda
3）、檢視新建立的分割槽
[[email protected] ~]# fdisk -l
磁碟 /dev/sda：32.2 GB, 32212254720 位元組，62914560 個扇區
Units = 扇區 of 1 * 512 = 512 bytes
扇區大小(邏輯/物理)：512 位元組 / 512 位元組
I/O 大小(最小/最佳)：512 位元組 / 512 位元組
磁碟標籤型別：dos
磁碟識別符號：0x00057c95
裝置 Boot Start End Blocks Id System
/dev/sda1 * 2048 1075199 536576 83 Linux
/dev/sda2 1075200 3174399 1049600 82 Linux swap / Solaris
/dev/sda3 3174400 36728831 16777216 83 Linux
/dev/sda4 36728832 40923135 2097152 8e Linux LVM -->標記為LVM
4）、建立物理卷
[[email protected] ~]# pvcreate /dev/sda4
Physical volume "/dev/sda4" successfully created.
5）、建立卷組
[[email protected] ~]# vgcreate myvg /dev/sda4
Volume group "myvg" successfully created
6）、建立大小為500M的邏輯卷
[[email protected] ~]# lvcreate -L 500M -n lv_mydata myvg
Logical volume "lv_mydata" created.
7）、格式化邏輯卷
[[email protected] ~]# mke2fs -j /dev/myvg/lv_mydata
mke2fs 1.42.9 (28-Dec-2013)
檔案系統標籤=
OS type: Linux
塊大小=1024 (log=0)
分塊大小=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
128016 inodes, 512000 blocks
25600 blocks (5.00%) reserved for the super user
第一個資料塊=1
Maximum filesystem blocks=67633152
63 block groups
8192 blocks per group, 8192 fragments per group
2032 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
Allocating group tables: 完成
正在寫入inode表: 完成
Creating journal (8192 blocks): 完成
Writing superblocks and filesystem accounting information: 完成
2、掛載共享儲存
1）、建立掛載的目錄/mydata作為nfs共享儲存介質
[[email protected] ~]# mkdir /mydata
2）、設定開機自動掛載
[[email protected] ~]# vim /etc/fstab
/dev/myvg/lv_mydata /mydata ext3 defaults 0 0
3）、掛載測試掛載成功與否
[[email protected] ~]# mount -a
[[email protected] ~]# ls /mydata/
lost+found
3、啟動nfs服務並配置共享儲存
1）、啟動nfs服務
[[email protected] ~]# systemctl start nfs.service
[[email protected] ~]# systemctl status nfs.service
● nfs-server.service - NFS server and services
Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled)
Active: active (exited) since 六 2018-08-04 03:43:47 CST; 6s ago
Process: 1442 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS)
Process: 1438 ExecStartPre=/bin/sh -c /bin/kill -HUP `cat /run/gssproxy.pid` (code=exited, status=0/SUCCESS)
Process: 1436 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
Main PID: 1442 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/nfs-server.service
8月 04 03:43:47 node3.field.com systemd[1]: Starting NFS server and services...
8月 04 03:43:47 node3.field.com systemd[1]: Started NFS server and services.
2）、新增共享儲存檔案，用於存放mysql資料的目錄
[[email protected] ~]# vim /etc/exports
#/www/hadocs 192.168.88.0/24(rw)
/mydata 192.168.88.0/24(rw,no_root_squash)
#該網段的主機可讀寫，不需root使用者
3）、編輯測試頁
[[email protected] ~]# echo "<h1>Test page on NFS Service</h1>" >/mydata/index.html
4）、重啟nfs服務並修改共享目錄屬組屬主為mysql，注意：此步驟為必須操作
[[email protected] ~]# systemctl restart nfs.service
[[email protected] ~]# chown -R mysql.mysql /mydata
chown: 無效的使用者: "mysql.mysql"
[[email protected] ~]# groupadd -g 306 -r mysql
[[email protected] ~]# useradd -g mysql -u 306 -s /sbin/nologin mysql
[[email protected] ~]# id mysql
uid=306(mysql) gid=306(mysql) 組=306(mysql)
[[email protected] ~]# chown -R mysql.mysql /mydata
二、編譯、安裝、配置、測試Mariadb
該步驟主要分兩大步：cen7節點安裝和node2節點安
（一）、cen7節點上的編譯、安裝、配置、測試Mariadb
1、安裝前配置
1）、建立mysql使用者、組和掛載目錄
[[email protected] ~]# groupadd -g 306 -r mysql
[[email protected] ~]# useradd -u 306 -g mysql -r mysql
[[email protected] ~]# mkdir /mydata
2）、掛載NFS檔案系統並確認
[[email protected] ~]# mount -t nfs 192.168.88.135:/mydata /mydata
[[email protected] ~]# mount |grep nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.88.135:/mydata on /mydata type nfs4 (rw,relatime,vers=4.1,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.88.132,local_lock=none,addr=192.168.88.135)
[[email protected] ~]# ls /mydata
index.html lost+found
3）、以mysql使用者身份建立data目錄，作為初始化時的資料存放目錄（datadir）
注意：mysql初始化只需在叢集節點中的一個進行一次，後續叢集會自動掛載配置，另一臺機mariadb不需初始化，所以不用以mysql使用者建立data目錄
[[email protected] ~]# su - mysql
su: 警告：無法更改到 /home/mysql 目錄: 沒有那個檔案或目錄
-bash-4.2$ cd /mydata
-bash-4.2$ mkdir data
-bash-4.2$ exit
登出
2、原始碼編譯安裝Mariadb
說明：此處採用10.0以上版本可能會因為c++版本低編譯無法通過，筆者採用mariadb-5.5.61.tar.gz安裝包
[[email protected] ~]# ls
anaconda-ks.cfg mariadb-5.5.61.tar.gz python-parallax-1.0.0a1-7.1.noarch.rpm
crmsh-3.0.0-6.2.noarch.rpm ~None python-pssh-2.3.1-7.3.noarch.rpm
crmsh-scripts-3.0.0-6.2.noarch.rpm pssh-2.3.1-7.3.noarch.rpm
1）、編譯安裝mariadb
[[email protected] ~]# tar xf mariadb-5.5.61.tar.gz -C /usr/local
[[email protected] ~]# cd /usr/local/
[[email protected] local]# ln -sv mariadb-5.5.61 mysql
"mysql" -> "mariadb-5.5.61"
[[email protected] local]# cd mysql
注意：修改檔案屬主屬組為mysql
[[email protected] mysql]# chown -R mysql:mysql .
[[email protected] mysql]# cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/mydata/data/ -DSYSCONFDIR=/etc -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DWITH_READLINE=1 -DWITH_SSL=system -DWITH_ZLIB=system -DWITH_LIBWRAP=0 -DMYSQL_UNIX_ADDR=/tmp/mysql.sock -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci
[[email protected] mysql]# make && make install
[[email protected] mysql]# chown -R mysql:mysql .
2）、初始化mysql資料庫：只需在該節點進行初始化，後續會自動掛載。
[[email protected] mysql]# scripts/mysql_install_db --user=mysql --datadir=/mydata/data
Installing MariaDB/MySQL system tables in '/mydata/data' ...
180803 6:02:53 [Note] ./bin/mysqld (mysqld 5.5.61-MariaDB) starting as process 60647 ...
OK
Filling help tables...
180803 6:02:55 [Note] ./bin/mysqld (mysqld 5.5.61-MariaDB) starting as process 60656 ...
OK
To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system
PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
To do so, start the server, then issue the following commands:
'./bin/mysqladmin' -u root password 'new-password'
'./bin/mysqladmin' -u root -h cen7.field.com password 'new-password'
Alternatively you can run:
'./bin/mysql_secure_installation'
which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.
See the MariaDB Knowledgebase at http://mariadb.com/kb or the
MySQL manual for more instructions.
You can start the MariaDB daemon with:
cd '.' ; ./bin/mysqld_safe --datadir='/mydata/data'
You can test the MariaDB daemon with mysql-test-run.pl
cd './mysql-test' ; perl mysql-test-run.pl
Please report any problems at http://mariadb.org/jira
The latest information about MariaDB is available at http://mariadb.org/.
You can find additional information about the MySQL part at:
http://dev.mysql.com
Consider joining MariaDB's strong and vibrant community:
https://mariadb.org/get-involved/
[[email protected] mysql]# chown -R root .
3）、複製my-large.cnf為mysql主配置檔案並修改
[[email protected] mysql]# cp support-files/my-large.cnf /etc/my.cnf
cp：是否覆蓋"/etc/my.cnf"？ y
[[email protected] mysql]# vim /etc/my.cnf
#thread_concurrency=CPU*2
thread_concurrency = 6
#指定mysql資料檔案的存放位置：
datadir = /mydata/data
4）、配置使用service管理mysqld服務並關閉開機自啟動，用CRM控制mysql啟停
[[email protected] mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysqld
[[email protected] mysql]# chkconfig --add mysqld
[[email protected] mysql]# chkconfig mysqld off
[[email protected] mysql]# service mysqld start
env: /etc/init.d/mysqld: 許可權不夠
[[email protected] mysql]# chmod +x /etc/rc.d/init.d/mysqld
3、啟動mysql服務並確認狀態
[[email protected] mysql]# service mysqld start
Starting MariaDB.180803 06:08:41 mysqld_safe Logging to '/mydata/data/cen7.field.com.err'.
180803 06:08:41 mysqld_safe Starting mysqld daemon with databases from /mydata/data
. SUCCESS!
[[email protected] mysql]# ps -ef | grep 3306
mysql 61180 60874 1 06:08 pts/0 00:00:00 /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/mydata/data --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=/mydata/data/cen7.field.com.err --pid-file=/mydata/data/cen7.field.com.pid --socket=/tmp/mysql.sock --port=3306
root 61233 3454 2 06:09 pts/0 00:00:00 grep --color=auto 3306
[[email protected] mysql]# service mysqld stop
Shutting down MariaDB..... SUCCESS!
4、針對使用者操作習慣的更改
使用原始碼包編譯安裝mariadb使用方法與使用者習慣有所出入，通過建立連結，配置環境變數可以使mysql符合使用者操作習慣，此步可以不進行。
1）、輸出mysql的man手冊至man命令的查詢路徑
[[email protected] mysql]# vim /etc/man.config
MANPATH /usr/local/mysql/man
2）、將mysql的標頭檔案連結至系統標頭檔案路徑/usr/include：
[[email protected] mysql]# ln -sv /usr/local/mysql/include /usr/include/mysql
"/usr/include/mysql" -> "/usr/local/mysql/include"
3）、輸出mysql的庫檔案給系統庫查詢路徑
[[email protected] mysql]# echo '/usr/local/mysql/lib' > /etc/ld.so.conf.d/mysql.conf
4）、讓系統重新載入系統庫
[[email protected] mysql]# ldconfig
5）、編輯/etc/profile新增mysql環境，讓系統所有使用者可以直接使用mysql的相關命令:
[[email protected] mysql]# vim /etc/profile
MYSQL_HOME=/usr/local/mysql
PATH=$PATH:$MYSQL_HOME/bin
6）、使配置生效
[[email protected] mysql]# . /etc/profile
5、驗證mysql是否安裝成功：啟動mysql，連線mysql客戶端。
[[email protected] mysql]# service mysqld start
Starting MariaDB.180803 06:12:02 mysqld_safe Logging to '/mydata/data/cen7.field.com.err'.
180803 06:12:02 mysqld_safe Starting mysqld daemon with databases from /mydata/data
.. SUCCESS!
[[email protected] mysql]# mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 2
Server version: 5.5.61-MariaDB Source distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> quit
Bye
6、複製配置檔案到node2節點：mysql配置檔案各節點相同
[[email protected] corosync]# scp /etc/my.cnf node2:/etc/my.cnf
[[email protected] corosync]# service mysqld stop
Shutting down MariaDB.... SUCCESS!
7、解除安裝共享目錄：最後必須解除安裝共享目錄
[[email protected] corosync]# umount /mydata
（二）、node2.field.com上的配置Mariadb
因node2節點上mariadb安裝配置同cen7節點，只是不需要再次初始化資料庫，此處不再針對操作做出說明。
以下是操作步驟及程式碼：
[[email protected] ~]# tar xf mariadb-5.5.61.tar.gz -C /usr/local/
[[email protected] ~]# groupadd -g 306 -r mysql
[[email protected] ~]# useradd -u 306 -g mysql -r mysql
[[email protected] ~]# mkdir /mydata
[[email protected] ~]# cd /usr/local/
[[email protected] local]# ln -sv mariadb-5.5.61 mysql
"mysql" -> "mariadb-5.5.61"
[[email protected] local]# cd mysql
[[email protected] mysql]# chown -R mysql:mysql .
[[email protected] mysql]# yum install cmake openssl-devel zlib-devel ncurses-devel bision gcc gcc-c++ -y
[[email protected] mysql]# cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/mydata/data/ -DSYSCONFDIR=/etc -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DWITH_READLINE=1 -DWITH_SSL=system -DWITH_ZLIB=system -DWITH_LIBWRAP=0 -DMYSQL_UNIX_ADDR=/tmp/mysql.sock -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci
[[email protected] mysql]# make && make install
[[email protected] mysql]# chown -R root:mysql .
[[email protected] mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysqld
[[email protected] mysql]# chkconfig --add mysqld
[[email protected] mysql]# chkconfig mysqld off
[roo[email protected] mysql]# vim /etc/man.config
[[email protected] mysql]# ln -sv /usr/local/mysql/include /usr/include/mysql
"/usr/include/mysql" -> "/usr/local/mysql/include"
[[email protected] mysql]# echo '/usr/local/mysql/lib' > /etc/ld.so.conf.d/mysql.conf
[[email protected] mysql]# ldconfig
[[email protected] mysql]# vim /etc/profile
MYSQL_HOME=/usr/local/mysql
PATH=$PATH:$MYSQL_HOME/bin
[[email protected] mysql]# vim /etc/profile
[[email protected] mysql]# . /etc/profile
[[email protected] mysql]# systemctl start mysqld.service
Failed to start mysqld.service: Unit not found.
[[email protected] mysql]# systemctl start mysql.service
Failed to start mysql.service: Unit not found.
[[email protected] mysql]# chmod +x /etc/rc.d/init.d/mysqld
[[email protected] mysql]# service mysqld start
Starting MariaDB.180803 06:49:01 mysqld_safe Logging to '/mydata/data/node2.field.com.err'.
180803 06:49:01 mysqld_safe Starting mysqld daemon with databases from /mydata/data
/usr/local/mysql/bin/mysqld_safe_helper: Can't create/write to file '/mydata/data/node2.field.com.err' (Errcode: 2)
ERROR!
#此報錯是因為未掛載mysql資料存放目錄
掛載/mydata目錄，驗證Mariadb是否安裝成功
[[email protected] corosync]# mount -t nfs 192.168.88.135:/mydata /mydata
[[email protected] corosync]# service mysqld start
Starting MariaDB.180803 07:04:16 mysqld_safe Logging to '/mydata/data/node2.field.com.err'.
180803 07:04:16 mysqld_safe Starting mysqld daemon with databases from /mydata/data
SUCCESS!
[[email protected] corosync]# mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 2
Server version: 5.5.61-MariaDB Source distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]>
MariaDB [(none)]> quit
Bye
#可以看到：掛載/mydata目錄後，Mariadb客戶端接入成功。
注意：測試成功後關閉mariadb並解除安裝共享目錄
[[email protected] corosync]# service mysqld stop
[[email protected] corosync]# umount /mydata
三、使用crmsh配置Corosync+Pacemaker
1、將叢集節點加入/etc/ansible/hosts中，便於管理
#或使用systemctl start|stop|restart xxxx
[[email protected] corosync]# vim /etc/ansible/hosts
[hamysql]
192.168.88.132
192.168.88.133
2、啟動corosync和pacemaker，刪除此前配置
[[email protected] mysql]# ansible hamysql -m service -a 'name=corosync state=started'
[[email protected] mysql]# ansible hamysql -m service -a 'name=pacemaker state=started'
[[email protected] mysql]# crm status
Stack: corosync
Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum
Last updated: Sat Aug 4 04:33:13 2018
Last change: Sat Aug 4 01:03:41 2018 by root via cibadmin on cen7.field.com
2 nodes configured
3 resources configured
Online: [ cen7.field.com node2.field.com ]
Full list of resources:
Resource Group: mycluster
mysqlip (ocf::heartbeat:IPaddr): Started node2.field.com
mynfs (ocf::heartbeat:Filesystem): Stopped
mysqld (lsb:mysqld): Stopped
使用crm互動模式：刪除此前定義的資源
[[email protected] mysql]# crm
crm(live)# resource
crm(live)resource# help
#刪除資源之前需要先停止資源，各命令用法可以使用“help [command]”檢視
crm(live)resource# help stop
crm(live)resource# show
Resource Group: mycluster
mysqlip (ocf::heartbeat:IPaddr): Started
mynfs (ocf::heartbeat:Filesystem): Stopped
mysqld (lsb:mysqld): Stopped
crm(live)resource# stop mysqlip
crm(live)resource# show
Resource Group: mycluster
mysqlip (ocf::heartbeat:IPaddr): Stopped (disabled)
mynfs (ocf::heartbeat:Filesystem): Stopped
mysqld (lsb:mysqld): Stopped
crm(live)resource# cd
crm(live)# configure
crm(live)configure# help delete
crm(live)configure# delete mycluster
crm(live)configure# delete mysqlip
crm(live)configure# delete mynfs
crm(live)configure# delete mysqld
crm(live)configure# verify
ERROR: Warnings found during check: config may not be valid
#注意：如果有無意義的配置會報如上error，使用show檢視配置，如此處的cluster-name。
#直接使用edit編輯刪除無效配置:wq儲存退出即可。
crm(live)configure# show
node 1: cen7.field.com \
attributes standby=off
node 2: node2.field.com \
attributes standby=off
property cib-bootstrap-options: \
have-watchdog=false \
dc-version=1.1.18-11.el7_5.3-2b07d5c5a9 \
cluster-infrastructure=corosync \
cluster-name=mycluster \
stonith-enabled=false \
default-resource-stickiness=0 \
no-quorum-policy=ignore
crm(live)configure# edit
crm(live)configure# verify
crm(live)configure# verify
#可以看到，刪除無效配置後verify不會報ERROR
crm(live)configure# show
node 1: cen7.field.com \
attributes standby=off
node 2: node2.field.com \
attributes standby=off
property cib-bootstrap-options: \
have-watchdog=false \
dc-version=1.1.18-11.el7_5.3-2b07d5c5a9 \
cluster-infrastructure=corosync \
stonith-enabled=false \
no-quorum-policy=ignore
crm(live)configure# commit
crm(live)configure# quit
bye
3、編輯配置檔案corosync.conf
[[email protected] corosync]# cp corosync.conf corosync.conf.bak080222
[[email protected] corosync]# cat corosync.conf.bak080215 >corosync.conf
[[email protected] corosync]# vim corosync.conf
[[email protected] corosync]# grep -v '^[[:space:]]*#' corosync.conf
totem {
version: 2
crypto_cipher: aes128
crypto_hash: sha1
secauth: on
interface {
ringnumber: 0
bindnetaddr: 192.168.88.0
mcastaddr: 239.188.1.188
mcastport: 5405
ttl: 1
}
}
nodelist {
node {
ring0_addr: 192.168.88.132
nodeid: 1
}
node {
ring0_addr: 192.168.88.133
nodeid: 2
}
}
logging {
fileline: off
to_stderr: no
to_logfile: yes
logfile: /var/log/cluster/corosync.log
to_syslog: no
debug: off
timestamp: on
logger_subsys {
subsys: QUORUM
debug: off
}
}
quorum {
provider: corosync_votequorum
}
4、生成節點間通訊時用到的認證金鑰檔案，並複製到node2節點
[[email protected] corosync]# corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Press keys on your keyboard to generate entropy (bits = 920).
Press keys on your keyboard to generate entropy (bits = 1000).
Writing corosync key to /etc/corosync/authkey.
[[email protected] corosync]# ls
authkey corosync.conf corosync.conf.bak080215 corosync.conf.bak080222 corosync.conf.example corosync.conf.example.udpu corosync.xml.example uidgid.d
[[email protected] corosync]# scp -p authkey corosync.conf node2:/etc/corosync/
authkey 100% 128 44.1KB/s 00:00
corosync.conf
5、啟動corosync和pacemaker，並確認狀態是否正常
[[email protected] corosync]# ansible hamysql -m service -a 'name=corosync state=started enabled=yes'
[[email protected] corosync]# ansible hamysql -m service -a 'name=pacemaker state=started enabled=yes'
1）、檢視是否能夠正常發出初始化成員節點通知
[[email protected] corosync]# grep TOTEM /var/log/messages|tail
Aug 3 06:25:24 cen7 corosync[62060]: [TOTEM ] A new membership (192.168.88.132:348) was formed. Members
Aug 3 06:25:27 cen7 corosync[62060]: [TOTEM ] A new membership (192.168.88.132:352) was formed. Members
Aug 3 06:25:29 cen7 corosync[62060]: [TOTEM ] A new membership (192.168.88.132:356) was formed. Members joined: 3
Aug 3 06:26:16 cen7 corosync[62060]: [TOTEM ] A processor failed, forming new configuration.
Aug 3 06:26:16 cen7 corosync[62060]: [TOTEM ] A new membership (192.168.88.132:368) was formed. Members joined: 3 2 left: 3 2
Aug 3 06:26:16 cen7 corosync[62060]: [TOTEM ] Failed to receive the leave message. failed: 3 2
Aug 3 06:28:21 cen7 corosync[62060]: [TOTEM ] A new membership (192.168.88.132:372) was formed. Members
Aug 3 06:30:14 cen7 corosync[62060]: [TOTEM ] A processor failed, forming new configuration.
Aug 3 06:30:14 cen7 corosync[62060]: [TOTEM ] A new membership (192.168.88.132:376) was formed. Members
Aug 3 06:34:54 cen7 corosync[62060]: [TOTEM ] A new membership (192.168.88.132:380) was formed. Members left: 2
2）、檢視corosync引擎狀態
[[email protected] corosync]# grep -e "Corosync Cluster Engine" -e "configuration file" /var/log/messages|tail -n 3
Aug 4 04:16:16 cen7 systemd: Starting Corosync Cluster Engine...
Aug 4 04:16:23 cen7 corosync: Starting Corosync Cluster Engine (corosync): [ 確定 ]
Aug 4 04:16:23 cen7 systemd: Started Corosync Cluster Engine.
3）、確認啟動過程無ERROR產生
[[email protected] corosync]# grep ERROR: /var/log/messages | grep -v unpack_resources
6、檢視叢集節點狀態
1）、檢視cen7節點狀態：正常
[[email protected] corosync]# systemctl status corosync.service
● corosync.service - Corosync Cluster Engine
Loaded: loaded (/usr/lib/systemd/system/corosync.service; enabled; vendor preset: disabled)
Active: active (running) since 六 2018-08-04 04:16:23 CST; 41min ago
Docs: man:corosync
man:corosync.conf
man:corosync_overview
Process: 765 ExecStart=/usr/share/corosync/corosync start (code=exited, status=0/SUCCESS)
Main PID: 795 (corosync)
CGroup: /system.slice/corosync.service
└─795 corosync
8月 04 04:16:16 cen7.field.com systemd[1]: Starting Corosync Cluster Engine...
8月 04 04:16:23 cen7.field.com corosync[765]: Starting Corosync Cluster Engine (corosync): [ 確定 ]
8月 04 04:16:23 cen7.field.com systemd[1]: Started Corosync Cluster Engine.
[[email protected] corosync]# corosync-cfgtool -s
Printing ring status.
Local node ID 1
RING ID 0
id = 192.168.88.132 -->說明：此處如果為127.0.0.1則說明啟動錯誤
#編輯/etc/hosts檔案，登出127.0.0.1行，只保留ip和域名後重啟Corosync即可
status = ring 0 active with no faults
2）、關閉stonith
[[email protected] corosync]# crm configure property stonith-enabled=false
使用crm_verify檢查配置有沒問題
[[email protected] corosync]# crm_verify -L -V
3）、檢視node2節點狀態：正常
[[email protected] ~]# service corosync status
Redirecting to /bin/systemctl status corosync.service
● corosync.service - Corosync Cluster Engine
Loaded: loaded (/usr/lib/systemd/system/corosync.service; enabled; vendor preset: disabled)
Active: active (running) since 六 2018-08-04 04:16:23 CST; 34min ago
Docs: man:corosync
man:corosync.conf
man:corosync_overview
Process: 839 ExecStart=/usr/share/corosync/corosync start (code=exited, status=0/SUCCESS)
Main PID: 858 (corosync)
CGroup: /system.slice/corosync.service
└─858 corosync
8月 04 04:16:17 node2.field.com systemd[1]: Starting Corosync Cluster Engine...