Node1：

CentOS6 192.168.9.168 192-168-9-168

CentOS6 192.168.9.176 192-168-9-176

VIP： 192.168.9.144

corosync和pacemaker為yum安裝

二、環境部署

服務器初始化腳本執行（主要時間同步，防火墻，主機名，ip配置，關閉不必要服務等）

1，更改yum源

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo

yum clean all

2，安裝epel

rpm -Uvh

rpm -Uvh http://mirrors.kernel.org/fedora-epel/epel-release-latest-6.noarch.rpm

3，設置主機名

hostname

vim /etc/sysconfig/network

4，綁定host

cat /etc/hosts

192.168.9.168 192-168-9-168

192.168.9.176 192-168-9-176

可配置秘鑰通信：

ssh-keygen -P ''

ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]

ssh 192.168.9.168 'date'

5，安裝corosync+pacemaker

yum install -y corosync pacemaker

安裝httpd

yum install -y httpd

6，配置

# grep -v "^[:space:]\{0,\}.*#" corosync.conf | grep -v ^$

compatibility: whitetank

totem {

version: 2

secauth: on

threads: 0

interface {

ringnumber: 0

bindnetaddr: 192.168.9.0

mcastaddr: 239.255.1.91

mcastport: 5405

ttl: 1

}

}

logging {

fileline: off

to_stderr: no

to_logfile: yes

logfile: /var/log/cluster/corosync.log

to_syslog: no

debug: off

timestamp: on

logger_subsys {

subsys: AMF

debug: off

}

}

service {

ver: 0

name: pacemaker

}

aisexec {

user: root

group: root

}

7，啟動

service corosync start

8，查看啟動是否正常

crm_mon

tail /var/log/cluster/corosync.log

grep -e "Corosync Cluster Engine" -e "configuration file" /var/log/cluster/corosync.log

grep TOTEM /var/log/cluster/corosync.log

grep ERROR: /var/log/cluster/corosync.log | grep -v unpack_resources

grep pcmk_startup /var/log/cluster/corosync.log

9，crm使用方法

http://blog.51yip.com/server/1680.html

（1）yum安裝

cd /etc/yum.repos.d/ ; wget http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo

yum install -y crmsh*

（2）×××下載包

http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/noarch/

yum install -y *.rpm

（3）crm使用

# crm #輸入crm命令，進入crm sh 模式

crm(live)# help #輸入help查看幫助

crm(live)# configure #輸入configure進入configure模式下，

crm(live)configure# #敲兩下tab鍵就會顯示configure下全部命令

crm(live)configure# help node

crm(live)configure# cd ../ #返回上一級

1）查看集群狀態

crm(live)# status

2）查看一下CIB配置：

crm(live)# configure

crm(live)configure# show

crm(live)configure# show xml #查xml格式文件

3）檢測一下配置文件是否有錯，可以看到剛開始默認STONITH配置錯誤，因為沒有STONITH設備

crm(live)configure# verify

4）查看當前集群系統所支持的資源代理類型

crm(live)# ra

crm(live)ra# classes

5）查看某種類別下的所用資源代理的列表

crm(live)ra# list lsb

crm(live)ra# list ocf heartbeat

crm(live)ra# list ocf pacemaker

6）查看某個資源代理的配置方法，例如查看了heartbeat的ip資源的配置方法：

crm(live)ra# info ocf:heartbeat:IPaddr

7）節點下線，上線

crm node standby

crm node online

crm status

crm node status

8）資源管理

crm(live)# resource

crm(live)# resource

crm(live)resource# status vip

crm(live)resource# migrate vip 192-168-9-176

crm(live)resource# unmigrate

crm(live)resource# stop webservice

9）排列約束

crm(live)configure# colocation webserver_with_webip inf: webserver webip

10）順序約束

crm(live)configure# order webip_before_webserver Mandatory: webip webserver

11）位置約束

crm(live)configure# location webip_on_node2 webip 50: 192-168-9-176

10，crmsh配置集群資源

說明：vip，nfs，httpd三個資源配置到資源組，並按順序啟動；雙節點投票設置為ignore並添加監控

（1）解決stonith配置錯誤，設置票數失效操作

crm(live)# configure

crm(live)configure# show

node 192-168-9-168

node 192-168-9-176

property cib-bootstrap-options: \

have-watchdog=false \

dc-version=1.1.15-5.el6-e174ec8 \

cluster-infrastructure="classic openais (with plugin)" \

expected-quorum-votes=2

crm(live)configure# verify

ERROR: error: unpack_resources: Resource start-up disabled since no STONITH resources have been defined

error: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option

error: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity

Errors found during check: config not valid

crm(live)configure# property stonith-enabled=false

crm(live)configure# property no-quorum-policy=ignore

crm(live)configure# verify

crm(live)configure# commit

（2）創建IP地址資源，IP資源是主資源

crm(live)configure# primitive vip ocf:heartbeat:IPaddr params ip=192.168.9.144 nic=eth0 cidr_netmask=24 op monitor interval=10s timeout=20s

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# show

node 192-168-9-168

node 192-168-9-176

primitive vip IPaddr \

params ip=192.168.9.144 nic=eth0 cidr_netmask=24

property cib-bootstrap-options: \

have-watchdog=false \

dc-version=1.1.15-5.el6-e174ec8 \

cluster-infrastructure="classic openais (with plugin)" \

expected-quorum-votes=2 \

stonith-enabled=false

（3）配置nfs和httpd服務

crm(live)configure# primitive nfs ocf:heartbeat:Filesystem params device=192.168.9.225:/web/hahtml directory=/var/www/html fstype=nfs op monitor interval=20s timeout=40s

crm(live)configure# primitive httpd lsb:httpd op monitor interval=10s timeout=20s

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# show

node 192-168-9-168

node 192-168-9-176

primitive httpd lsb:httpd

primitive nfs Filesystem \

params device="192.168.9.225:/web/hahtml" directory="/var/www/html" fstype=nfs

primitive vip IPaddr \

params ip=192.168.9.144 nic=eth0 cidr_netmask=24

property cib-bootstrap-options: \

have-watchdog=false \

dc-version=1.1.15-5.el6-e174ec8 \

cluster-infrastructure="classic openais (with plugin)" \

expected-quorum-votes=2 \

stonith-enabled=false

crm(live)configure# cd

crm(live)# status

Stack: classic openais (with plugin)

Current DC: 192-168-9-168 (version 1.1.15-5.el6-e174ec8) - partition with quorum

Last updated: Fri Mar 2 14:53:37 2018 Last change: Fri Mar 2 14:52:46 2018 by root via cibadmin on 192-168-9-168

, 2 expected votes

2 nodes and 3 resources configured

Online: [ 192-168-9-168 192-168-9-176 ]

Full list of resources:

vip (ocf::heartbeat:IPaddr): Started 192-168-9-168

nfs (ocf::heartbeat:Filesystem): Started 192-168-9-176

httpd (lsb:httpd): Started 192-168-9-168

（4）創建資源組，將三個資源在同一節點啟動

crm(live)# configure

crm(live)configure# group group_webservice vip nfs httpd

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# show

node 192-168-9-168

node 192-168-9-176

primitive httpd lsb:httpd

primitive nfs Filesystem \

params device="192.168.9.225:/web/hahtml" directory="/var/www/html" fstype=nfs

primitive vip IPaddr \

params ip=192.168.9.144 nic=eth0 cidr_netmask=24

group group_webservice vip nfs httpd

property cib-bootstrap-options: \

have-watchdog=false \

dc-version=1.1.15-5.el6-e174ec8 \

cluster-infrastructure="classic openais (with plugin)" \

expected-quorum-votes=2 \

stonith-enabled=false

crm(live)configure# cd

crm(live)# status

Stack: classic openais (with plugin)

Current DC: 192-168-9-168 (version 1.1.15-5.el6-e174ec8) - partition with quorum

Last updated: Fri Mar 2 14:55:12 2018 Last change: Fri Mar 2 14:55:04 2018 by root via cibadmin on 192-168-9-168

, 2 expected votes

2 nodes and 3 resources configured

Online: [ 192-168-9-168 192-168-9-176 ]

Full list of resources:

Resource Group: group_webservice

vip (ocf::heartbeat:IPaddr): Started 192-168-9-168

nfs (ocf::heartbeat:Filesystem): Started 192-168-9-168

httpd (lsb:httpd): Started 192-168-9-168

11，測試

（1）使節點為standy 測試切換

（2）service corosync stop關閉corosync 測試選票系統

（3）killall httpd服務測試監控

corosync+pacemaker配置高可用集群