Linux系統——MHA-Atlas-MySQL高可用叢集

阿新 • • 發佈：2018-12-08

MHA

MHA介紹
MHA（Master High Availability）目前在MySQL高可用方面是一個相對成熟的解決方案，是一套優秀的作為MySQL高可用性環境下故障切換和主從提升的高可用軟體。在MySQL故障切換過程中，MHA能做到0~30秒之內自動完成資料庫的故障切換操作，並且在進行故障切換過程中，MHA能最大程度上保證資料庫的一致性，以達到真正意義上的高可用。

MHA組成
MHA由兩部分組成：MHA Manager（管理節點）和MHA Node（資料節點）。MHA Manager可以獨立部署在一臺獨立的機器上管理多個Master-Slave叢集，也可以部署在一臺Slave上，管理一臺Slave。當Master出現故障時，它可以自動將最新資料的Slave提升為新的Master，然後將所有其他的Slave重新指向新的Master。整個故障轉移過程對應程式是完全透明的。

MHA工作流程
（1）從宕機崩潰的master儲存二進位制日誌事件（binlog events）；
（2）識別含有最新更新的slave；
（3）應用差異的中繼日誌（relay log）到其他的slave；
（4）應用從master儲存的二進位制日誌事件（binlog events）；
（5）提升一個slave為新的master；
（6）使其他的slave連線新的master進行復制；
如果主資料庫宕機，複製主資料庫的二進位制日誌出來，然後去找出三個從庫的中繼日誌relay log最全最新的一個從庫，將最全的relay log日誌在所有從庫中同步（第一次資料同步）；然後將之前找到的最全的從庫提升為主資料庫，將複製出來的bin-log日誌放到新提升的主資料庫裡，再將其他所有從庫重新指向新提升的主庫，繼續主從複製。

MHA架構圖

MHA工具介紹
MHA軟體由兩部分組成，Manager工具包和Node工具包

#Manager工具包主要包括以下幾個工具：

masterha_check_ssh #檢查MHA的SSH配置狀況
masterha_check_repl #檢查MySQL複製狀況
masterha_check_status #檢測當前MHA執行狀態
masterha_master_monitor #檢測master是否宕機
masterha_manger #啟動MHA
masterha_master_switch #控制故障轉移（自動或者手動）
masterha_conf_host #新增或刪除配置的server資訊
masterha_secondary_check #試圖建立TCP連線從遠端伺服器
masterha_stop #停止MHA

#Node工具包主要包括以下幾個工具：

save_binary_logs #儲存和複製master的二進位制日誌
apply_diff_relay_logs #識別差異的中繼日誌事件
filter_mysqlbinlog #去除不必要的ROLLBACK事件
purge_relay_logs #清除中繼日誌

MySQL環境準備

實驗環境：
mysql-master eth0:192.168.239.128
mysql-slaveA eth0:192.168.239.129
mysql-slaveB eth0:192.168.239.130

需要的軟體包
1） mha管理節點安裝包：
mha4mysql-manager-0.56-0.el6.noarch.rpm
mha4mysql-manager-0.56.tar.gz
2） mha node節點安裝包：
mha4mysql-node-0.56-0.el6.noarch.rpm
mha4mysql-node-0.56.tar.gz
3） mysql中介軟體：
Atlas-2.2.1.el6.x86_64.rpm
4） mysql原始碼安裝包
mysql-5.6.17-linux-glibc2.5-x86_64.tar

安裝MySQL

主機名對映
[[email protected] ~]# ls
anaconda-ks.cfg mha4mysql-manager-0.56.tar.gz
Atlas-2.2.1.el6.x86_64.rpm mha4mysql-node-0.56-0.el6.noarch.rpm
install.log mha4mysql-node-0.56.tar.gz
install.log.syslog mysql-5.6.17-linux-glibc2.5-x86_64.tar.gz
mha4mysql-manager-0.56-0.el6.noarch.rpm yum.sh
[[email protected] ~]# ll mysql-5.6.17-linux-glibc2.5-x86_64.tar.gz
-rw-r--r--. 1 root root 305102088 Sep 4 2017 mysql-5.6.17-linux-glibc2.5-x86_64.tar.gz
[[email protected] ~]# echo "192.168.239.128 MySQL-Master" >> /etc/hosts
[[email protected] ~]# echo "192.168.239.129 MySQL-SlaveA" >> /etc/hosts
[[email protected] ~]# echo "192.168.239.130 MySQL-SlaveB" >> /etc/hosts
[[email protected] ~]# scp /etc/hosts [email protected]:/etc/
The authenticity of host '192.168.239.129 (192.168.239.129)' can't be established.
RSA key fingerprint is 2d:76:57:08:d0:37:08:9e:bd:f2:2c:c6:30:28:d7:be.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.239.129' (RSA) to the list of known hosts.
[email protected]'s password:
hosts 100% 245 0.2KB/s 00:00
[[email protected] ~]# scp /etc/hosts [email protected]:/etc/
The authenticity of host '192.168.239.130 (192.168.239.130)' can't be established.
RSA key fingerprint is 2d:76:57:08:d0:37:08:9e:bd:f2:2c:c6:30:28:d7:be.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.239.130' (RSA) to the list of known hosts.
[email protected]'s password:
hosts 100% 245 0.2KB/s 00:00

安裝MySQL（3臺都裝）
[[email protected] ~]# mount /dev/sr0 /media/cdrom/
mount: block device /dev/sr0 is write-protected, mounting read-only
[[email protected] ~]# yum -y install ncurses-devel libaio
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
[[email protected] ~]# tar xf mysql-5.6.17-linux-glibc2.5-x86_64.tar.gz -C /usr/local/
[[email protected] ~]# ln -s /usr/local/mysql-5.6.17-linux-glibc2.5-x86_64 /usr/local/mysql
[[email protected] ~]# useradd mysql -s /sbin/nologin -M
[[email protected] ~]# /usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data/
[[email protected] ~]# /bin/cp /usr/local/mysql/support-files/my-default.cnf /etc/my.cnf
[[email protected] ~]# /bin/cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysqld
[[email protected] ~]# ln -s /usr/local/mysql/bin/* /usr/local/bin/
[[email protected] ~]# which mysqladmin
/usr/local/bin/mysqladmin
[[email protected] ~]# mysqladmin -uroot password '123456'
#加入開機自啟動並啟動mysql
[[email protected] ~]# chkconfig mysqld on
[[email protected] ~]# chkconfig mysqld --list
mysqld 0:off 1:off 2:on 3:on 4:on 5:on 6:off
[[email protected] ~]# /etc/init.d/mysqld start
Starting MySQL.. SUCCESS!

配置基於GTID的主從複製

先決條件

（1）主庫和從庫都要開啟binlog
（2）主庫和從庫server-id不同
（3）要有主從複製使用者

主庫操作（MySQL-Master）

修改配置檔案
[[email protected] mysql]# vim /etc/my.cnf
[client]
socket = /usr/local/mysql/data/mysql.sock

[mysqld]
lower_case_table_names = 1
default-storage-engine = InnoDB
port = 3306
datadir = /usr/local/mysql/data
character-set-server = utf8
socket = /usr/local/mysql/data/mysql.sock

log_bin = mysql-bin #開啟binlog日誌
server_id = 1 #設定server_id

innodb_buffer_pool_size = 200M
slave-parallel-workers = 8
thread_cache_size = 600
back_log = 600
slave_net_timeout = 60
max_binlog_size = 512M
key_buffer_size = 8M
query_cache_size = 64M
join_buffer_size = 2M
sort_buffer_size = 2M
query_cache_type = 1
thread_stack = 192K
#重啟動MySQL服務
[[email protected] mysql]# /etc/init.d/mysqld restart
Shutting down MySQL.. SUCCESS!
Starting MySQL. SUCCESS!
#將主庫的配置檔案複製到從庫
[[email protected] mysql]# scp /etc/my.cnf [email protected]:/etc/
[email protected]'s password:
my.cnf 100% 595 0.6KB/s 00:00
[[email protected] mysql]# scp /etc/my.cnf [email protected]:/etc/
[email protected]'s password:
my.cnf 100% 595 0.6KB/s 00:00

建立主從複製使用者
[[email protected] mysql]# mysql -uroot -p123456
mysql> grant replication slave on *.* to [email protected]'192.168.239.%' identified by '123456';
Query OK, 0 rows affected (0.04 sec)

mysql> show grants for [email protected]'192.168.239.%';
+----------------------------------------------------------------------------------------------------------------------------+
| Grants for [email protected]% |
+----------------------------------------------------------------------------------------------------------------------------+
| GRANT REPLICATION SLAVE ON *.* TO 'rep'@'192.168.239.%' IDENTIFIED BY PASSWORD '*6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9' |
+----------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> exit
Bye

從庫操作（mysql-slaveA和mysql-slaveB）

修改配置檔案
#修改mysql-slaveA配置檔案(和mysql-master配置檔案一致)
#只需要修改server-id = 5選項
[[email protected] ~]# vim /etc/my.cnf
[client]
socket = /usr/local/mysql/data/mysql.sock

[mysqld]
lower_case_table_names = 1
default-storage-engine = InnoDB
port = 3306
datadir = /usr/local/mysql/data
character-set-server = utf8
socket = /usr/local/mysql/data/mysql.sock

log_bin = mysql-bin #從庫的binlog也要開啟
server_id = 5 #僅需修改此項

#修改mysql-slaveB配置檔案(和mysql-master配置檔案一致)
#只需要修改server-id = 10選項
[[email protected] ~]# vim /etc/my.cnf
[client]
socket = /usr/local/mysql/data/mysql.sock

[mysqld]
lower_case_table_names = 1
default-storage-engine = InnoDB
port = 3306
datadir = /usr/local/mysql/data
character-set-server = utf8
socket = /usr/local/mysql/data/mysql.sock

log_bin = mysql-bin #從庫的binlog也要開啟
server_id = 10 #僅需修改此項

在MySQL5.6版本里多了一個Gtid的功能，可以自動記錄主從複製位置點的資訊，並在日誌中輸出出來。

開啟GTID

編輯mysql配置檔案（主庫從庫都需要修改）

[[email protected] ~]# vim /etc/my.cnf

#在[mysqld]模組下面加入這三行
gtid_mode = ON
log_slave_updates #作用：雙主模式必須加入的引數
enforce_gtid_consistency
[[email protected] mysql]# /etc/init.d/mysqld restart
Shutting down MySQL.. SUCCESS!
Starting MySQL... SUCCESS!

再次在MySQL-Master中檢視GTID狀態

配置主從複製(mysql-slaveA,mysql-slaveB)

[[email protected] ~]# mysql -uroot -p123456
mysql> change master to master_host='192.168.239.128',master_user='rep',master_password='123456',master_auto_position=1; #主庫IP，主庫複製使用者，主庫複製用密碼，GTID位置點（自動追蹤需要同步的position）
Query OK, 0 rows affected, 2 warnings (0.22 sec)

開啟從庫的主從複製功能(mysql-slaveA,mysql-slaveB)

mysql> start slave;
Query OK, 0 rows affected, 1 warning (0.47 sec)

mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.239.128
Master_User: rep
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000003
Read_Master_Log_Pos: 151
Relay_Log_File: localhost-relay-bin.000002
Relay_Log_Pos: 361
Relay_Master_Log_File: mysql-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 151
Relay_Log_Space: 569
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID: 9a7e8455-f90b-11e8-b581-000c29e4bd28
Master_Info_File: /usr/local/mysql/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 1
1 row in set (0.05 sec)

GTID

GTID介紹
GTID（Global Transaction）全域性事務識別符號：是一個唯一的識別符號，它建立並與源伺服器（主）上提交的每個事務相關聯。此識別符號不僅對其發起的伺服器是唯一的，而且在給定複製設定中的所有伺服器上都是唯一的。所有交易和所有GTID之間都有1對1的對映。
GTID實際上是由UUID+TID組成的。其中UUID是一個MySQL例項的唯一標識。TID代表了該例項上已經提交的事務數量，並且隨著事務提交單調遞增。
這是一個GTID的具體形式：3E11FA47-71CA-11E1-9E33-C80AA9429562:23
GTID的新特性
（1）支援多執行緒複製：事實上是針對每個database開啟相應的獨立執行緒，即每個庫有一個單獨的（sql thread）
（2）支援啟用GTID，在配置主從複製，傳統的方式裡，你需要找到binlog和POS點，然後change master to 指向。在mysql5.6裡，無須再知道binlog和POS點，只需要知道master的IP/埠/賬號密碼即可，因為同步複製是自動的，MySQL通過內部機制GTID自動找點同步。
（3）基於Row複製只儲存改變的列，大大節省磁碟空間，網路，記憶體等
（4）支援把Master和Slave的相關資訊記錄在Table中；原來是記錄在檔案裡，現在則記錄在表裡，增強可用性
（5）支援延遲複製
開啟方法
#mysql配置檔案：
[mysqld]
gtid_mode=ON
enforce_gtid_consistency

#檢視
show global variables like ‘%gtid%’；

配置從庫，禁止自己刪除MySQL中繼日誌
方法一：臨時禁止
#登陸從庫
[[email protected] ~]# mysql -uroot -p123456

#臨時禁用自動刪除relay log功能
mysql> set global relay_log_purge = 0;
Query OK, 0 rows affected (0.00 sec)

#設定只讀
mysql> set global read_only=1;
Query OK, 0 rows affected (0.00 sec)

方法二：永久禁止
編輯從庫配置檔案/etc/my.cnf

[[email protected] ~]# vim /etc/my.cnf
#在[mysqld]模組下加入一行，永久禁止自動刪除relay log功能
relay_log_purge = 0
[[email protected] ~]# /etc/init.d/mysqld restart
Shutting down MySQL.. SUCCESS!
Starting MySQL.. SUCCESS!

部署MHA

環境準備（所有節點mysql-master,mysql-slaveA,mysql-slaveB）

#光碟安裝依賴包
[[email protected] ~]# yum -y install perl-DBD-MySQL
#安裝mha4mysql-node-0.56-0.el6.noarch.rpm
[[email protected] ~]# rpm -ivh mha4mysql-node-0.56-0.el6.noarch.rpm
Preparing... ########################################### [100%]
1:mha4mysql-node ########################################### [100%]

庫上建立mha管理賬號

[[email protected] ~]# mysql -uroot -p123456
mysql> grant all privileges on *.* to [email protected]'192.168.239.%' identified by '123456';
Query OK, 0 rows affected (0.10 sec)

mysql> select user,host from mysql.user where user='mha';
+------+---------------+
| user | host |
+------+---------------+
| mha | 192.168.239.% |
+------+---------------+
1 row in set (0.13 sec)
#主庫上建立從庫會自動複製

1
從庫上建立主從複製使用者（MySQL-slaveA和MySQL-slaveB）

[[email protected] ~]# mysql -uroot -p123456

mysql> grant replication slave on *.* to [email protected]'192.168.239.%' identified by '123456';
Query OK, 0 rows affected (0.01 sec)

mysql> exit
Bye

部署管理節點（mha-manager）

在mysql-slaveB上部署管理節點
#使用阿里雲源+epel源
[[email protected] ~]# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
[[email protected] ~]# wget -O /etc/yum.repos.d/epel-6.repo http://mirrors.aliyun.com/repo/epel-6.repo

#安裝manager依賴包（需要公網源）
[[email protected] ~]# yum -y install perl-Config-Tiny epel-release perl-Log-Dispatch perl-Parallel-ForkManager perl-Time-HiRes

#安裝manager包
[[email protected] ~]# rpm -ivh mha4mysql-manager-0.56-0.el6.noarch.rpm
Preparing... ########################################### [100%]
1:mha4mysql-manager ########################################### [100%]

編輯配置檔案
#建立配置檔案目錄
[[email protected] ~]# mkdir -p /etc/mha
#建立日誌目錄
[[email protected] ~]# mkdir -p /var/log/mha/mha1 #如果有多組主從複製架構可以延mha2、mha3...
#建立配置檔案（預設沒有）
[[email protected] ~]# cd /etc/mha/
[[email protected] mha]# ls
[[email protected] mha]# vim /etc/mha/mha1.cnf
[server default]
manager_log=/var/log/mha/mha1/manager #manager管理日誌存放路徑
manager_workdir=/var/log/mha/mha1 #manager管理日誌的目錄路徑
master_binlog_dir=/usr/local/mysql/data #binlog日誌的存放路徑
user=mha #管理賬戶
password=123456 #管理賬戶密碼
ping_interval=2 #存活檢查的間隔時間
repl_user=rep #主從複製的授權賬戶
repl_password=123456 #主從複製的授權賬戶密碼
ssh_user=root #用於ssh連線的賬戶

[server1]
hostname=192.168.239.128
port=3306

[server2]
#candidate_master=1 #此條暫時註釋掉，設定為候選master，如果設定該引數以後，發生主從切換以後會將此從庫提升為主庫，即使這個主庫不是叢集中事件最新的slave
#check_repl_delay=0 #此條暫時註釋掉，預設情況下如果一個slave落後master 100M的relay logs 的話，MHA將不會選擇該slave作為一個新的master，因為對於這個slave的恢復需要花費很長時間，通過設定check_repl_delay=0,MHA觸發切換在選擇一個新的master的時候將會忽略複製延時，這個引數對於設定了candidate_master=1的主機非常有用，因為這個候選主在切換的過程中一定是新的master
hostname=192.168.239.129
port=3306

[server3]
hostname=192.168.239.130
port=3306

配置ssh信任（所有節點mysql-master,mysql-slaveA,mysql-slaveB）
#建立金鑰對
[[email protected] mha]# ssh-keygen -t dsa -P "" -f ~/.ssh/id_dsa
Generating public/private dsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
6d:1c:e1:9f:35:ff:6e:5d:69:ca:08:69:79:9f:f9:80 [email protected]
The key's randomart image is:
+--[ DSA 1024]----+
| . |
| . . |
| o o |
| o o o o |
| S * o o|
| * .. oo|
| . oE+.= +|
| . *..o|
| .o.|
+-----------------+
#傳送mysql-master公鑰，包括自己
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]

#傳送mysql-slaveA公鑰，包括自己
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]

#傳送mysql-slaveB公鑰，包括自己
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]
[[email protected] ~]# ssh-copy-id -i /root/.ssh/id_dsa.pub [email protected]

啟動測試
（1）ssh檢查檢測（在mha-master這臺伺服器上進行檢查）
[[email protected] mha]# which masterha_check_ssh
/usr/bin/masterha_check_ssh
[[email protected] mha]# masterha_check_ssh --conf=/etc/mha/mha1.cnf #ssh檢查命令
Thu Dec 6 17:26:01 2018 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Dec 6 17:26:01 2018 - [info] Reading application default configuration from /etc/mha/mha1.cnf..
Thu Dec 6 17:26:01 2018 - [info] Reading server configuration from /etc/mha/mha1.cnf..
Thu Dec 6 17:26:01 2018 - [info] Starting SSH connection tests..
...
Thu Dec 6 17:26:02 2018 - [debug] Connecting via SSH from [email protected](192.168.239.130:22) to [email protected](192.168.239.128:22)..
Thu Dec 6 17:26:03 2018 - [debug] ok.
Thu Dec 6 17:26:03 2018 - [debug] Connecting via SSH from [email protected](192.168.239.130:22) to [email protected](192.168.239.129:22)..
Thu Dec 6 17:26:04 2018 - [debug] ok.
Thu Dec 6 17:26:04 2018 - [info] All SSH connection tests passed successfully. #出現這個就表示成功

（2）主從複製檢測

[[email protected] mha]# masterha_check_repl --conf=/etc/mha/mha1.cnf
Thu Dec 6 17:36:23 2018 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
...
Thu Dec 6 17:36:24 2018 - [info] Checking replication health on 192.168.239.129..
Thu Dec 6 17:36:24 2018 - [info] ok.
Thu Dec 6 17:36:24 2018 - [info] Checking replication health on 192.168.239.130..
Thu Dec 6 17:36:24 2018 - [info] ok.
Thu Dec 6 17:36:24 2018 - [warning] master_ip_failover_script is not defined.
Thu Dec 6 17:36:24 2018 - [warning] shutdown_script is not defined.
Thu Dec 6 17:36:24 2018 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.

啟動MHA
#啟動
[[email protected] mha]# nohup masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/mha1/manager.log 2>&1 &
[1] 4090
[[email protected] mha]# ps -ef | grep perl | grep -v grep
root 4090 1631 0 17:57 pts/1 00:00:00 perl /usr/bin/masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover

#說明：
nohup:啟動命令
--conf：指定配置檔案位置
--remove_dead_master_conf：如果有master down了，就去掉配置檔案裡該master的部分。

進行mha自動切換master的測試

初始狀態：

（1）登陸mysql-slaveA(192.168.239.129)檢視資訊狀態

[[email protected] ~]# mysql -uroot -p123456
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.239.128 #這是主庫IP地址
Master_User: rep
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000003
Read_Master_Log_Pos: 405
Relay_Log_File: localhost-relay-bin.000004
Relay_Log_Pos: 615
Relay_Master_Log_File: mysql-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:

（2）停掉mysql-master(192.168.239.128)上的MySQL服務

[[email protected] ~]# /etc/init.d/mysqld stop
Shutting down MySQL...... SUCCESS!

（3）檢視mysql-slaveB上的MySQL從庫同步狀態

[[email protected] mha]# mysql -uroot -p123456 -e 'show slave status\G'
Warning: Using a password on the command line interface can be insecure.
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.239.129 #現在的主庫IP地址
Master_User: rep
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000003 #binlog日誌
Read_Master_Log_Pos: 662 #binlog日誌位置
Relay_Log_File: localhost-relay-bin.000002
Relay_Log_Pos: 665
Relay_Master_Log_File: mysql-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:

（4）檢視mysql-slaveA上的MySQL，主庫同步狀態。

[[email protected] ~]# mysql -uroot -p123456 -e 'show master status'
Warning: Using a password on the command line interface can be insecure.
+------------------+----------+--------------+------------------+--------------------------------------------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+--------------------------------------------------------------------------------+
| mysql-bin.000003 | 662 | | | 9a7e8455-f90b-11e8-b581-000c29e4bd28:1,
9c940ddb-f90b-11e8-b581-000c29ca6c91:1 |
+------------------+----------+--------------+------------------+--------------------------------------------------------------------------------+

（5）檢視mysql-slaveB上的mha程序狀態

[[email protected] mha]# ps -ef | grep perl | grep -v grep #查詢發現mha程序已經沒了
[[email protected] mha]#

6）檢視mha配置檔案資訊

[[email protected] mha]# cat /etc/mha/mha1.cnf
[server default]
manager_log=/var/log/mha/mha1/manager
manager_workdir=/var/log/mha/mha1
master_binlog_dir=/usr/local/mysql/data
password=123456
ping_interval=2
repl_password=123456
repl_user=rep
ssh_user=root
user=mha

[server2]
hostname=192.168.239.129
port=3306

[server3]
hostname=192.168.239.130
port=3306

#此時[server1]模組的資訊已經消失了，當作為主庫的mysql-master上的MySQL宕機以後，mha通過檢測發現mysql-master宕機，那麼會將binlog日誌最全的從庫立刻提升為主庫，而其他的從庫會指向新的主庫進行再次同步。

進行mha的故障還原測試
mha的程序只能完成1次切換，不能持續切換（完成一次後，後臺程序就當掉了，需要重新手動啟動）
由於mysql-master的MySQL服務宕機，因此mha將mysql-slaveA提升為了主庫。因此，我們需要將宕機的mysql-master的MySQL服務啟動，然後作為主庫mysql-slaveA的從庫。

初始狀態：

（1）將故障宕機的mysql-master的MySQL服務啟動並授權進行從同步

[[email protected] mha]# cat /var/log/mha/mha1/manager
#找到這句話，複製到文字中
CHANGE MASTER TO MASTER_HOST='192.168.239.129', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='rep', MASTER_PASSWORD='xxx';

[[email protected] ~]# mysql -uroot -p123456

mysql> CHANGE MASTER TO MASTER_HOST='192.168.239.129', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='rep', MASTER_PASSWORD='123456'; #複製/var/log/mha/mha1/manager日誌中的這句話，只需修改密碼
Query OK, 0 rows affected, 2 warnings (0.18 sec)

mysql> start slave;
Query OK, 0 rows affected, 1 warning (0.43 sec)

mysql> show slave status\G #檢視從同步狀態
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.239.129
Master_User: rep
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000003
Read_Master_Log_Pos: 662
Relay_Log_File: localhost-relay-bin.000002
Relay_Log_Pos: 665
Relay_Master_Log_File: mysql-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:

（2）將mha配置檔案裡缺失的部分補全（mysql-slaveB伺服器）

[[email protected] mha]# vim /etc/mha/mha1.cnf

[server default]
manager_log=/var/log/mha/mha1/manager
manager_workdir=/var/log/mha/mha1
master_binlog_dir=/usr/local/mysql/data
password=123456
ping_interval=2
repl_password=123456
repl_user=rep
[server default]
manager_log=/var/log/mha/mha1/manager
manager_workdir=/var/log/mha/mha1
master_binlog_dir=/usr/local/mysql/data
password=123456
ping_interval=2
repl_password=123456
repl_user=rep
ssh_user=root
user=mha

[server1] #新增此模組
hostname=192.168.239.128
port=3306

[server2]
hostname=192.168.239.129
port=3306

[server3]
hostname=192.168.239.130
port=3306
[[email protected] mha]# pkill perl
[[email protected] mha]# ps -ef | grep mha
root 8641 1631 1 21:02 pts/1 00:00:00 grep mha
[1]+ Exit 1 nohup masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/mha1/manager.log 2>&1
[[email protected] mha]# ps -ef | grep mha
root 8643 1631 0 21:02 pts/1 00:00:00 grep mha

（3）啟動mha程序

[[email protected] mha]# nohup masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/mha1/manager.log 2>&1 &
[1] 5215
[[email protected] mha]# ps -ef | grep perl | grep -v grep
root 5215 1631 3 19:14 pts/1 00:00:00 perl /usr/bin/masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover

1
2
3
4
5
（4）停掉mysql-slaveA上的MySQL服務

[[email protected] ~]# /etc/init.d/mysqld stop
Shutting down MySQL....... SUCCESS!

1
2
3
（5）檢視mysql-slaveB上的主從同步狀態：

[[email protected] mha]# mysql -uroot -p123456 -e 'show slave status\G'
Warning: Using a password on the command line interface can be insecure.
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.239.128 #此時主庫為mysql-master
Master_User: rep
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 448
Relay_Log_File: localhost-relay-bin.000002
Relay_Log_Pos: 408
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:

（6）啟動mysql-slaveA上的MySQL服務

[[email protected] ~]# /etc/init.d/mysqld start
Starting MySQL.................. SUCCESS!
[[email protected] ~]# mysql -uroot -p123456

mysql> CHANGE MASTER TO MASTER_HOST='192.168.239.128', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='rep', MASTER_PASSWORD='123456'; #這條資訊到mysql-slaveB中的/var/log/mha/mha1/manager中查詢
Query OK, 0 rows affected, 2 warnings (0.07 sec)

mysql> start slave;
Query OK, 0 rows affected, 1 warning (0.15 sec)

（7）再次補全mha配置檔案後，啟動mha程序

[[email protected] mha]# vim /etc/mha/mha1.cnf

[server default]
manager_log=/var/log/mha/mha1/manager
manager_workdir=/var/log/mha/mha1
master_binlog_dir=/usr/local/mysql/data
password=123456
ping_interval=2
repl_password=123456
repl_user=rep
ssh_user=root
[server default]
manager_log=/var/log/mha/mha1/manager
manager_workdir=/var/log/mha/mha1
master_binlog_dir=/usr/local/mysql/data
password=123456
ping_interval=2
repl_password=123456
repl_user=rep
ssh_user=root
user=mha

[server1]
hostname=192.168.239.128
port=3306

[server2] #新增此模組
hostname=192.168.239.129
port=3306

[server3]
hostname=192.168.239.130
port=3306
[[email protected] mha]# nohup masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/mha1/manager.log 2>&1 &
[1] 8730
[[email protected] mha]# ps -ef | grep perl | grep -v grep
root 8730 1631 1 21:19 pts/1 00:00:00 perl /usr/bin/masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover

此時的初始狀態還原為下圖：

原始碼安裝mha的方法

node節點的原始碼安裝方法：

[[email protected] ~]# yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Params-Validate perl-CPAN perl-devel perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker

[[email protected] ~]# tar xf mha4mysql-node-0.56.tar.gz -C /usr/src/

[[email protected] ~]# cd /usr/src/mha4mysql-node-0.56/

[[email protected] mha4mysql-node-0.56]# perl Makefile.PL

[[email protected] mha4mysql-node-0.56]# make && make install

manager節點的原始碼安裝方法：

[[email protected] ~]# yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Params-Validate perl-CPAN perl-devel perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker

[[email protected] ~]# tar xf mha4mysql-manager-0.56.tar.gz -C /usr/src/

[[email protected] ~]# cd /usr/src/mha4mysql-manager-0.56/

[[email protected] mha4mysql-manager-0.56]# perl Makefile.PL

[[email protected] mha4mysql-manager-0.56]# make && make install

配置VIP漂移

IP漂移的兩種方式
（1）通過keepalived的方式，管理虛擬IP的漂移
（2）通過MHA自帶指令碼方式，管理虛擬IP的漂移
MHA指令碼管理方式
（1）獲取管理指令碼master_ip_failover
[[email protected] ~]# ls
anaconda-ks.cfg mha4mysql-manager-0.56-0.el6.noarch.rpm
Atlas-2.2.1.el6.x86_64.rpm mha4mysql-manager-0.56.tar.gz
install.log mha4mysql-node-0.56-0.el6.noarch.rpm
install.log.syslog mha4mysql-node-0.56.tar.gz
master_ip_failover mysql-5.6.17-linux-glibc2.5-x86_64.tar.gz
mha4mysql-manager-0.56 yum.sh
[[email protected] scripts]# cp master_ip_failover /usr/local/bin/ #複製到/usr/local/bin目錄下
[[email protected] ~]# cd /usr/local/bin/
[[email protected] bin]# chmod +x master_ip_failover
[[email protected] bin]# which master_ip_failover
/usr/local/bin/master_ip_failover

（2）修改mha配置檔案

[[email protected] ~]# vim /etc/mha/mha1.cnf

[server default]
[server default]
manager_log=/var/log/mha/mha1/manager
manager_workdir=/var/log/mha/mha1
master_binlog_dir=/usr/local/mysql/data
master_ip_failover_script=/usr/local/bin/master_ip_failover #新增指令碼位置
password=123456
ping_interval=2
repl_password=123456
repl_user=rep
ssh_user=root
user=mha

#candidate_master=1 #留著備用的註釋項
#check_repl_delay=0 #留著備用的註釋項
[server1]
hostname=192.168.239.128
port=3306

[server2]
hostname=192.168.239.129
port=3306

[server3]
hostname=192.168.239.130
port=3306

（3）修改VIP漂移指令碼（master_ip_failover指令碼）

[[email protected]lhost ~]# vim /usr/local/bin/master_ip_failover
#!/usr/bin/env perl

use strict;
use warnings FATAL => 'all';

use Getopt::Long;

my (
$command, $ssh_user, $orig_master_host,
$orig_master_ip, $orig_master_port, $new_master_host,
$new_master_ip, $new_master_port
);

my $vip = '192.168.239.60/24'; #定義VIP
my $key = '0'; #定義網絡卡字尾數值，如果eth0就是0，eth1就是1
my $ssh_start_vip = "/sbin/ifconfig eth0:$key $vip"; #繫結VIP的命令
my $ssh_stop_vip = "/sbin/ifconfig eth0:$key down"; #關閉VIP的命令

GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);

exit &main();

sub main {
if ( $command eq "stop" || $command eq "stopssh" ) {
my $exit_code = 1;
eval {
print "Disabling the VIP on old master:$orig_master_host \n";
&stop_vip();
$exit_code = 0;
};
if ([email protected]) {
warn "Got Error: [email protected]\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ([email protected]) {
warn [email protected];
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script..OK \n";
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}

sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}

sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}

sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}

（4）重啟動mha管理端

[[email protected] ~]# pkill perl
[[email protected] ~]# nohup masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/mha1/manager.log 2>&1 &
[1] 10984

#提示：
如果啟動mha程序失敗，需要進行mha的連線檢測
masterha_check_ssh --conf=/etc/mha/mha1.cnf ssh連線檢測
masterha_check_repl --conf=/etc/mha/mha1.cnf 主從複製檢測

VIP漂移指令碼驗證測試
架構初始狀態：

（1）檢視mysql-master網路狀態
[[email protected] ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:E4:BD:28
inet addr:192.168.239.128 Bcast:192.168.239.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fee4:bd28/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:249193 errors:0 dropped:0 overruns:0 frame:0
TX packets:74203 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:334114738 (318.6 MiB) TX bytes:6310911 (6.0 MiB)

eth0:0 Link encap:Ethernet HWaddr 00:0C:29:E4:BD:28 #出現VIP
inet addr:192.168.239.60 Bcast:192.168.239.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:50 errors:0 dropped:0 overruns:0 frame:0
TX packets:50 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8386 (8.1 KiB) TX bytes:8386 (8.1 KiB)

（2）停掉mysql-master的MySQL資料庫服務

[[email protected] ~]# /etc/init.d/mysqld stop
Shutting down MySQL..... SUCCESS!
[[email protected] ~]# ifconfig #VIP消失了
eth0 Link encap:Ethernet HWaddr 00:0C:29:59:7C:EF
inet addr:192.168.239.128 Bcast:192.168.239.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe59:7cef/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:316130 errors:0 dropped:0 overruns:0 frame:0
TX packets:58058 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:363635227 (346.7 MiB) TX bytes:6462008 (6.1 MiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:69 errors:0 dropped:0 overruns:0 frame:0
TX packets:69 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:9752 (9.5 KiB) TX bytes:9752 (9.5 KiB)

（3）檢視mysql-slaveA

[[email protected] ~]# ifconfig #VIP出現了
eth0 Link encap:Ethernet HWaddr 00:0C:29:4C:D5:92
inet addr:192.168.239.129 Bcast:192.168.239.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe4c:d592/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:287225 errors:0 dropped:0 overruns:0 frame:0
TX packets:47133 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:361247254 (344.5 MiB) TX bytes:5159560 (4.9 MiB)

eth0:0 Link encap:Ethernet HWaddr 00:0C:29:4C:D5:92
inet addr:192.168.239.60 Bcast:192.168.239.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:45 errors:0 dropped:0 overruns:0 frame:0
TX packets:45 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:7718 (7.5 KiB) TX bytes:7718 (7.5 KiB)

（4）檢視mysql-slaveB的主從同步情況

[[email protected] ~]# mysql -uroot -p123456 -e 'show slave status\G'
Warning: Using a password on the command line interface can be insecure.
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.239.129 #mysql-slaveB的主庫已經切換為mysql-slaveA
Master_User: rep
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000008
Read_Master_Log_Pos: 271
Relay_Log_File: mysql-db03-relay-bin.000002
Relay_Log_Pos: 361
Relay_Master_Log_File: mysql-bin.000008
Slave_IO_Running: Yes
Slave_SQL_Running: Yes

（5）mysql-master故障恢復

[[email protected] ~]# /etc/init.d/mysqld start
Starting MySQL. SUCCESS!
[[email protected] ~]# mysql -uroot -p123456
mysql> CHANGE MASTER TO MASTER_HOST='192.168.239.129', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='rep', MASTER_PASSWORD='123456';
Query OK, 0 rows affected, 2 warnings (0.02 sec)

mysql> start slave;
Query OK, 0 rows affected, 1 warning (0.01 sec)

[[email protected] mha]# vim /etc/mha/mha1.cnf

[server default]
manager_log=/var/log/mha/mha1/manager
manager_workdir=/var/log/mha/mha1
master_binlog_dir=/usr/local/mysql/data
password=123456
ping_interval=2
repl_password=123456
repl_user=rep
ssh_user=root
[server default]
manager_log=/var/log/mha/mha1/manager
manager_workdir=/var/log/mha/mha1
master_binlog_dir=/usr/local/mysql/data
master_ip_failover_script=/usr/local/bin/master_ip_failover
password=123456
ping_interval=2
repl_password=123456
repl_user=rep
ssh_user=root
user=mha

[server1] #新增此模組
hostname=192.168.239.128
port=3306

[server2]
hostname=192.168.239.129
port=3306

[server3]
hostname=192.168.239.130
port=3306

（7）啟動mha管理程序

[[email protected] ~]# ps -ef | grep mha | grep -v grep
[[email protected] ~]# nohup masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/mha1/manager.log 2>&1 &
[1] 14921
[[email protected] ~]# ps -ef | grep mha | grep -v grep
root 14921 13211 3 20:11 pts/3 00:00:00 perl /usr/bin/masterha_manager --conf=/etc/mha/mha1.cnf --remove_dead

Linux系統——MHA-Atlas-MySQL高可用叢集

Linux系統——MHA-Atlas-MySQL高可用叢集

MHA-Atlas-MYSQL高可用叢集

MHA-Atlas-MySQL高可用叢集主機名對映

Linux實戰教學筆記40： Mha-Atlas-MySQL高可用方案實踐(二)

Mha-Atlas-MySQL高可用

（專案六）Mha-Atlas-MySQL高可用方案實踐

（項目六）Mha-Atlas-MySQL高可用方案實踐

Mha-Atlas-MySQL高可用方案實踐。

Mha-Atlas-MySQL高可用方案實踐

專案課---Mha-Atlas-MySQL高可用方案實踐（六）

MHA-Atlas-MySQL高可用上（6）

配置MySQL高可用叢集MHA

MySQL 高可用叢集架構 MHA 詳解

實現MHA，MySQL高可用集群架構

構建MHA實現MySQL高可用集群架構

使用MHA實現mysql高可用性（centos7.5+mysql5.7.23+MHA0.58）

搭建PXC叢集，實現MySQL高可用叢集

MHA 實現 Mysql 高可用配置

企業中MySQL高可用叢集架構三部曲之MM+keepalived

Corosync+Pacemaker+NFS+Mysql高可用叢集部署

Linux系統——MHA-Atlas-MySQL高可用叢集

相關推薦