整個MHA+keepalived+lvs+mysql高可用架構配置說明

阿新 • • 發佈：2019-01-11

dex diff vim could not 分區創建 pos ann 試驗 pts

整個MHA+keepalived+lvs+mysql高可用架構配置說明
1.1. 環境簡介
1.1.1、vmvare虛擬機，系統版本CentOS7.5 x86_64位最小化安裝，mysql的版本5.7.21，
1.1.2、虛擬機器的ssh端口均為默認22,
1.1.3、虛擬機的iptables全部關閉，
1.1.4、虛擬機的selinux全部關閉，
1.1.5、虛擬機服務器時間全部一致 ntpdate 0.asia.pool.ntp.org
1.1.6、3臺機器的ssh端口為22**
1.2、此次試驗采用的是3臺機器，機器具體部署如下：
角色 IP地址（內網）主機名稱節點機器部署服務業務用途

Monitor 192.168.52.250 db250 mha4mysql-manager-0.58-0.el6
Master 192.168.52.251 db251 mha4mysql-node-0.58-0.el6 寫入(keepalived)

slave(備master) 192.168.52.252 db252 mha4mysql-node-0.58-0.el6 keepalived
Slave 192.168.52.253 db253 mha4mysql-node-0.58-0.el6

讀+備份數據
Lvs01 192.168.52.248 lvs248 lvs+keepalived
從庫負載均衡
Lvs02 192.168.52.249 lvs249 lvs+keepavlied
從庫負載均衡
Mysql write vip： 192.168.52.199
Mysql read vip： 192.168.52.159
1.3 系統架構圖：

故障後架構圖

1.4 架構實現原理：
1．讀操作
1 ） LVS 實現讀操作的負載均衡；
2） Keepalived 在上層管理 LVS，並對兩臺從庫進行健康檢測（通過定義 Ch

eck 腳本）；
3）一臺從庫出現故障後， Keepalived 將其剔除出負載均衡集群；

寫操作
1 ）在 Master 上綁定寫 VIP（MHA 啟動後會通過腳本進行操作）；
2） MHA 監控 Master 狀態，當 Master 出現故障後（宕機、復制暫停）時;
3）通過 Failover 腳本，卸載 Master 上的 WVIP；
4）通過 Failover 腳本在 Backup Master 上綁定 WVIP，提升其為主庫；
5）同步並應用差異日誌，並將從庫指向新主庫；
問題：當 MHA 把 Master 切換到了 Backup Master 上後， LVS 如何處理分發在 Ba
ckup Master 上的讀操作？
解釋：由於 Keepalived 會通過腳本定期監控 Backup Master 的狀態，包括同步、
SQL 線程、 I/O 線程，所以當 Backup Master 升級為主庫後，這些狀態都將消失，
Keepalived 將自動將 Backup Master 剔除出負載均衡集群。
1.5 說明介紹：
Db252和db253是db251的slave從庫，復制環境搭建後面會簡單演示,其中master對外提供寫服務，備選master（實際的slave，主機名server03）提供讀服務，slave也提供相關的讀服務，一旦master宕機，將會把備
選備master提升為新的master，slave指向新的master,db250上部署Monitor（MHA Manager監控），主要是監控主從復制的集群中主庫master是否正常，一旦master掛掉，MHA Manager會自動完成主庫和slave從庫的自動切換
1.6 安裝mysql
Mater和slave都需要安裝
wget http://dev.mysql.com/get/mysql57-community-release-el7-8.noarch.rpm
yum localinstall mysql57-community-release-el7-8.noarch.rpm
yum install mysql-community-server -y
systemctl enable mysqld
systemctl daemon-reload
systemctl start mysqld
systemctl status mysqld
grep ‘temporary password‘ /var/log/mysqld.log
mysql -uroot -p
[root@server04 ~]# mysql -u root
ERROR 2002 (HY000): Can‘t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock‘ (2)

service mysqld restart
192.168.52.251
vim /etc/my.cnf

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
character-set-server=utf8
server-id=1
log-bin=master-log
relay-log=relay-log
innodb_file_per_table = ON
skip_name_resolve = ON
max_connections = 5000#不在配置文件中限定只讀，但是要記得在slave上限制只讀

symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
192.168.52.252
vim /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
character-set-server=utf8
server-id=2
log-bin=master-log
relay-log=relay-log
relay_log_purge=0
read_only=1
skip_name_resolve=1
innodb_file_per_table=1
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
192.168.52.253
vim /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
character-set-server=utf8

server-id=3
log-bin=master-log
relay-log=relay-log
relay_log_purge=0
read_only=1
skip_name_resolve=1
innodb_file_per_table=1
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

第二：新建用戶repl_user設置密碼123456
重置root密碼：123456

#更改密碼
mysql -u root
#更改密碼策略
set global validate_password_policy=0;
set global validate_password_length=4;
SET PASSWORD = PASSWORD(‘123456‘);
GRANT ALL PRIVILEGES ON . TO ‘root‘@‘192.168.52.%‘ IDENTIFIED BY ‘123456‘ WITH GRANT OPTION;
flush privileges;
主從復制授權：
第三：在 3 個 mysql 節點做授權配置(主從復制授權)
Master:
grant replication slave,replication client on . to ‘repluser‘@‘192.168.52.%‘ identified by ‘123456‘ ;
flush privileges;
#授權MHA管理用戶-mhaadmin
grant all on . to ‘mhaadmin‘@‘192.168.52.%‘ identified by ‘mhapass‘ ;
flush privileges;
mysql> show master status;
+-------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+-------------------+----------+--------------+------------------+-------------------+
| master-log.000005 | 154 | | | |
+-------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
slave（兩個從庫）
#配置主從復制起點
change master to master_host=‘192.168.52.251‘,master_user=‘repluser‘,master_password=‘123456‘,master_log_file=‘master-log.000005‘,master_log_pos=154;
start slave;
show slave status\G
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
mysql> set global read_only=1; #查看slave IO和slave sql是否都正常
#查看主從復制情況
show grants for ‘repluser‘@‘192.168.52.%‘;
mysql> flush privileges; #刷新權限
Query OK, 0 rows affected (0.00 sec)
刪除多余用戶
mysql> drop user root@‘localhost‘;
mysql> select user,host from mysql.user;

1.7 ssh授信
配置三臺機器的ssh互信（三臺都要操作）
ssh-keygen -t rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]
ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]
ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]
ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]

#測試是否成功
ssh 192.168.52.251 date
1.8 安裝MHA軟件
安裝MHA軟件（在三個節點上都裝mha的node軟件）
#先安裝依賴
wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -ivh epel-release-latest-7.noarch.rpm
yum install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager　-y
下載軟件
wget https://qiniu.wsfnk.com/mha4mysql-node-0.58-0.el7.centos.noarch.rpm
rpm -ivh mha4mysql-node-0.58-0.el7.centos.noarch.rpm

僅在manager節點上安裝mha管理軟件(192.168.52.250)
wget https://qiniu.wsfnk.com/mha4mysql-manager-0.58-0.el7.centos.noarch.rpm
yum install perl-Parallel-ForkManager -y
rpm -ivh mha4mysql-manager-0.58-0.el7.centos.noarch.rpm

yum install mailx -y    #該軟件是用來發送郵件的

-rwxr-xr-x 1 root root 5172 Jan 7 14:09 masterha_secondary_check
-rwxr-xr-x 1 root root 1739 Jan 7 14:09 masterha_stop
-rwxr-xr-x 1 root root 8337 Jan 7 14:14 purge_relay_logs
-rwxr-xr-x 1 root root 7525 Jan 7 14:14 save_binary_logs
1.9 配置MHA軟件
192.168.52.250機器操作：

[root@db250 bin]# cd /usr/bin/
[root@db250 bin]# find ./ -name apply_diff_relay_logs
./apply_diff_relay_logs
[root@db250 bin]# cp /usr/bin/save_binary_logs /usr/local/bin/
[root@db250 bin]# cp /usr/bin/purge_relay_logs /usr/local/bin/
[root@db250 bin]# cp /usr/bin/filter_mysqlbinlog /usr/local/bin/
[root@db250 bin]# cp /usr/bin/apply_diff_relaylogs /usr/local/bin/
[root@db250 bin]# ln -s /usr/local/mysql/bin/mysql /usr/bin/mysql
[root@db250 bin]# ln -s /usr/local/mysql/bin/mysqlbinlog /usr/bin/mysqlbinlog
[root@db250 bin]# cp -r /usr/bin/masterha* /usr/local/bin
[root@db250 bin]# cd /usr/local/bin/
[root@db250 bin]# ll
total 88
-rwxr-xr-x 1 root root 17639 Jan 7 14:14 apply_diff_relay_logs
-rwxr-xr-x 1 root root 4807 Jan 7 14:14 filter_mysqlbinlog
-rwxr-xr-x 1 root root 1995 Jan 7 14:09 masterha_check_repl
-rwxr-xr-x 1 root root 1779 Jan 7 14:09 masterha_check_ssh
-rwxr-xr-x 1 root root 1865 Jan 7 14:09 masterha_check_status
-rwxr-xr-x 1 root root 3201 Jan 7 14:09 masterha_conf_host
-rwxr-xr-x 1 root root 2517 Jan 7 14:09 masterha_manager
-rwxr-xr-x 1 root root 2165 Jan 7 14:09 masterha_master_monitor
-rwxr-xr-x 1 root root 2373 Jan 7 14:09 masterha_master_switch

MHA配置文件如下
mkdir -p /etc/masterha
[root@db250 app1]# cat /etc/masterha/app1.cnf
[server default]
manager_log=/var/log/masterha/app1/manager.log
manager_workdir=/var/log/masterha/app1.log
master_binlog_dir=/var/lib/mysql
master_ip_failover_script=/usr/local/bin/master_ip_failover
master_ip_online_change_script=/usr/local/bin/master_ip_online_change
password=123456
ping_interval=1
remote_workdir=/tmp
repl_password=123456
repl_user=repluser
secondary_check_script=/usr/local/bin/masterha_secondary_check -s db251 -s db252 --user=root --master_host=db252 --master_ip=192.168.52.252 --master_port=3306
shutdown_script=""
ssh_port=22
ssh_user=root

[server1]
hostname=192.168.52.251
candidate_master=1
port=3306

[server2]
candidate_master=1
check_repl_delay=0
hostname=192.168.52.252
port=3306

[server3]
hostname=192.168.52.253
port=3306

二、設置relay log的清除方式（在每個slave節點上）：
三個節點服務器本地hosts解析
[root@db250 app1]# vim /etc/hosts
192.168.52.250 db250
192.168.52.251 db251
192.168.52.252 db252
192.168.52.253 db253

在slave master01 192.168.52.252操作：
[root@db251 ~]# mysql -uroot -p123456 -e "set global relay_log_purge=0"
mysql: [Warning] Using a password on the command line interface can be insecure.

在slave master02 192.168.52.253操作：
[root@ db252 ~]# mysql -uroot -p123456 -e "set global relay_log_purge=0"
mysql: [Warning] Using a password on the command line interface can be insecure.

註意：
MHA在發生切換的過程中，從庫的恢復過程中依賴於relay log的相關信息，所以這裏要將relay log的自動清除設置為OFF，采用手動清除relay log的方式。在默認情況下，從服務器上的中繼日誌會在SQL線程執行完畢後被自動刪除。但是在MHA環境中，這些中繼日誌在恢復其他從服務器時可能會被用到，因此需要禁用中繼日誌的自動刪除功能。定期清除中繼日誌需要考慮到復制延時的問題。在ext3的文件系統下，刪除大的文件需要一定的時間，會導致嚴重的復制延時。為了避免復制延時，需要暫時為中繼日誌創建硬鏈接，因為在linux系統中通過硬鏈接刪除大文件速度會很快。（在mysql數據庫中，刪除大表時，通常也采用建立硬鏈接的方式）

2.2設置定期清理relay腳本（兩臺slave服務器）：
[root@ db252~]# cat /data/scripts/purge_relay_log.sh
#!/bin/bash
user=root
passwd=123456
port=3306
log_dir=‘/data/masterha/log‘
work_dir=‘/data‘
purge=‘/usr/local/bin/purge_relay_logs‘

if [ ! -d $log_dir ]
then
mkdir $log_dir -p
fi

$purge --user=$user --password=$passwd --disable_relay_log_purge --port=$port --workdir=$work_dir >> $log_dir/purge_relay_logs.log 2>

腳本介紹：
--user mysql //用戶名
--password mysql //密碼
--port //端口號
--workdir //指定創建relay log的硬鏈接的位置，默認是/var/tmp，由於系統不同分區創建硬鏈接文件會失敗，故需要執行硬鏈接具體位置，成功執行腳本後，硬鏈接的中繼日誌文件被刪除
--disable_relay_log_purge //默認情況下，如果relay_log_purge=1，腳本會什麽都不清理，自動退出，通過設定這個參數，當relay_log_purge=1的情況下會將relay_log_purge設置為0。清理relay log之後，最後將參數設置為OFF。

purge_relay_logs腳本刪除中繼日誌不會阻塞SQL線程。下面我們手動執行看看什麽情況:
[root@ db252 ~]# purge_relay_logs --user=root --password=123456 --port=3306 -disable_relay_log_purge --workdir=/data/
2018-07-01 11:53:16: purge_relay_logs script started.
Found relay_log.info: /data/mysql/relay-log.info
Opening /data/mysql/logs/relay-log/relay-bin.000001 ..
Opening /data/mysql/logs/relay-log/relay-bin.000002 ..
Executing SET GLOBAL relay_log_purge=1; FLUSH LOGS; sleeping a few seconds so that SQL thread can delete older relay log files (if it keeps up); SET GLOBAL relay_log_purge=0; .. ok.
2018-07-01 11:53:20: All relay log purging operations succeeded.

主從failover腳本
[root@db250 app1]# cat /usr/local/bin/master_ip_failover
#!/usr/bin/env perl

#This program is free software; you can redistribute it and/or modify
#t under the terms of the GNU General Public License as published by
#the Free Software Foundation; either version 2 of the License, or
#(at your option) any later version.

#This program is distributed in the hope that it will be useful,
#but WITHOUT ANY WARRANTY; without even the implied warranty of
#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#GNU General Public License for more details.
#You should have received a copy of the GNU General Public License
#along with this program; if not, write to the Free Software
#Foundation, Inc.,
#51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

##Note: This is a sample script and is not complete. Modify the script based on your environment.
######################################################
use strict;
use warnings FATAL => ‘all‘;

use Getopt::Long;
use MHA::DBHelper;

my (
$command, $ssh_user, $orig_master_host,
$orig_master_ip, $orig_master_port, $new_master_host,
$new_master_ip, $new_master_port, $new_master_user,
$new_master_password
);
my $vip = ‘192.168.52.199‘;
my $ssh_start_vip = "systemctl start keepalived";
my $ssh_stop_vip = "systemctl stop keepalived ";

GetOptions(
‘command=s‘ => \$command,
‘ssh_user=s‘ => \$ssh_user,
‘orig_master_host=s‘ => \$orig_master_host,
‘orig_master_ip=s‘ => \$orig_master_ip,
‘orig_master_port=i‘ => \$orig_master_port,
‘new_master_host=s‘ => \$new_master_host,
‘new_master_ip=s‘ => \$new_master_ip,
‘new_master_port=i‘ => \$new_master_port,
‘new_master_user=s‘ => \$new_master_user,
‘new_master_password=s‘ => \$new_master_password,
);

exit &main();

sub main {

print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";

if ( $command eq "stop" || $command eq "stopssh" ) {

#$orig_master_host, $orig_master_ip, $orig_master_port are passed.
#If you manage master ip address at global catalog database,
#invalidate orig_master_ip here.
my $exit_code = 1;
eval {
  print "Disabling the VIP on old master: $orig_master_host \n";
  &stop_vip();
  #updating global catalog, etc
  $exit_code = 0;
};
if ($@) {
  warn "Got Error: $@\n";
  exit $exit_code;
}
exit $exit_code;

}
elsif ( $command eq "start" ) {

#all arguments are passed.
#If you manage master ip address at global catalog database,
#activate new_master_ip here.
#You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
  print "Enabling the VIP - $vip on the new master - $new_master_host \n";
  &start_vip();
  $exit_code = 0;
    };
if ($@) {
  warn $@;

  #If you want to continue failover, exit 10.
  exit $exit_code;
}
exit $exit_code;

}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
#do nothing
exit 0;
}
else {
&usage();
exit 1;
}
}

sub start_vip() {
ssh $ssh_user\@$new_master_host \" $ssh_start_vip \";
}

#A simple system call that disable the VIP on the old_master
sub stop_vip() {
ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \";
}

sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}

chmod +x /usr/local/bin/master_ip_failover

1.10 MHA相關測試
測試SSH免密碼登錄
masterha_check_ssh --conf=/etc/masterha/app1.cnf

測試MHA數據庫同步
masterha_check_repl --conf=/etc/masterha/app1.cnf
chmod +x /usr/local/bin/masterha_check_repl

查看MHA Manager監控是否正常：
masterha_check_status --conf=/etc/masterha/app1.cnf

關閉MHA Manage監控
masterha_stop --conf=/etc/masterha/app1.cnf

1.11 啟動mha數據監控
masterha_check_status --conf=/etc/masterha/app1.cnf
mkdir -p /var/log/masterha/app1/

nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/masterha/app1/manager.log 2>&1 &
ps -ef|grep perl

1.12 配置vip實現MHA架構中主庫故障自動切換
192.168.52.251和192.168.52.252
Yum install keepalived -y

192.168.52.251
[root@db251 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {[email protected]
br/>[email protected]
notification_email_from [email protected]
smtp_server 192.168.52.251
smtp_connect_timeout 30
router_id LVS_01
}

vrrp_instance VI_1 {
#state MASTER
state BACKUP
interface ens33
virtual_router_id 51
priority 100
advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.2.199/24
}
}

systemctl start keepalived

192.168.52.252
[root@db252 ~]# vim /etc/keepalived/keepalived.conf

global_defs {
notification_email {[email protected]
br/>[email protected]
notification_email_from [email protected]
smtp_server 192.168.52.252
smtp_connect_timeout 30
router_id LVS_01
}**

vrrp_instance VI_1 {
#state MASTER
state BACKUP
interface ens33
virtual_router_id 51
priority 90
advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.52.199/24
}
}

systemctl start keepalived

#####特別註意！！！！！
上面兩臺服務器的keepalived都設置為了BACKUP模式，在keepalived中2種模式，分別是master->backup模式和backup->backup模式。這兩種模式有很大區別。在master->backup模式下，一旦主庫宕機，虛擬ip會自動漂移到從庫，當主庫修復後，keepalived啟動後，還會把虛擬ip搶占過來，即使設置了非搶占模式（nopreempt）搶占ip的動作也會發生。在backup->backup模式下，當主庫宕機後虛擬ip會自動漂移到從庫上，當原主庫恢復和keepalived服務啟動後，並不會搶占新主的虛擬ip，即使是優先級高於從庫的優先級別，也不會發生搶占。為了減少ip漂移次數，通常是把修復好的主庫當做新的備庫。
++++到此處MHA架構中keepalived服務安裝配置完成++++

1.13檢查故障切換後MHA集群相關服務的變化
配置文件/etc/masterha/app1.cnf變化
1、在db250 192.168.52.250管理節點查看一下配置文件/etc/masterha/app1.cnf可以發現[server1]的內容已經被自動去掉了：
2、masterha_manager 服務自動退出
3、源master192.168.52.251機器上keepalived服務被停掉了
4、查看日誌/var/log/masterha/app1/manager.log

#######重要提示！！！
當db251 192.168.52.251 機器上的mysql掛掉後，db252 192.168.52.252機器提升為master時，192.168.52.251 機器上的keepalived會停掉，而 192.168.52.252機器的keepalived會開啟，VIP票到199機器上。
此時需要重啟192.168.52.251上的mysql,一般都是要恢復它作為252新主的從庫，此時192.168.52.251機器上的keepalived千萬不要開啟，因為開啟keepalived，會搶占252機器上的VIP,導致程序連接數據庫出現混亂。同時192.168.52.251機器和192.168.52.252機器上的keepalived服務不要設置為開機自啟動
1.14 mha高可用重新加入監控
1、出問題的master:
show master status;

Last_IO_Error: Fatal error: The slave I/O thread stops because master and slave have equal MySQL server UUIDs;
these UUIDs must be different for replication to work.

2、解決方法：
rm -rf /var/lib/mysql/auto.cnf
systemctl restart mysqld

3、數據庫從庫同步主庫

show master status；
#配置主從復制起點

change master to master_host=‘192.168.52.251‘,master_user=‘repluser‘,master_password=‘123456‘,master_log_file=‘master-log.000004‘,master_log_pos=154; #註意masterIP地址
start slave;
show slave status\G

4、切換完畢後缺少

secondary_check_script=/usr/local/bin/masterha_secondary_check -s db251 -s db252 --user=root --master_host=db252 --master_ip=192.168.52.252 --master_port=3306

添加：
[server1]
hostname=192.168.52.252
candidate_master=1
port=3306

5、測試
測試ssh
masterha_check_ssh --conf=/etc/masterha/app1.cnf

測試MHA數據庫同步，並自動啟動
masterha_check_repl --conf=/etc/masterha/app1.cnf

查看MHA Manager監控是否正常：
masterha_check_status --conf=/etc/masterha/app1.cnf

6、日誌查看
tailf /var/log/masterha/app1/manager.log

7、開啟MHA Manager監控
nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/masterha/app1/manager.log 2>&1 &

8、查詢manger程序

ps -ef|grep perl
1.15 安裝配置LVS+keepalived
192.168.52.251（backup_master）和192.168.52.252（slave）
[root@db251 ~]# vi /etc/rc.d/init.d/realserver
#!/bin/sh
# chkconfig: - 80 90
# description:realserver
# mysql_vip start realserver*
mysql_vip=192.168.52.199
. /etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $mysql_vip netmask 255.255.255.255 broadcast $mysql_vip
/sbin/route add -host $mysql_vip dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $mysql_vip >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0

chmod +x /etc/rc.d/init.d/realserver #添加腳本執行權限
chkconfig realserver on #添加開機啟動
/etc/rc.d/init.d/realserver start #開啟，參數stop為關閉

Lvs01-192.168.52.248,配置文件249和248一致
[root@lvs248 ~]# yum install ipvsadm keepalived gcc gcc-c++ make openssl-devel kernel-devel ncurses-devel -y
[root@lvs248 ~]# lsmod |grep ip_vs
ip_vs 141432 0
nf_conntrack 133053 1 ip_vs
libcrc32c 12644 3 xfs,ip_vs,nf_conntrack
[root@lvs248 ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf-bak
[root@lvs248 ~]# >/etc/keepalived/keepalived.conf
[root@lvs248 ~]# vim /etc/keepalived/keepalived.conf

    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
    auth_type PASS
    auth_pass 1111
    }

    virtual_ipaddress {
            192.168.52.159
    }

}

virtual_server 192.168.52.159 3306 {

            delay_loop 6
            lb_algo wlc
            lb_kind DR
            #persistence_timeout 120
            protocol TCP

    real_server 192.168.52.252 3306 {

            weight 1
            TCP_CHECK {
                    connect_timeout 3
                    nb_get_retry 3
                    delay_before_retry 3
                    connect_port 3306
            }
            MISC_CHECK {
                     misc_path "/etc/keepalived/check_slave.py 192.168.52.252 3306"
                     misc_dynamic

}
}

    real_server 192.168.52.253 3306 {

            weight 1
            TCP_CHECK {
                    connect_timeout 3
                    nb_get_retry 3
                    delay_before_retry 3
                    connect_port 3306
    }
             MISC_CHECK {
                     misc_path "/etc/keepalived/check_slave.py 192.168.52.253 3306"
                     misc_dynamic
    }
}

}

[root@lvs248 keepalived]# pwd
/etc/keepalived
[root@lvs248 keepalived]# ll
total 12
-rw-r--r-- 1 root root 816 Jan 11 11:05 check_slave.py
-rw-r--r-- 1 root root 1489 Jan 11 11:35 keepalived.conf
-rw-r--r-- 1 root root 3598 Jan 11 10:01 keepalived.conf-bak

vim /etc/keepalived/check_slave.py
#!/usr/bin/env python
#encoding:utf-8
import MySQLdb
import sys
ip=sys.argv[1]
user=‘repluser‘
pwd=‘123456‘
port=int(sys.argv[2])
sbm=200
Slave_IO_Running = ‘‘
Slave_SQL_Running = ‘‘
Seconds_Behind_Master = ‘‘
e=‘‘
try:
conn = MySQLdb.connect(host=ip,user=user,passwd=pwd,port=port,charset=‘utf8‘)
cur = conn.cursor()
cur.execute(‘show slave status‘)
db_info = cur.fetchall()
for n in db_info:
Slave_IO_Running = n[10]
Slave_SQL_Running = n[11]
Seconds_Behind_Master = n[32]
cur.close()
conn.close()
except MySQLdb.Error,e:
print "MySQLdb Error",e
ife == "":
ifdb_info != ():
ifSlave_IO_Running == "No" or Slave_SQL_Running == "No":
#print ‘thread err‘
exit(1)
else:
ifSeconds_Behind_Master > sbm:
#print ‘timeout err‘
exit(1)
else:
#print ‘OK‘
exit(0)
else:
#print ‘slave err‘
exit(1)
else:
#print ‘db err‘
exit(1)
~

[root@lvs248 keepalived]# systemctl start keepalived
[root@lvs248 keepalived]# systemctl status keepalived
[root@lvs248 keepalived]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.52.199:3306 wlc
-> 192.168.52.252:3306 Route 1 0 0
-> 192.168.52.253:3306 Route 1 0 0
1.16 測試
測試 read vip 負載均衡
分別在從庫 db252、 db253、創建可區分的庫，庫名分別為
read_one、 read_two
# db252
mysql> create database read_one;
Query OK, 1 row affected (0.00 sec)
mysql> show databases like ‘read_one‘;
+---------------------+
| Database (read_one) |
+---------------------+
| read_one |
+---------------------+
1 row in set (0.00 sec)
# db252 操作略
[root@db251 ~]# mysql -uroot -p123456 -h 192.168.52.159 -P3306 -e "show databases"|egrep -v "schema|mysql|test|Database"
mysql: [Warning] Using a password on the command line interface can be insecure.
read_one
read_two
sys
[root@db251 ~]# mysql -uroot -p123456 -h 192.168.52.159 -P3306 -e "show databases"|egrep -v "schema|mysql|test|Database"
mysql: [Warning] Using a password on the command line interface can be insecure.
read_one
sys

測試從庫故障被剔除，恢復被掛起
將 db253 數據庫關閉，觀察 lvs 狀態
#lvs 未關閉 mysql 時的狀態
[root@master1 keepalived]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.52.159:3306 wlc
-> 192.168.52.252:3306 Route 1 0 0
-> 192.168.52.253:3306 Route 1 0 0
[root@master1 keepalived]# msyql -uroot -h 192.168.52.159 -p
-bash: msyql: command not found
[root@master1 keepalived]# mysql -uroot -h 192.168.52.159 -p
Enter password:
^Z
[1]+ Stopped mysql -uroot -h 192.168.52.159 -p
#lvs 關閉 mysql 後的狀態
[root@master1 keepalived]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.52.159:3306 wlc
-> 192.168.52.252:3306 Route 1 0 0

註：數據庫開啟後，會自動加入到節點中
[root@master1 keepalived]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.52.159:3306 wlc
-> 192.168.52.252:3306 Route 1 0 0
-> 192.168.52.253:3306 Route 1 0 0

測試 keepalived 高可用 vip 切換
192.168.52.248
[root@lvs248 keepalived]# systemctl stop keepalived

192.168.52.249
[root@lvs249 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.52.159:3306 wlc
-> 192.168.52.252:3306 Route 1 0 0
-> 192.168.52.253:3306 Route 1 0 0

192.168.52.249
[root@lvs249 keepalived]# systemctl stop keepalived

192.168.52.248
[root@lvs248 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.52.159:3306 wlc
-> 192.168.52.252:3306 Route 1 0 0
-> 192.168.52.253:3306 Route 1 0 0

1.17 報錯日誌和解決方法
報錯日誌：
[root@ MHA]# ln -s /usr/local/mysql/bin/mysqlbinlog /usr/bin/mysqlbinlog
12345678910111213 Binlog found at /log/mysql, up to mysql-bin.000009
Tue Apr 5 22:43:55 2016 - [info] Binlog setting check done.
Tue Apr 5 22:43:55 2016 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Tue Apr 5 22:43:55 2016 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user=‘root‘ --slave_host=192.168.1.248 --slave_ip=192.168.1.248 --slave_port=4008 --workdir=/usr/local/mha --target_version=5.6.16-log --manager_version=0.57 --relay_log_info=/data/mysql/relay-log.info --relay_dir=/data/mysql/ --slave_pass=xxx
Tue Apr 5 22:43:55 2016 - [info] Connecting to [email protected](192.168.1.248:22)..
mysqlbinlog: unknown variable ‘default-character-set=utf8‘
mysqlbinlog version command failed with rc 7:0, please verify PATH, LD_LIBRARY_PATH, and client options
at /usr/local/bin/apply_diff_relay_logs line 493
Tue Apr 5 22:43:55 2016 - [error][/usr/local/lib64/perl5/MHA/MasterMonitor.pm, ln205] Slaves settings check failed!
Tue Apr 5 22:43:55 2016 - [error][/usr/local/lib64/perl5/MHA/MasterMonitor.pm, ln413] Slave configuration failed.
Tue Apr 5 22:43:55 2016 - [error][/usr/local/lib64/perl5/MHA/MasterMonitor.pm, ln424] Error happened on checking configurations. at /usr/local/bin/masterha_check_repl line 48
Tue Apr 5 22:43:55 2016 - [error][/usr/local/lib64/perl5/MHA/MasterMonitor.pm, ln523] Error happened on monitoring servers.
Tue Apr 5 22:43:55 2016 - [info] Got exit code 1 (Not master dead).
解決方案
註釋掉my.cnf中的
[client]
#default-character-set=utf8
報錯2：
[root@db52 ~]# tailf /var/log/mysqld.log
2019-01-10T03:10:01.260664Z 0 [ERROR] Error during --relay-log-recovery: Could not locate rotate event from the master.
2019-01-10T03:10:01.294452Z 0 [Note] InnoDB: Buffer pool(s) load completed at 190110 11:10:01
2019-01-10T03:10:01.294559Z 0 [ERROR] Slave: Failed to initialize the master info structure for channel ‘‘; its record may still be present in ‘mysql.slave_master_info‘ table, consider deleting it.
2019-01-10T03:10:01.294590Z 0 [ERROR] Failed to create or recover replication info repositories.
2019-01-10T03:10:01.294599Z 0 [Note] Some of the channels are not created/initialized properly. Check for additional messages above. You will not be able to start replication on those channels until the issue is resolved and the server restarted.
2019-01-10T03:10:01.301143Z 0 [Note] Event Scheduler: Loaded 0 events
2019-01-10T03:10:01.301320Z 0 [Note] /usr/sbin/mysqld: ready for connections.
解決方法：
mysql> reset slave;
Query OK, 0 rows affected (0.31 sec)
mysql> change master to master_host=‘192.168.20.52‘,master_user=‘repluser‘,master_password=‘123456‘,master_log_file=‘master-log.000015‘,master_log_pos=2923;
Query OK, 0 rows affected, 2 warnings (0.32 sec)
mysql> start slave;
Query OK, 0 rows affected (0.51 sec)
報錯3：
Got fatal error 1236 from master when reading data from binary log: ‘Could not find first log file name in binary log index file‘

解決方法：
/usr//bin/mysqlbinlog --start-position=2923 /var/lib/mysql/mysql-bin.000015

整個MHA+keepalived+lvs+mysql高可用架構配置說明

dex diff vim could not 分區創建 pos ann 試驗 pts 整個MHA+keepalived+lvs+mysql高可用架構配置說明1.1. 環境簡介1.1.1、vmvare虛擬機，系統版本CentOS7.5 x86_64位最小化安裝，mysql的版

整個MHA+keepalived+lvs+mysql高可用架構配置說明

整個MHA+keepalived+lvs+mysql高可用架構配置說明

MHA+keepalived實現Mysql高可用及讀寫分離

MySQL高可用架構之MHA

MySQL主主復制+LVS+Keepalived實現MySQL高可用性

互聯網金融MySQL高可用架構之-MHA故障切換

MySQL高可用架構之基於MHA的搭建

MySQL高可用架構-MHA環境部署記錄

Mysql高可用架構——MHA

MySQL 主主複製 + LVS + Keepalived 實現 MySQL 高可用性

mysql高可用架構之MHA和mysql日誌優化

MySQL高可用架構之MHA搭建以及測試（二）

Nginx+Tomcat+lvs+Keepalived搭建的高可用架構

mysql高可用架構mha之master_ip_failover腳本

實現基於Haproxy_NAT+Keepalived負載均衡高可用架構

實現基於Haproxy+Keepalived負載均衡高可用架構

基於DR模式的keepalived主從模式高可用架構搭建

LVS+Keepalied高可用架構

keepalived+lvs實現高可用負載均衡集群

keepalived+lvs+nginx 高可用

Memcached 主主復制+ Keepalived 實現 Memcached 高可用架構集群

整個MHA+keepalived+lvs+mysql高可用架構配置說明

相關推薦