keepalived + mysqlroute +mysql實現mysql高可用
1.1服務器角色配置:
mysql-master 10.0.0.101(外) 172.168.1.101(內)
mysql-slave 10.0.0.103 (外) 172.168.1.103(內)
mysql-router01 10.0.0.102 (外) 172.168.1.102(內)
mysql-router02 10.0.0.104 (外) 172.168.1.104(內)
jumpserver 10.0.0.128 (外) 172.168.1.128(內)
1.2配置數據庫的一主兩從庫
Slave01 和master都是允許讀寫的,slave02提供只讀
同時MySQL的主從復制都是事先配置成功的(此環境是開啟GTID模式的復制)
1.3部署web應用jumpserver
jumpserver web 10.0.0.128(外) 172.168.1.128(內)
jumpsever配置文件如下:
[root@localhost jumpserver]# cat /opt/jumpserver/jumpserver.conf [base] url = http://10.0.0.128 key = f1tty6elu8h03x2k ip = 0.0.0.0 ###默認監聽任何IP port = 8000 ###為jumpsever的默認的web端口 log = debug [db] engine = mysql host = 172.168.1.20 ####為虛擬VIP port = 7001 ####為mysqlroute讀寫模式的默認端口 ####jumpserver的連接數據庫的地址 user = jumpserver password = jumpserver database = jumpserver [mail] mail_enable = 1 email_host = [email protected] email_port = 25 email_host_user = [email protected] email_host_password = weuidnre email_use_tls = False email_use_ssl = False [connect] nav_sort_by = ip
啟動jumpsever服務:
[root@localhost jumpserver]# /opt/jumpserver/service.sh stop [root@localhost jumpserver]# /opt/jumpserver/service.sh start [root@localhost jumpserver]# /opt/jumpserver/service.sh status jumpserver is running... [ OK ] [root@localhost jumpserver]# [root@localhost jumpserver]# ps -ef|grep python root 2443 2441 0 May14 ? 00:00:00 /bin/bash -c ulimit -S -c 0 >/dev/null 2>&1 ; python ./run_server.py root 2444 2443 0 May14 ? 00:00:04 python ./run_server.py root 9081 8974 0 14:50 pts/0 00:00:00 grep python
1.4安裝keepalived和mysql-router
mysql-router01 10.0.0.102 mysql-router02 10.0.0.104
在10.0.0.102 機器上安裝keepalived 和mysql-router
在10.0.0.104 機器上安裝keepalived 和mysql-router
[root@master01 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
slave01 172.168.1.103
master01 172.168.1.101
router01 172.168.1.102
router02 172.168.1.104
slave02 172.168.1.105
重要提示:
為快速實現環境達到演示效果,建議6臺臺虛擬機先關閉selinux和iptables,保證服務器的系統時間都是一致的,必須保證iptables不是開機自啟動,安裝keepalived和mysql-router後,要保證這2個服務是開啟自啟動的。
二、keepalived相關原理介紹:
Keepalived 是一種高性能的服務器高可用或熱備解決方案,Keepalived可以用來防止服務器單點故障(單點故障是指一旦某一點出現故障就會導致整個系統架構的不可用)的發生,通過配合Nginx可以實現web前端服務的高可用。以下本文會介紹keepalived結合mysql-router實現數據庫得高可用
Keepalived實現的基礎是VRRP協議,Keepalived就是巧用VRRP協議來實現高可用性(HA)的.
VRRP(Virtual Router Redundancy Protocol)協議是用於實現路由器冗余的協議,VRRP協議將兩臺或多臺路由器設備虛擬成一個設備,對外提供虛擬路由器IP(一個或多個),而在路由器組內部,如果實際擁有這個對外IP的路由器如果工作正常的話就是MASTER,或者是通過算法選舉產生,MASTER實現針對虛擬路由器IP的各種網絡功能,如ARP請求,ICMP,以及數據的轉發等;其他設備不擁有該IP,狀態是BACKUP,除了接收MASTER的VRRP狀態通告信息外,不執行對外的網絡功能。當主機失效時,BACKUP將接管原先MASTER的網絡功能。
VRRP協議使用多播數據來傳輸VRRP數據,VRRP數據使用特殊的虛擬源MAC地址發送數據而不是自身網卡的MAC地址,VRRP運行時只有MASTER路由器定時發送VRRP通告信息,表示MASTER工作正常以及虛擬路由器IP(組),BACKUP只接收VRRP數據,不發送數據,如果一定時間內沒有接收到MASTER的通告信息,各BACKUP將宣告自己成為MASTER,發送通告信息,重新進行MASTER選舉狀態。
三.Mysql-router相關原理介紹:
3.1.什麽是mysql route
MySQL Router是處於應用client和dbserver之間的輕量級代理程序,它能檢測,分析和轉發查詢到後端數據庫實例,並把結果返回給client。是mysql-proxy的一個替代品。其架構圖和功能如下:
(1)Router實現讀寫分離,程序不是直接連接數據庫IP,而是固定連接到mysql router。MySQL Router對前端應用是透明的。應用程序把MySQL Router當作是普通的mysql實例,把查詢發給MySQL Router,而MySQL Router會把查詢結果返回給前端的應用程序。
(2)從數據庫服務器故障,業務可以正常運行。由MySQL Router來進行自動下線不可用服務器。程序配置不需要任何修改。
(3)主數據庫故障,由MySQL Router來決定主從自動切換,業務可以正常訪問。程序配置不需要做任何修改。
3.2.讀寫分離原理
MySQL Router接受前端應用程序請求後,根據不同的端口來區分讀寫,把連接讀寫端口的所有查詢發往主庫,把連接只讀端口的select查詢以輪詢方式發往多個從庫,從而實現讀寫分離的目的。讀寫返回的結果會交給MySQL Router,由MySQL Router返回給客戶端的應用程序。
3.3.Mysql router用途
MySQL Router的主要用途是讀寫分離,主主故障自動切換,負載均衡,連接池等。
3.4.Mysql router主主故障自動切換的坑
Mysql router主主故障切換功能經過測試沒有問題,但是有一個比較大的坑需要註意
Mysql router的主主故障切換如果是運行在一主一從的情況下,從庫作為主庫的備份,這種情況是使用mysql router主主故障切換是沒有問題的,但是在一主多從的情況下使用主主故障切換就會處在如下問題:mysql主庫掛掉了,從庫slave01和slave02卻一直是同步的原來的主庫master的IP地址,導致slave上的IO線程一直是connecting狀態,導致slave02復制失敗。
四.軟件具體安裝部署
10.0.0.102機器上安裝mysql-router 和keepalived
此機器上的keepalived是主
yum -y install openssl-devel
wget http://www.keepalived.org/software/keepalived-1.4.0.tar.gz
tar xf keepalived-1.4.0.tar.gz -C /usr/local/
cd /usr/local/keepalived-1.4.0/
./configure
make
make install
mkdir /etc/keepalived
find /usr/local/keepalived-1.4.0/ -name "keepalived.conf"
cp /usr/local/keepalived-1.4.0/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
find / -name "keepalived"
cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived-1.4.0/keepalived/etc/init.d/keepalived /etc/init.d/
chmod +x /etc/init.d/keepalived
chkconfig keepalived on
cp /usr/local/sbin/keepalived /usr/sbin/
which keepalived
cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.ori
下面的配置文件是master主keepalived的配置文件
vim /etc/keepalived/keepalived.conf
[root@master01 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
**router_id LVS_01**
}
vrrp_instance VI_1 {
** state MASTER**
**interface eth1**
virtual_router_id 51
**priority 120**
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
**172.168.1.20/24**
}
}
/etc/init.d/keepalived status
/etc/init.d/keepalived start
同樣的方式在10.0.0.104機器上安裝mysql-router 和keepalived
此機器上的keepalived是從
下面是從keepalived機器的配置文件介紹:
[root@router02 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
** router_id LVS_02**
}
vrrp_instance VI_1 {
** state BACKUP
interface eth1**
virtual_router_id 51
** priority 80**
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
** 172.168.1.20/24**
}
}
主從keepalived參數介紹:
先是主服務器:
global_defs
{
notification_email #通知email,根據實際情況配置{
[email protected]
br/>{
[email protected]
notification_email_from [email protected]
smtp_server 127.0.0.1
stmp_connect_timeout 30
router_id node1 #節點名標識,主要用於通知中
}
vrrp_instance VI_NODE {
state MASTER #配置為主服務器
interface eth0 #通訊網卡
virtual_router_id 100 #路由標識
priority 200 #優先級,0-254
advert_int 5 #通知間隔,實際部署時可以設置小一點,減少延時
authentication {
auth_type PASS
auth_pass 123456 #驗證密碼,用於通訊主機間驗證
}
virtual_ipaddress {
192.168.1.206 #虛擬ip,可以定義多個
}
}
接下是從服務器設置:
global_defs {
notification_email {[email protected]
br/>[email protected]
notification_email_from [email protected]
smtp_server 127.0.0.1
stmp_connect_timeout 30
router_id node2
}
vrrp_instance VI_NODE {
state BACKUP #與主服務器對應
interface eth0 #從服務器的通信網卡
virtual_router_id 100 #路由標識,和主服務器相同
priority 100 #優先級,小於主服務器即可
advert_int 5 #這裏是接受通知間隔,與主服務器要設置相同
authentication {
auth_type PASS
auth_pass 123456 #驗證密碼,與主服務器相同
}
virtual_ipaddress {
192.168.1.206 #虛擬IP,也要和主服務器相同
}
}
上面的設置是最基礎的設置,實現的功能是如果主服務器的Keepalived停止服務(一般情況下服務器宕機),則將虛擬IP切換至從服務器,主服務器恢復後從新切換回主服務器。
4.1、Keepalived的配置文件介紹來實現虛擬VIP漂移
註意:1、兩個keepalived.conf配置文件中添加的虛擬網卡要一樣,並且不要和已使用服務器的內外網卡一樣這樣就實現不了心跳了。
可以是同一網段局域內網卡,也可以是同一網段外網網卡
2、當主keepalived應用開啟的時候,是可以看到虛擬IP地址的
[root@router01 ~]# ps -ef|grep keepalived
root 9265 1 0 07:24 ? 00:00:00 keepalived -D
root 9267 9265 0 07:24 ? 00:00:00 keepalived -D
root 9268 9265 0 07:24 ? 00:00:05 keepalived -D
root 9517 9492 0 10:02 pts/3 00:00:00 grep keepalived
[root@router01 ~]# ip addr|grep 172.168.1.20
inet 172.168.1.20/24 scope global secondary eth1
這時從keepalived程序也開啟,但是在從本地是grep不到虛擬IP地址的
[root@router02 ~]# ps -ef|grep keepalived
root 13115 1 0 07:08 ? 00:00:00 keepalived -D
root 13117 13115 0 07:08 ? 00:00:00 keepalived -D
root 13118 13115 0 07:08 ? 00:00:01 keepalived -D
root 13271 13232 0 10:02 pts/2 00:00:00 grep keepalived
[root@router02 ~]# ip a|grep 172.168.1.20
[root@router02 ~]#
一旦主keepalived程序關閉或者是服務器掛掉,這時虛擬VIP就可以實現跳躍到從keepalived機器上。
[root@router01 ~]# /etc/init.d/keepalived stop
Stopping keepalived: [ OK ]
[root@router01 ~]# ps -ef|grep keepalived
root 9540 9492 0 10:21 pts/3 00:00:00 grep keepalived
[root@router01 ~]#
[root@router02 ~]# ip a|grep 172.168.1.20
inet 172.168.1.20/24 scope global secondary eth1
[root@router02 ~]#
提示:
如果從負載均衡服務器grep不到虛擬網卡原因有:
1、本地服務器keepalived程序可能關閉,得restart從啟才行,才能實現虛擬網卡的跳躍。
2、就是主負載均衡服務器keepalived程序沒有關閉而導致
3、主負載均衡服務器只要keepalived服務啟動就不會出現上述的問題。
4.服務器的iptables沒關閉導致,或者是selinxu沒關閉也有可能導致
4.2.安裝mysql-router
10.0.0.102 10.0.0.104兩個機器安裝mysql-router
采用二進制安裝mysql-router:
tar xf mysql-router-2.1.6-linux-glibc2.12-x86-64bit.tar.gz -C /usr/local/
cd /usr/local/
ln -s mysql-router-2.1.6-linux-glibc2.12-x86-64bit mysql-router
mkdir /etc/mysql-route/
mkdir /data/log/mysql-route -p
cp /usr/local/mysql-router/share/doc/mysqlrouter/sample_mysqlrouter.conf /etc/mysql-route/mysqlrouter.conf
cp /etc/mysql-route/mysqlrouter.conf /etc/mysql-route/mysqlrouter.conf.ori
4.3配置文件介紹如下:
[root@router01 mysql-route]# cat mysqlrouter.conf
[DEFAULT]
##日誌存放目錄
logging_folder = /data/log/mysql-route
##插件存放目錄
plugin_folder = /usr/local/mysql-router/lib/mysqlrouter
###配置文件存放目錄
config_folder = /etc/mysql-route
###運行目錄
runtime_folder = /var/run
[logger]
###日誌運行級別
level = INFO
#[fabric_cache]
#address = your_fabric_node.example.com:32275
#user =
###主節點故障轉移
[routing:basic_failover]
#To be more transparent, use MySQL Server port 3306
##寫節點地址
bind_address=172.168.1.20 ##{此IP地址為虛擬VIP}
##寫節點端口
bind_port = 7001
##主庫為讀寫模式
mode = read-write
###172.168.1.101為mysql 主 172.168.1.103為mysql從
#### 主節點地址:默認情況下第一臺主數據庫為寫主庫,當第一臺主數據庫DOWN機後,第二臺數據庫被提升為主庫
destinations = 172.168.1.101:3306,172.168.1.103:3306
[routing:balancing]
bind_address=172.168.1.20
bind_port = 7002
connect_timeout = 3
max_connections = 1024
###為mysql從庫(此處只負責讀)
destinations = 172.168.1.105:3306
mode = read-only
#[routing:homepage_reads_fabric]
#bind_port = 7002
#destinations = fabric+cache:///group/homepage_group?allow_primary_reads=yes
#mode = read-only
#If no plugin is configured which starts a service, keepalive
#will make sure MySQL Router will not immediately exit. It is
#safe to remove once Router is configured.
[keepalive]
interval = 60
啟動mysqlrouter服務
/usr/local/mysql-router/bin/mysqlrouter -c /etc/mysql-route/mysqlrouter.conf &
關閉iptables 保證mysql-router開機自啟動
[root@router01 ~]# chkconfig --list|grep iptables
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@router01 ~]# chkconfig iptables off
[root@router01~]# vim /etc/rc.local
#!/bin/sh
touch /var/lock/subsys/local
/usr/local/mysql-router/bin/mysqlrouter -c /etc/mysql-route/mysqlrouter.conf &
[root@router02 ~]# chkconfig --list|grep iptables
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@router02 ~]# chkconfig iptables off
[root@router02 ~]# vim /etc/rc.local
#!/bin/sh
touch /var/lock/subsys/local
/usr/local/mysql-router/bin/mysqlrouter -c /etc/mysql-route/mysqlrouter.conf &
4.4創建測試賬戶進行測試:
登錄mysql master主庫進行創建後,賬戶會同步到各個slave上
mysql> grant all on jumpserver.* to jumpserver@‘172.168.1.%‘ identified by ‘jumpserver‘;
[root@master01 ~]# mysql -h172.168.1.20 -P7001 -ujumpserver -pjumpserver -e "show variables like ‘hostname‘;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+---------------+----------+
| Variable_name | Value |
+---------------+----------+
| hostname | master01 |
[root@router01 ~]# mysql -h172.168.1.20 -P7001 -ujumpserver -pjumpserver -e "show variables like ‘hostname‘;"
+---------------+----------+
| Variable_name | Value |
+---------------+----------+
| hostname | master01 |
[root@slave01 ~]# mysql -h172.168.1.20 -P7001 -ujumpserver -pjumpserver -e "show variables like ‘hostname‘;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+---------------+----------+
| Variable_name | Value |
+---------------+----------+
| hostname | master01 |
keepalived + mysqlroute +mysql實現mysql高可用