keepalived+lvs+mysql cluster架構實現負載均衡

阿新 • • 發佈：2019-02-06

一、環境

用途	ip	os	memory
排程器	172.16.48.204	CentOS 5.5 64bit	32G
sql節點1	172.16.48.206	CentOS 5.5 64bit	32G
sql節點2	172.16.48.207	CentOS 5.5 64bit	32G
sql節點3	172.16.48.208	CentOS 5.5 64bit	32G
sql節點4	172.16.48.211	CentOS 5.5 64bit	32G

VIP設定為：172.100.100.251 （注：不能和排程器、sql節點在一個網段上)

在排程器172.16.48.204電腦上，用route add default dev eth1
命令新增路由，使得

172.16.48.204節點有對100網段的訪問許可權。

二、架構圖

這裡排程器只用了一個節點，沒有設定Backup

三、軟體下載

lvs: http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz

keepalived: http://www.keepalived.org/download.html

四、軟體安裝

編輯lvs的時候需要作業系統的核心，不然會報錯

configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!

解決方法：yum install openssl*

可能還會報錯：libipvs.c:253: 錯誤：提領指向不完全型別的指標

解決方法：yum -y install kernel-devel 可參考：http://692344.blog.51cto.com/blog/682344/804278

如果還是不行，可能是由於lvs keepalived的版本過高

1.在排程器172.16.48.204上安裝lvs

ln -s /usr/src/kernels/2.6.18-164.el5-i686/ /usr/src/linux
tar zxvf ipvsadm-1.24.tar.gz
cd ipvsadm-1.24
make && make install

2.在排程器172.16.48.204上安裝keepalived

tar zxvf keepalived-1.1.19.tar.gz
cd keepalived-1.1.19
./configure --prefix=/usr/local/keepalived
make
make install
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived

3.配置keepalived.conf

vi /etc/keepalived/keepalived.conf
新增如下內容：

global_defs {
router_id HaMySQL_1
}
vrrp_sync_group VGM {
group {
VI_MYSQL
}
}
vrrp_instance VI_MYSQL {
state MASTER
interface eth1
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.100.100.251/24 etho label eth0:1
}
}
virtual_server 172.100.100.251 3306 {
delay_loop 6
lb_algo rr
lb_kind DR
# nat_mask 255.255.0.0
#persistence_timeout 50
protocol TCP
real_server 172.16.48.206 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 172.16.48.207 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 172.16.48.208 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 172.16.48.211 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}

4.sql服務節點的配置

vi /etc/rc.d/init.d/realserver.sh 新增如下配置：

#!/bin/bash
# description: Config realserver lo and apply noarp
SNS_VIP=172.100.100.251
/etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0

新增加可執行的許可權
chmod +x /etc/rc.d/init.d/realserver.sh

注：四臺sql節點都要進行此步操作

五、啟動LVS節點（mysql cluster中的sql節點)

/etc/rc.d/init.d/realserver.sh start

六、啟動排程器（keepalived的伺服器端啟動）
/etc/rc.d/init.d/keepalived start

啟動後可以用命令 tail -f /var/log/messages 進行檢視

若看到如下提示，則表示配置成功。

[[email protected] mysql-cluster]# tail -f /var/log/messages
May 29 17:21:01 sg204 Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
May 29 17:21:01 sg204 Keepalived_vrrp: Configuration is using : 63921 Bytes
May 29 17:21:01 sg204 Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
May 29 17:21:01 sg204 Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
May 29 17:21:01 sg204 Keepalived_healthcheckers: Activating healtchecker for service [172.16.48.206:3306]
May 29 17:21:01 sg204 Keepalived_healthcheckers: Activating healtchecker for service [172.16.48.207:3306]
May 29 17:21:01 sg204 Keepalived_healthcheckers: Activating healtchecker for service [172.16.48.208:3306]
May 29 17:21:01 sg204 Keepalived_healthcheckers: Activating healtchecker for service [172.16.48.211:3306]
May 29 17:21:01 sg204 Keepalived_vrrp: VRRP sockpool: [ifindex(3), proto(112), fd(10,11)]
May 29 17:21:02 sg204 Keepalived_vrrp: VRRP_Instance(VI_MYSQL) Transition to MASTER STATE
May 29 17:21:03 sg204 Keepalived_vrrp: VRRP_Instance(VI_MYSQL) Entering MASTER STATE
May 29 17:21:03 sg204 Keepalived_vrrp: VRRP_Instance(VI_MYSQL) setting protocol VIPs.
May 29 17:21:03 sg204 Keepalived_healthcheckers: Netlink reflector reports IP 172.100.100.251 added
May 29 17:21:03 sg204 Keepalived_vrrp: VRRP_Instance(VI_MYSQL) Sending gratuitous ARPs on eth1 for 172.100.100.251
May 29 17:21:03 sg204 Keepalived_vrrp: VRRP_Group(VGM) Syncing instances to MASTER state
May 29 17:21:03 sg204 avahi-daemon[4768]: Registering new address record for 172.100.100.251 on eth1.
May 29 17:21:03 sg204 Keepalived_vrrp: Netlink reflector reports IP 172.100.100.251 added
May 29 17:21:08 sg204 Keepalived_vrrp: VRRP_Instance(VI_MYSQL) Sending gratuitous ARPs on eth1 for 172.100.100.251

在排程器上用ipvsadm命令檢視，可以看到如下資訊

[[email protected] mysql-cluster]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.100.100.251:mysql rr
-> 172.16.48.211:mysql Route 3 0 0
-> 172.16.48.208:mysql Route 3 0 0
-> 172.16.48.207:mysql Route 3 0 0
-> 172.16.48.206:mysql Route 3 0 0

七、遠端應用通過vip訪問mysql資料庫
要遠端應用程式訪問mysql資料庫，還需要一個遠端訪問的賬號和密碼，在mysql剛建立後，mysql是沒有遠端訪問賬號的，

在sql節點上增加遠端認證客戶：

grant 許可權1,許可權2,…許可權n on 資料庫名稱.表名稱 to 使用者名稱@使用者地址 identified by ‘連線口令’

Mysql>:select user,host,password from mysql.user;
Mysql>: grant all privileges on *.* to [email protected]"%" identified by '111111' with grant option;

八、測試

在其他節點上（排程器和sql節點之外的電腦），ping 172.100.100.251發現可以ping通

這裡選用的是172.16.48.201節點：

[[email protected] ~]# ping 172.100.100.251
PING 172.100.100.251 (172.100.100.251) 56(84) bytes of data.
64 bytes from 172.100.100.251: icmp_seq=1 ttl=64 time=0.093 ms
64 bytes from 172.100.100.251: icmp_seq=2 ttl=64 time=0.164 ms
64 bytes from 172.100.100.251: icmp_seq=3 ttl=64 time=0.162 ms

在172.16.48.201上安裝mysql客戶端後，用命令 mysql -u admin -p -h 172.100.100.251測試發現可以連線資料庫，

將172.16.48.208上的mysql服務停掉後，這時在高度器上tail -f var/log/messages會看到排程器自動將無效sql節點移除lvs了

May 30 11:04:11 sg204 Keepalived_healthcheckers: TCP connection to [172.16.48.208:3306] failed !!!
May 30 11:04:11 sg204 Keepalived_healthcheckers: Removing service [172.16.48.208:3306] from VS [172.100.100.251:3306]

再將172.16.48.208上的mysql服務啟動起來，這時

May 30 11:06:53 sg204 Keepalived_healthcheckers: TCP connection to [172.16.48.208:3306] success.
May 30 11:06:53 sg204 Keepalived_healthcheckers: Adding service [172.16.48.208:3306] to VS [172.100.100.251:3306]

哈哈！成功搞定

keepalived+lvs+mysql cluster架構實現負載均衡

keepalived+lvs+mysql cluster架構實現負載均衡

Linux學習總結（五十四）keepalived+lvs 雙機熱備負載均衡架構

Docker compose（多容器架構實現負載均衡）

MySQL Cluster SQL節點負載均衡、讀寫分離驗證

LVS-DR簡單模型實現負載均衡

lvs+keepalived實現負載均衡

LVS（DR） + Keepalived 實現負載均衡

Keepalived + LVS 實現負載均衡

Keepalived結合LVS-DR實現負載均衡高可用集群

lvs負載均衡的四種模式原理，LVS之DR模型，LVS+keeaplived實現負載均衡，keepalived實現高可用

LVS+keepalived 實現負載均衡

Nginx+keepalived做雙機熱備，實現負載均衡（主主模式）

Haproxy+keepalived實現負載均衡

haproxy+keepalived實現負載均衡及高可用

Keepalived + Haproxy實現負載均衡高可用

LVS實現負載均衡

項目實戰02：LVS 實現負載均衡

Keepalived+Nginx實現負載均衡高可用

使用LVS實現負載均衡原理及安裝配置詳解

LVS-NAT與LVS-DR類型的負載均衡實現原理

keepalived+lvs+mysql cluster架構實現負載均衡

相關推薦