005.HAProxy+Keepalived高可用負載均衡
阿新 • • 發佈：2018-12-23
一 基礎準備
1.1 部署環境及說明
系統OS：CentOS 6.8 64位 HAProxy軟體：HA-Proxy version 1.5.18 Keepalived軟體：keepalived-1.3.6.tar.gz 官方連結：http://www.haproxy.org/ （國內可能無法開啟） 下載連線：http://pkgs.fedoraproject.org/repo/pkgs/haproxy/ 部署說明：當用戶訪問對應的域名時，HAProxy能將請求傳送到對應的後端主機上，同時當主HAProxy伺服器發生故障後，能立刻將負載均衡服務切換到備用HAProxy伺服器上。1.2 架構規劃
二 後端httpd叢集部署
2.1 部署httpd叢集
1 [[email protected] ~]# yum -y install httpd 2 [[email protected] ~]# vi /var/www/html/index.html 3 This is my www.lz.com! 4 [[email protected] ~]# vi /var/www/html/index.html 5 This is my static.lz.com! 6 [[email protected] ~]# vi /var/www/html/index.html 7 This is my video.lz.com! 8 [[email protected] ~]# systemctl start httpd.service 9 [[email protected] ~]# systemctl enable httpd.service 10 [[email protected] ~]# systemctl stop firewalld.service 11 [[email protected] ~]# systemctl disable firewalld.service 12 [[email protected] ~]# vi /etc/selinux/config 13 SELINUX=disabled 14 [[email protected] ~]# setenforce 0 #關閉SELinux及防火牆注意：後端real server節點都需要安裝，本環境針對httpd簡單安裝即可，無需過多配置。
三 基礎NTP部署
3.1 NTP部署
1 [[email protected] ~]# yum -y install ntp 2 [[email protected] ~]# systemctl start ntpd.service建議：替換附件中的ntp配置檔案，建議採用阿里雲時鐘進行同步。 注意：為了保證叢集的穩定性，強烈建議在所有節點均部署NTP同步服務，保證所有時鐘一致。
四 Keepalived部署
4.1 編譯環境
安裝基礎環境及依賴： 1 # yum -y install gcc gcc-c++ make kernel-devel kernel-tools kernel-tools-libs kernel libnl libnl-devel libnfnetlink-devel openssl-devel wget openssh-clients
4.2 安裝Keepalived
1 [[email protected]_master ~]# wget http://www.keepalived.org/software/keepalived-1.3.6.tar.gz 2 [[email protected]_master ~]# tar -zxvf keepalived-1.3.6.tar.gz 3 [[email protected]_master ~]# cd keepalived-1.3.6/ 4 [[email protected] keepalived-1.3.6]# ./configure --prefix=/usr/local/keepalived 5 [[email protected]_master keepalived-1.3.6]# make && make install注意：CentOS6.8安裝高於1.3.6版本會出現未知錯誤。
4.3 新增Keepalived啟動相關服務
1 [[email protected]_master ~]# mkdir /etc/keepalived 2 [[email protected]_master ~]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/ 3 [[email protected]_master ~]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ 4 [[email protected]_master ~]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ 5 [[email protected]_master ~]# vi /etc/init.d/keepalived #建立Keepalived啟動指令碼，見附件 6 [[email protected]_master ~]# chmod u+x /etc/rc.d/init.d/keepalived
4.4 配置Keepalived
1 [[email protected]_master ~]# vi /etc/keepalived/keepalived.conf 2 global_defs { 3 notification_email { 4 #…… 5 } 6 notification_email_from [email protected] 7 smtp_server 192.168.200.1 8 smtp_connect_timeout 30 9 router_id LVS_DEVEL 10 } 11 12 vrrp_script check_haproxy { 13 script "/usr/bin/killall -0 haproxy" 14 interval 2 15 weight 21 16 } 17 18 vrrp_instance HAProxy_HA { 19 state BACKUP #在HAProxy主備均設定為BACKUP 20 interface eth0 21 virtual_router_id 80 22 priority 100 23 advert_int 2 24 nopreempt #不搶佔模式 25 authentication { 26 auth_type PASS 27 auth_pass 1111 28 } 29 30 notify_master "/etc/keepalived/master.sh" 31 notify_backup "/etc/keepalived/backup.sh" 32 notify_fault "/etc/keepalived/fault.sh" 33 34 track_script { 35 check_haproxy 36 } 37 38 virtual_ipaddress { 39 172.24.8.100 dev eth0 40 } 41 }
4.5 編寫Keepalived指令碼
1 [[email protected]_master ~]# vi /etc/keepalived/master.sh 2 #!/bin/bash 3 LOGFILE=/var/log/keepalived-mysql-state/log 4 date >>$LOGFILE 5 echo "[Master]" >>$LOGFILE 6 [[email protected]_master ~]# vi /etc/keepalived/backup.sh 7 #!/bin/bash 8 LOGFILE=/var/log/keepalived-mysql-state/log 9 date >>$LOGFILE 10 echo "[BACKUP]" >>$LOGFILE 11 [[email protected]_master ~]# vi /etc/keepalived/fault.sh 12 #!/bin/bash 13 LOGFILE=/var/log/keepalived-mysql-state/log 14 date >>$LOGFILE 15 echo "[FAULT]" >>$LOGFILE 16 [[email protected]_master ~]# chmod u+x /etc/keepalived/backup.sh 17 [[email protected]_master ~]# chmod u+x /etc/keepalived/master.sh 18 [[email protected]_master ~]# chmod u+x /etc/keepalived/fault.sh提示：本環境採用測試指令碼，真實環境建議採用自動傳送郵件通知運維員的指令碼。
4.5 Bakcup節點配置
1 [[email protected]_master ~]# scp /etc/keepalived/keepalived.conf 172.24.8.11:/etc/keepalived/keepalived.conf #將配置好的Master節點的配置檔案複製到Backup節點 2 [[email protected]_slave ~]# vi /etc/keepalived/keepalived.conf 3 state BACKUP 4 priority 80注意：在HAProxy備節點也設定為BACKUP，priority修改為低於HAProxy主角色的優先順序即可，同時去掉nopreempt。
1 [[email protected]_master ~]# scp /etc/keepalived/*.sh 172.24.8.11:/etc/keepalived/ 2 #將對應的指令碼也複製至backup節點。
五 HAProxy部署
5.1 HAProxy安裝
1 [[email protected]_master ~]# yum -y install haproxy提示：需要在主備HAProxy節點都安裝。
5.2 HAProxy配置
1 [[email protected]_master ~]# vi /etc/haproxy/haproxy.cfg 2 global 3 log 127.0.0.1 local0 info 4 chroot /var/lib/haproxy 5 pidfile /var/run/haproxy.pid 6 maxconn 4096 7 user haproxy 8 group haproxy 9 daemon 10 nbproc 1 11 defaults 12 mode http 13 log global 14 retries 3 15 timeout connect 5s 16 timeout client 30s 17 timeout server 30s 18 timeout check 2s 19 frontend www 20 bind 172.24.8.100:80 21 mode http 22 option httplog 23 option forwardfor 24 log global 25 26 acl host_www hdr_dom(host) -i www.lz.com #配置不同域名分發不同後端策略 27 acl host_static hdr_dom(host) -i static.lz.com 28 acl host_video hdr_dom(host) -i video.lz.com 29 30 use_backend server_www if host_www #配置不同域名分發不同後端策略 31 use_backend server_static if host_static 32 use_backend server_video if host_video 33 backend server_www #後端真是伺服器 34 mode http 35 option redispatch 36 option abortonclose 37 balance roundrobin 38 option httpchk GET /index.html 39 server webapp1 172.24.8.30:80 weight 6 check inter 2000 rise 2 fall 3 40 backend server_static #後端真是伺服器 41 mode http 42 option redispatch 43 option abortonclose 44 balance roundrobin 45 option httpchk GET /index.html 46 server webapp2 172.24.8.31:80 weight 6 check inter 2000 rise 2 fall 3 47 backend server_video #後端真是伺服器 48 mode http 49 option redispatch 50 option abortonclose 51 balance roundrobin 52 option httpchk GET /index.html 53 server webapp3 172.24.8.32:80 weight 6 check inter 2000 rise 2 fall 3 54 [[email protected]_master ~]# scp /etc/haproxy/haproxy.cfg [email protected]:/etc/haproxy/haproxy.cfg #將配置檔案複製至HAProxy備節點
5.3 開啟轉發
1 [[email protected]_master ~]# vi /etc/sysctl.conf 2 net.ipv4.ip_nonlocal_bind = 1 3 [[email protected]_master ~]# sysctl -p注意：繫結非本機的IP必須在sysctl.conf檔案中配置。
六 啟動服務
1 [[email protected] ~]# systemctl start httpd提示：三個節點的httpd服務均啟動。
1 [[email protected]_master ~]# service haproxy start提示：必須先啟動HAProxy服務，之後啟動Keepalived，因為Keepalived會先檢測HAProxy服務程序。
1 [[email protected]_master ~]# service keepalived start
七 驗證測試
7.1 高可用驗證
1 [[email protected]_master ~]# ip add
1 [[email protected]_master ~]# service haproxy stop #停止HAProxy主節點的HAProxy程序 2 [[email protected]_master ~]# tail -f /var/log/messages #觀察HAProxy主節點日誌
1 [[email protected]_slave ~]# ip addr #檢視備HAProxy節點的IP
結論：通過測試可知當主HAProxy節點服務異常，Keepalived會檢測到，同時HAProxy會將vip從主節點移除，備HAProxy會接管。
1 [[email protected]_master ~]# service haproxy start
結論：由於配置了非搶佔模式，主HAProxy恢復程序之後，依舊由備HAProxy提供服務，vip不會切回至主HAProxy。
7.2 負載均衡測試
主機hosts中新增如下解析：1 172.24.8.100 www.lz.com 2 172.24.8.100 static.lz.com 3 172.24.8.100 video.lz.com瀏覽器分別訪問不用的三個域名：
