1. 程式人生 > >基於HAProxy+Keepalived高可用負載均衡web服務的搭建

基於HAProxy+Keepalived高可用負載均衡web服務的搭建

1.2 epo cnblogs oba backup 保持 ica mysql redis

一 原理簡介

1.HAProxy
HAProxy提供高可用性、負載均衡以及基於TCP和HTTP應用的代理,支持虛擬主機,它是免費、快速並且可靠的一種解決方案。HAProxy特別適用於那些負載特大的web站點,這些站點通常又需要會話保持或七層處理。HAProxy運行在時下的硬件上,完全可以支持數以萬計的並發連接。並且它的運行模式使得它可以很簡單安全的整合進當前的架構中, 同時可以保護web服務器不被暴露到網絡上。
2.Keepalived
Keepalived 是一個基於VRRP協議來實現的LVS服務高可用方案,可以利用其來避免單點故障。一個LVS服務會有2臺服務器運行Keepalived,一臺為主服務器(MASTER),一臺為備份服務器(BACKUP),但是對外表現為一個虛擬IP,主服務器會發送特定的消息給備份服務器,當備份服務器收不到這個消息的時候,即主服務器宕機的時候,備份服務器就會接管虛擬IP,繼續提供服務,從而保證了高可用性。Keepalived是VRRP的完美實現。
3.vrrp協議


在現實的網絡環境中,兩臺需要通信的主機大多數情況下並沒有直接的物理連接。對於這樣的情況,它們之間路由怎樣選擇?主機如何選定到達目的主機的下一跳路由,這個問題通常的解決方法有兩種:
在主機上使用動態路由協議(RIP、OSPF等)
在主機上配置靜態路由
很明顯,在主機上配置動態路由是非常不切實際的,因為管理、維護成本以及是否支持等諸多問題。配置靜態路由就變得十分流行,但路由器(或者說默認網關default gateway)卻經常成為單點故障。VRRP的目的就是為了解決靜態路由單點故障問題,VRRP通過一競選(election)協議來動態的將路由任務交給LAN中虛擬路由器中的某臺VRRP路由器。


二 運行環境

1.系統
CentOS Linux release 7.2.1511 (Core)
2.應用軟件
haproxy-1.5.14-3.el7.x86_64
keepalived-1.2.13-7.el7.x86_64
httpd-2.4.6-45.el7.centos.x86_64
php-5.4.16-42.el7.x86_64
mariadb-server-5.5.52-1.el7.x86_64
mariadb.x86_64
wordpress-4.3.1-zh_CN.zip
nfs-utils-1.3.0-0.33.el7.x86_64
rpcbind-0.2.0-38.el7.x86_64
3.IP配置
負載均衡器
DIP1:172.18.67.13
DIP2:172.18.67.14
後端Real Server
RIP1:172.18.67.11
RIP2:172.18.67.12
數據庫服務器
MIP:172.18.67.1
客戶端IP
IP:172.18.67.3
VIP:172,.18.67.33


三 架構拓撲及應用軟件安裝

1.拓撲圖

技術分享
2.部署應用軟件
在IP為172.18.67.13與172.18.67.14的服務器上安裝部署haproxy、keepalived

[[email protected] ~]# yum install haproxy keepalived -y
[[email protected] ~]# yum install haproxy keepalived -y


在IP為172.18.67.11的服務器上安裝部署httpd、php

[[email protected] ~]# yum install httpd php -y


在IP為172.18.67.12的服務器上安裝部署httpd、php、mariadb-server、php-mysql

[[email protected] ~]# yum install httpd php mariadb mariadb-server php-mysql -y


在IP為172.18.67.1的服務器上部署mariadb、mariadb-server、php-mysql

[[email protected] ~]# yum install mariadb mariadb-server php-mysql -y


另外由於數據庫采用文件共享的方式,所以還要安裝nfs,分別在Real Server端和數據庫服務器端安裝nfs的應用軟件nfs-utils、rpcbind

[[email protected] ~]# yum install nfs-utils rpcbind -y
[[email protected] ~]# yum install nfs-utils rpcbind -y

四 配置

1.Real Server配置
在這裏我們將IP為172.18.67.11的服務器設置為動態資源站,將IP為172.18.67.12的服務器設置成靜態資源棧:將wordpress應用解壓至/var/www/html/下,並修改該目錄的屬主和屬組

[[email protected] ~]# unzip wordpress-4.3.1-zh_CN.zip -C /var/www/html/
[[email protected] ~]# chown -R apache:apache /var/www/html/wordpress
[[email protected] ~]# unzip wordpress-4.3.1-zh_CN.zip -C /var/www/html/
[[email protected] ~]# chown -R apache:apache /var/www/html/wordpress


修改Server1的httpd的配置文件將網站目錄從默認的/var/www/html修改為/var/www/html/wordpress,Server2的網站目錄默認。
2.nfs配置
數據庫端修改配置文件

[[email protected] ~]# vim /etc/exports
/data/ 172.18.67.11(rw,async)
/var/www/html/ 172.18.67.12(rw,async)


修改保存完畢後啟動Real Server和數據庫服務器的nfs應用

[[email protected] ~]# systemctl start rpcbind
[[email protected] ~]# systemctl start nfs
[[email protected] ~]# systemctl start rpcbind
[[email protected] ~]# systemctl start nfs
[[email protected] ~]# systemctl start rpcbind
[[email protected] ~]# systemctl start nfs


創建數據存放目錄及修改權

[[email protected] ~]# mkdir /data
[[email protected] ~]# chown -R mysql:mysql /data


修改數據庫配置文件數據存儲目錄

[[email protected] ~]# vim /etc/my.cnf
datadir=/data/


啟動數據庫

[[email protected] ~]# systemctl start mysql-server


將數據庫目錄掛載至web動態資源服務器

[[email protected] ~]# mount -t nfs 172.18.67.1:/data/ /mnt


3.keepalived配置
MASTER

[[email protected] keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs {
    notification_email {
 [email protected]
    }
    notification_email_from [email protected]
    smtp_server 127.0.0.1 #郵件本地地址
    smtp_connect_timeout 30 #連接超時
    router_id inode2
    vrrp_mcast_group4 224.0.67.67 #多播地址
}
vrrp_script chk_down {
    script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0" #檢測keepalived服務是否啟用
    interval 1
    weight -5 #如果檢測到down,則權重減去5
}
vrrp_script chk_nginx {
    script "killall -0 nginx && exit 0 || exit 1" #向nginx發信號檢測nginx進程是否存在
    interval 1
    weight -5
    fall 2
    rise 1
}
vrrp_instance myr {
    state MASTER
    interface eno16777736
    virtual_router_id 167
    priority 100 #優先級
    advert_int 1
    authentication {
 auth_type PASS
 auth_pass 571f97b2
    }
    virtual_ipaddress {
 172.18.67.33/16 dev eno16777736
    }
    track_script {
 chk_down
 chk_nginx
    }
    notify_master "/etc/keepalived/notify.sh master" #啟用腳本通知功能
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}


BACKUP

[[email protected] keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs {
    notification_email {
 [email protected]
    }
    notification_email_from [email protected]
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id inode3
    vrrp_mcast_group4 224.0.67.67
}
vrrp_script chk_down {
    script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
    interval 1
    weight -5
}
vrrp_script chk_nginx {
    script "killall -0 nginx && exit 0 || exit 1"
    interval 1
    weight -5
    fall 2
    rise 1
}
vrrp_instance myr {
    state BACKUP
    interface eno16777736
    virtual_router_id 167
    priority 95
    advert_int 1
    authentication {
 auth_type PASS
 auth_pass 571f97b2
    }
    virtual_ipaddress {
 172.18.67.33/16 dev eno16777736
    }
    track_script {
 chk_down
 chk_nginx
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

通知腳本

#!/bin/bash
#
[email protected]

notify() {
	mailsubject="$(hostname) to be $1, vip floating"
	mailbody="$(date +‘%F %T‘): vrrp transition, $(hostname) changed to be $1"
	echo "$mailbody" | mail -s "$mailsubject" $contact
}

case $1 in
master)
	notify master
	;;
backup)
	notify backup
	;;
fault)
	notify fault
	;;
*)
	echo "Usage: $(basename $0) {master|backup|fault}"
	exit 1
	;;
esac


4.haproxy配置
兩節點的配置內容是一樣的,如下:

[[email protected] haproxy]# vim haproxy.cfg
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    stats socket /var/lib/haproxy/stats
defaults
    mode                    http #啟用七層模型
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000
frontend          web *:80
    mode            http
    maxconn          2000 #最大連接數量
    acl url_static     path_beg       -i  /static /images /javascript /stylesheets #啟用acl
    acl url_static     path_end       -i  .jpg .gif .png .css .js .html .txt .htm
    use_backend staticsrvs  if url_static
    default_backend    appsrvs
backend staticsrvs #靜態資源
    balance      roundrobin #輪循算法
    server       stcsrvs 172.18.67.12:80 check
backend appsrvs #動態資源
    balance      roundrobin
    server       wp 172.18.67.11:80 check
listen stats
    bind :10086 #綁定端口
    stats   enable #啟用stats功能
    stats   uri     /admin?stats #設置非默認uri
    stats   auth    admin:admin #認證賬號密碼
    stats   admin   if TRUE

五 啟動服務並測試

1.啟動haproxy和keepalived

[[email protected] ~]# systemctl restart haproxy
[[email protected] ~]# systemctl restart keepalived
[[email protected] ~]# systemctl restart haproxy
[[email protected] ~]# systemctl restart keepalived


2.測試
inode2:

[[email protected] ~]# systemctl status -l keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2017-05-17 20:53:58 CST; 10min ago
  Process: 2156 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 2157 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─2157 /usr/sbin/keepalived -D
           ├─2158 /usr/sbin/keepalived -D
           └─2159 /usr/sbin/keepalived -D
May 17 20:53:58 inode2 Keepalived_healthcheckers[2158]: Opening file ‘/etc/keepalived/keepalived.conf‘.
May 17 20:53:58 inode2 Keepalived_healthcheckers[2158]: Configuration is using : 7521 Bytes
May 17 20:53:58 inode2 Keepalived_healthcheckers[2158]: Using LinkWatch kernel netlink reflector...
May 17 20:53:59 inode2 Keepalived_vrrp[2159]: VRRP_Instance(myr) Transition to MASTER STATE
May 17 20:54:00 inode2 Keepalived_vrrp[2159]: VRRP_Instance(myr) Entering MASTER STATE
May 17 20:54:00 inode2 Keepalived_vrrp[2159]: VRRP_Instance(myr) setting protocol VIPs.
May 17 20:54:00 inode2 Keepalived_vrrp[2159]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736 for 172.18.67.33
May 17 20:54:00 inode2 Keepalived_vrrp[2159]: Opening script file /etc/keepalived/notify.sh
May 17 20:54:00 inode2 Keepalived_healthcheckers[2158]: Netlink reflector reports IP 172.18.67.33 added
May 17 20:54:05 inode2 Keepalived_vrrp[2159]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736 for 172.18.67.33


我們看到inode2節點進入了MASTER模式,再查看inode3的狀態

[[email protected] ~]# systemctl status -l keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2017-05-17 21:55:22 CST; 2s ago
  Process: 20030 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 19577 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─19577 /usr/sbin/keepalived -D
           ├─19578 /usr/sbin/keepalived -D
           └─19579 /usr/sbin/keepalived -D

May 17 21:55:22 inode3 systemd[1]: Starting LVS and VRRP High Availability Monitor...
May 17 21:55:22 inode3 Keepalived[20030]: Starting Keepalived v1.2.13 (11/20,2015)
May 17 21:55:22 inode3 Keepalived[20030]: daemon is already running
May 17 21:55:22 inode3 systemd[1]: Started LVS and VRRP High Availability Monitor.


不難看出inode3節點處於BACKUP狀態,此時我們將inode2的keepalived服務停掉

[[email protected] ~]# systemctl stop keepalived
[[email protected] ~]# systemctl status -l keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2017-05-17 20:54:37 CST; 12min ago
  Process: 1653 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 1654 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─1654 /usr/sbin/keepalived -D
           ├─1655 /usr/sbin/keepalived -D
           └─1656 /usr/sbin/keepalived -D

May 17 20:54:38 inode3 Keepalived_healthcheckers[1655]: Registering Kernel netlink command channel
May 17 20:54:38 inode3 Keepalived_healthcheckers[1655]: Opening file ‘/etc/keepalived/keepalived.conf‘.
May 17 20:54:38 inode3 Keepalived_healthcheckers[1655]: Configuration is using : 7519 Bytes
May 17 20:54:38 inode3 Keepalived_healthcheckers[1655]: Using LinkWatch kernel netlink reflector...
May 17 21:06:34 inode3 Keepalived_vrrp[1656]: VRRP_Instance(myr) Transition to MASTER STATE
May 17 21:06:35 inode3 Keepalived_vrrp[1656]: VRRP_Instance(myr) Entering MASTER STATE
May 17 21:06:35 inode3 Keepalived_vrrp[1656]: VRRP_Instance(myr) setting protocol VIPs.
May 17 21:06:35 inode3 Keepalived_vrrp[1656]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736 for 172.18.67.33
May 17 21:06:35 inode3 Keepalived_vrrp[1656]: Opening script file /etc/keepalived/notify.sh
May 17 21:06:35 inode3 Keepalived_healthcheckers[1655]: Netlink reflector reports IP 172.18.67.33 added


我們發現inode3節點進入了MASTER狀態,因此體現出了高可用的特性
接下來我們測試haproxy的特性,在haproxy的配置文件裏有下面這一段

listen stats
    bind :10086
    stats   enable
    stats   uri     /admin?stats
    stats   auth    admin:admin
    stats   admin   if TRUE


這段配置可以讓我們在瀏覽器中查看和修改haproxy統計接口啟用相關的參數,在瀏覽器中輸入http://172.18.67.33:10086/admin?stats,就會出現下面這種狀態,輸入賬號和密碼,就進入了haproxy相關參數配置頁面。

技術分享
在下圖中可以看出負載均衡的兩臺web服務器一臺負責動態資源解析,另一臺負責靜態資源。動態資源的數據存放於後端的nfs服務器上。

技術分享
接下來我們在瀏覽器中訪問http://172.18.67.33就可以安裝wordpress了。至此一個簡單的高可用負載均衡服務搭建完畢。

技術分享

基於HAProxy+Keepalived高可用負載均衡web服務的搭建