1. 程式人生 > >Keepalived案例一:Keepalived雙機熱備(HA)精講

Keepalived案例一:Keepalived雙機熱備(HA)精講

這裡我們僅僅只利用Keepalive做雙機熱備,也就是保證伺服器的高可用性,其他的不用管。可能您會說這樣在實際應用中很少會這樣用,這您可就錯了,Keepalived僅僅做雙機熱備的情況還是有的,我就碰到過幾次這樣的案例,下面就我碰到的幾個案例做個小結

一,Keepalived雙機熱備的應用場景

1,網站流量不高,壓力不大,但是對伺服器的可靠性要求極其高,例如實時線上OA系統,政府部門網站系統,醫院實時報醫系統,公安局線上報案系統,股市後臺網站系統等等,他們的壓力不是很大,但是對可靠性要求是非常高的

2,有錢沒地方花的,典型的政府企業,公辦學校等等

二,Keepalived雙機熱備的特性以及優缺點

特性:

1,至少需要兩臺伺服器,其中一臺為master始終提供服務,另外一臺作為backup始終處於空閒狀態,只有在主伺服器掛掉的時候他就來幫忙了,這是典型的雙擊熱備

2,能根據需求判斷服務是否可用,在不可用的時候要即使切換
優缺點:

優點:資料同步非常簡單,不像負載均衡對資料一致性要求非常高,實現起來相對複雜維護也頗為不便,雙機熱備用rsync就可以實現了操作和維護非常簡單

缺點:伺服器有點浪費,始終有一臺處於空閒狀態


三,Keepalived雙機熱備的配置
首先畫個雙機熱備拓撲圖吧:



這裡我只寫最終實現的配置,至於Keepalived的理論知識請參考《Keepalived原理與實戰精講

1,本例通過Keepalived來實現兩臺LNMP(也就是linux+nginx+mysql+php)架構伺服器的雙機熱備


LNMP的配置請參考:《Lnmp配置精講第一版

2,Keepalived配置雙機安裝配置

1》Keepalived安裝

keepalived官方地址:http://www.keepalived.org/download.html,大家可以到這裡下載最新版本的keepalived

作業系統:centos 5.5 32bit
系統安裝:最小化安裝,也就是去掉所有元件
環境配置:安裝make 和 gcc openssl openssl-devel等等
  1. yum -y install gcc make openssl openssl-devel wget kernel-devel
  2. mkdir -p /usr/local/src/hasoft
  3. cd /usr/local/src/hasoft
  4. wget http://www.keepalived.org/software/keepalived-1.2.2.tar.gz
  5. tar -zxvf keepalived-1.2.2.tar.gz
  6. cd keepalived-1.2.2
  7. ./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.18-238.19.1.el5-i686/
複製程式碼 預編譯後出現:
  1. Keepalived configuration
  2. ------------------------
  3. Keepalived version       : 1.2.2
  4. Compiler                 : gcc
  5. Compiler flags           : -g -O2 -DETHERTYPE_IPV6=0x86dd
  6. Extra Lib                : -lpopt -lssl -lcrypto
  7. Use IPVS Framework       : Yes
  8. IPVS sync daemon support : Yes
  9. IPVS use libnl           : No
  10. Use VRRP Framework       : Yes
  11. Use Debug flags          : No
複製程式碼
  1. make && make install
複製程式碼 這裡注意哦,我上面是指通用的安裝方法,如果你沒有用到LVS可以把lvs去掉即
./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.18-238.19.1.el5-i686/ --disable-lvs-syncd --disable-lvs

但這個沒有影響,就按照我的來配置吧,不過如果你要是集成了LVS,那麼就不可加這兩個引數了哦

整理管理檔案:
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/

建立配置檔案目錄(注意:keepalived的配置檔案預設在/etc/keepalived/目錄)
mkdir -p /etc/etc/keepalived/

兩臺伺服器(兩個節點)都這樣安裝即可

2》配置

節點A配置如下:
vi /etc/keepalived/keepalived.conf
  1. global_defs
  2. {
  3. notification_email
  4. {
  5. [email protected]
  6. [email protected]
  7. }
  8. notification_email_from [email protected]
  9. smtp_server 127.0.0.1
  10. stmp_connect_timeout 30
  11. router_id lnmp_node1
  12. }
  13. vrrp_instance lnmp {
  14. state MASTER
  15. interface eth0
  16. virtual_router_id 100
  17. priority 200
  18. advert_int 5
  19. track_interface {
  20. eth0
  21. eth1
  22. }
  23. authentication {
  24. auth_type PASS
  25. auth_pass 123456
  26. }
  27. virtual_ipaddress {
  28. 192.168.17.200
  29. }
  30. }
複製程式碼
節點B配置如下:
vi /etc/keepalived/keepalived.conf

  1. global_defs
  2. {
  3. notification_email
  4. {
  5. [email protected]
  6. [email protected]
  7. }
  8. notification_email_from [email protected]
  9. smtp_server 127.0.0.1
  10. stmp_connect_timeout 30
  11. router_id lnmp_node1
  12. }
  13. vrrp_instance lnmp {
  14. state MASTER
  15. interface eth0
  16. virtual_router_id 100
  17. priority 150
  18. advert_int 5
  19. track_interface {
  20. eth0
  21. eth1
  22. }
  23. authentication {
  24. auth_type PASS
  25. auth_pass 123456
  26. }
  27. virtual_ipaddress {
  28. 192.168.17.200
  29. }
  30. }
複製程式碼 四,啟動除錯
在節點A上啟動
/usr/local/keepalived/sbin/keepalived

啟動日誌:
Sep  8 18:26:02 centosa Keepalived_vrrp: Registering Kernel netlink reflector
Sep  8 18:26:02 centosa Keepalived_vrrp: Registering Kernel netlink command channel
Sep  8 18:26:02 centosa Keepalived_vrrp: Registering gratutious ARP shared channel
Sep  8 18:26:02 centosa Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Sep  8 18:26:02 centosa Keepalived_vrrp: Configuration is using : 36076 Bytes
Sep  8 18:26:02 centosa Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
Sep  8 18:26:02 centosa Keepalived: Starting VRRP child process, pid=5606
Sep  8 18:26:07 centosa Keepalived_vrrp: VRRP_Instance(lnmp) Transition to MASTER STATE
Sep  8 18:26:12 centosa Keepalived_vrrp: VRRP_Instance(lnmp) Entering MASTER STATE
Sep  8 18:26:12 centosa avahi-daemon[2528]: Registering new address record for 192.168.17.200 on eth0.


在節點B上啟動
/usr/local/keepalived/sbin/keepalived

開機自動啟動
echo /usr/local/keepalived/sbin/keepalived >> /etc/rc.local

啟動日誌:
Sep  8 18:30:02 centosb Keepalived: Starting Keepalived v1.2.2 (09/08,2011)
Sep  8 18:30:02 centosb Keepalived: Starting Healthcheck child process, pid=5837
Sep  8 18:30:02 centosb Keepalived_vrrp: Registering Kernel netlink reflector
Sep  8 18:30:02 centosb Keepalived_vrrp: Registering Kernel netlink command channel
Sep  8 18:30:02 centosb Keepalived_vrrp: Registering gratutious ARP shared channel
Sep  8 18:30:02 centosb Keepalived: Starting VRRP child process, pid=5839
Sep  8 18:30:02 centosb kernel: IPVS: Registered protocols (TCP, UDP, AH, ESP)
Sep  8 18:30:02 centosb kernel: IPVS: Connection hash table configured (size=4096, memory=32Kbytes)
Sep  8 18:30:02 centosb kernel: IPVS: ipvs loaded.
Sep  8 18:30:02 centosb Keepalived_healthcheckers: Registering Kernel netlink reflector
Sep  8 18:30:02 centosb Keepalived_healthcheckers: Registering Kernel netlink command channel
Sep  8 18:30:02 centosb Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Sep  8 18:30:02 centosb Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Sep  8 18:30:02 centosb Keepalived_vrrp: Configuration is using : 36252 Bytes
Sep  8 18:30:02 centosb Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
Sep  8 18:30:02 centosb Keepalived_healthcheckers: Configuration is using : 6271 Bytes
Sep  8 18:30:02 centosb Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
Sep  8 18:30:02 centosb Keepalived_vrrp: VRRP_Instance(lnmp) Entering BACKUP STATE

從日誌可以看出,啟動都沒有問題,並且安裝我給的優先順序完成了競選,各自成就了各自的狀態

關閉節點A的網絡卡測試切換是否正常
ifdown eth0

觀察節點B的日誌:
Sep  8 18:32:55 centosb Keepalived_vrrp: VRRP_Instance(lnmp) Transition to MASTER STATE
Sep  8 18:33:00 centosb Keepalived_vrrp: VRRP_Instance(lnmp) Entering MASTER STATE
Sep  8 18:33:00 centosb avahi-daemon[2531]: Registering new address record for 192.168.17.200 on eth0.


啟動節點A的網絡卡測試切換是否正常
ifup eth0
觀察節點B的日誌:
Sep  8 18:33:31 centosb Keepalived_vrrp: VRRP_Instance(lnmp) Received higher prio advert
Sep  8 18:33:31 centosb Keepalived_vrrp: VRRP_Instance(lnmp) Entering BACKUP STATE
Sep  8 18:33:31 centosb avahi-daemon[2531]: Withdrawing address record for 192.168.17.200 on eth0.

Received higher prio advert:表示接收到更高優先順序的公告(advert公告的意思)
Withdrawing:撤回的意思,可以看出切換過程一目瞭然


OK,到這裡我們的安裝部分完成,下面我們來看看如何監控服務吧,我們這裡僅僅是監控了網路故障和keepalived本身程序,在網路或者keepalived程序出現問題的時候會切換,但是我的節點A裡面還有很多服務呢,例如nginx,PHP,mysql程序出問題或高負載的時候相應過慢怎麼辦,怎麼切換的呢,這時就要用到指令碼了,下面我們來看看keepalived是如何控制指令碼來實現對伺服器的監控和切換的

寫個指令碼來實時監控三個服務,若有一個出現問題遍切換mkdir /root/shell/
cd /root/shell
vi keepcheck.sh
  1. #!/bin/bash
  2. while  :
  3. do
  4. mysqlcheck=`/usr/local/lnmp/mysql/bin/mysqladmin -uroot ping 2>&1`
  5. mysqlcode=`echo $?`
  6. phpcheck=`ps -C php-fpm --no-header | wc -l`
  7. nginxcheck=`ps -C nginx --no-header | wc -l`
  8. keepalivedcheck=`ps -C keepalived --no-header | wc -l`
  9. if [ $nginxcheck -eq 0 ]|| [ $phpcheck -eq 0 ]||[ $mysqlcode -ne 0 ];then
  10.                 if [ $keepalivedcheck -ne 0 ];then
  11.                    killall -TERM keepalived
  12.                 else
  13.                    echo "keepalived is stoped"
  14.                 fi
  15.         else
  16.                 if [ $keepalivedcheck -eq 0 ];then
  17.                    /etc/init.d/keepalived start
  18.                 else
  19.                    echo "keepalived is running"
  20.                 fi
  21. fi
  22. sleep 5
  23. done
複製程式碼 注意,用/etc/init.d/keepalived start如果起不來,可以用/usr/local/keepalived/sbin/keepalived二進位制檔案直接執行啟動即可
啟動指令碼:
  1. chmod +x /root/shell/keepcheck.sh
  2. nohup sh /root/shell/keepcheck.sh &
複製程式碼 節點B也用這個指令碼

寫入/etc/rc.local開機自動啟動

  1. echo "nohup sh /root/shell/keepcheck.sh &" >> /etc/rc.loal
複製程式碼

可以測試了

開了防火牆之後雙節點都變成master了,日誌如下
Sep 13 21:21:27 centosb avahi-daemon[2528]: Withdrawing address record for fe80::20c:29ff:fede:99ab on eth1.
Sep 13 21:21:27 centosb avahi-daemon[2528]: Withdrawing address record for 192.168.27.212 on eth1.
Sep 13 21:21:27 centosb avahi-daemon[2528]: Withdrawing address record for fe80::20c:29ff:fede:99a1 on eth0.
Sep 13 21:21:27 centosb avahi-daemon[2528]: Withdrawing address record for 192.168.17.212 on eth0.
Sep 13 21:21:27 centosb avahi-daemon[2528]: Host name conflict, retrying with <centosb-48>
Sep 13 21:21:27 centosb avahi-daemon[2528]: Registering new address record for fe80::20c:29ff:fede:99ab on eth1.
Sep 13 21:21:27 centosb avahi-daemon[2528]: Registering new address record for 192.168.27.212 on eth1.
Sep 13 21:21:27 centosb avahi-daemon[2528]: Registering new address record for fe80::20c:29ff:fede:99a1 on eth0.
Sep 13 21:21:27 centosb avahi-daemon[2528]: Registering new address record for 192.168.17.200 on eth0.
Sep 13 21:21:27 centosb avahi-daemon[2528]: Registering new address record for 192.168.17.212 on eth0.

解決方法如下:(一般使用第二種情況)

第一種情況,如果用的是預設防火牆
只需要新增:iptables -I RH-Firewall-1-INPUT -d 224.0.0.18 -j ACCEPT

第二種情況:如果是自己用指令碼設定的防火牆,只需要新增西門規則即可
iptables -A OUTPUT -o eth0-d 224.0.0.18 -j ACCEPT
iptables -A OUTPUT -o eth0  -s 224.0.0.18 -j ACCEPT
iptables -A INPUT -i eth0-d 224.0.0.18 -j ACCEPT
iptables -A INPUT -i eth0  -s 224.0.0.18 -j ACCEPT

修改完後,記得使用/etc/rc.d/init.d/iptables save 儲存修改的規則,並會將規則寫入/etc/sysconfig/iptables檔案中去,否則重啟後會失效。

在/etc/sysconfig/iptables檔案中,要將INPUT鏈和OUTPUT鏈中的REJECT規則放到最後,否則會影響上面設定的功能。

最後,使用service iptables restart ,重啟iptables使規則生效。


轉自 http://bbs.nanjimao.com/thread-855-1-1.html