1. 程式人生 > >Nginx+keepalived雙機熱備

Nginx+keepalived雙機熱備

nginx ha keepalive

一、nginx簡介

Nginx是一款輕量級的Web 服務器/反向代理服務器及電子郵件(IMAP/POP3)代理服務器,並在一個BSD-like 協議下發行。其特點是占有內存少,並發能力強,事實上nginx的並發能力確實在同類型的網頁服務器中表現較好,中國大陸使用nginx網站用戶有:百度、京東、新浪、網易、騰訊、淘寶等。


二、測試環境

下面拿2臺虛擬機進行環境測試,實驗環境為centos6.6 x86_64,具體用途和ip如下

服務器類型

IP地址

Keepalived vip

192.168.214.70

Nginx1

192.168.214.76

Nginx2

192.168.214.77

三、安裝nginx

這邊給大家使用源碼包來安裝

nginx

groupadd nginx

useradd -g nginx -s /sbin/nologin nginx

mkdir /var/log/nginx&&chown -R nginx:nginx /var/log/nginx

mkdir /usr/local/www&&chown -R nginx:nginx /usr/local/www

安裝nginx所需pcre

wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz

tar zxvf pcre-8.38.tar.gz

cd pcre-8.38

./configure

make && make install

wget http://nginx.org/download/nginx-1.14.0.tar.gz

cd nginx-1.14.0

./configure --prefix=/usr/local/nginx--user=nginx --group=nginx \

--with-http_ssl_module \

--with-http_stub_status_module \

--with-http_flv_module \

--with-http_gzip_static_module

make && make install

ln -s /usr/local/lib/libpcre.so.1 /lib64

使用/usr/local/nginx/sbin/nginx來啟動nginx服務

訪問下默認nginx頁面

技術分享圖片

四、修改linux文件句柄數

使用ulimit -a查看默認為1024

open files (-n) 1024

1、直接在終端修改 ulimit -SHn 65535

2、修改linux系統參數

vi /etc/security/limits.conf 添加保存後,重啟機器就永久生效了

* soft nofile 65535

* hard nofile 65535


五、配置nginx
這邊給大家提供了一個最簡單的配置文件給大家參考


cat /usr/local/nginx/conf/nginx.conf
user  nginx nginx;
worker_processes  8;

error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

pid        /usr/local/nginx/nginx.pid;
worker_rlimit_nofile 65535;

events {
    use epoll;
    worker_connections  65535;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    #charset gb2312;
    server_names_hash_bucket_size 128;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;
    client_max_body_size 8m;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  60;
    tcp_nodelay on;
    gzip  on;
    gzip_min_length 1k;
    gzip_buffers 4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_types text/plain application/x-javascript text/css application/xml;
    gzip_vary on;

    server {
        listen       80;
        server_name  192.168.214.76;
        index index.html index.htm;
        root /usr/local/www;
        #charset koi8-r;
location /nginx_status {
        stub_status on;
        access_log   off;
                }
        access_log  /var/log/nginx/$server_name.log  combined;
}
}

六、安裝keepalived

yum install -y keepalived

chkconfig keepalived on

註:在centos7系列系統中開機自動啟動使用systemctl enable keepalived


七、keepalived文件配置

查看192.168.214.76keepalived配置

cat keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server mail.test.com
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_http_port {                  
script "/usr/local/scripts/nginx_pid.sh"     
interval 2                  
weight 2 
 }  

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
track_interface {
            eth0
  }
track_script {           
chk_http_port          
}      
virtual_ipaddress {         
192.168.214.70     
}  
}


查看192.168.214.77備的keepalived配置

! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server mail.test.com
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_http_port {                  
script "/usr/local/scripts/nginx_pid.sh"                  
interval 2                  
weight 2 
 }

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 99 
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
track_interface {
           eth0
    }
track_script {           
chk_http_port          
}      

virtual_ipaddress {         
192.168.214.70    
}  
}

最後附上Nginx的檢測腳本,當主服務器探測到nginx服務停止後,嘗試開啟nginx服務,失敗後關閉主服務器的keepalived服務,把vip切換到備上

cat nginx_pid.sh 
#!/bin/bash  
A=`ps -C nginx --no-header |wc -l`                 
if [ $A -eq 0 ];then                                                         
               /usr/local/nginx/sbin/nginx                  
              sleep 3                  
              if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then                         
                     killall keepalived                  
              fi  
fi

八、啟動keepalived服務及查看相關信息

192.168.214.76上通過ip addr 查看,vip192.168.214.70已經綁定在eth0網口上了

[root@localhost scripts]# ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

link/ether 52:54:00:55:b2:d4 brd ff:ff:ff:ff:ff:ff

inet 192.168.214.76/24 brd 192.168.214.255 scope global eth0

inet 192.168.214.70/32 scope global eth0

inet6 fe80::5054:ff:fe55:b2d4/64 scope link

valid_lft forever preferred_lft forever

3: eth1:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

link/ether 52:54:00:85:11:95 brd ff:ff:ff:ff:ff:ff

inet 192.168.211.76/24 brd 192.168.211.255 scope global eth1

inet6 fe80::5054:ff:fe85:1195/64 scope link

valid_lft forever preferred_lft forever


214.76上查看日誌信息,看到已成功進入keepalived主機模式

[root@localhost scripts]# tail -f/var/log/messages

May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Registering Kernel netlink command channel

May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Opening file '/etc/keepalived/keepalived.conf'.

May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Configuration is using : 7681 Bytes

May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Using LinkWatch kernel netlink reflector...

May 17 16:29:01 localhost Keepalived_vrrp[1748]:VRRP_Instance(VI_1) Transition to MASTER STATE

May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) EnteringMASTER STATE

May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) setting protocol VIPs.

May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 192.168.214.70

May 17 16:29:02 localhostKeepalived_healthcheckers[1747]: Netlink reflector reports IP 192.168.214.70 added

May 17 16:29:07 localhost Keepalived_vrrp[1748]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.214.70


214.77上查看日誌信息,看到已成功進入keepalived備機模式

May 17 16:29:06 localhostKeepalived_vrrp[22914]: Opening file '/etc/keepalived/keepalived.conf'.

May 17 16:29:06 localhost Keepalived_vrrp[22914]:Configuration is using : 66478 Bytes

May 17 16:29:06 localhostKeepalived_vrrp[22914]: Using LinkWatch kernel netlink reflector...

May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) EnteringBACKUP STATE

May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP sockpool: [ifindex(2), proto(112), unicast(0),fd(10,11)]

May 17 16:29:06 localhostKeepalived_healthcheckers[22912]: Using LinkWatch kernel netlink reflector...


九、keepalived測試

使用vip192.168.214.70訪問nginx

技術分享圖片

最後,我們模擬下192.168.214.76nginx服務宕機,且無法通過腳本恢復,看下vip地址是否會漂移過去,nginx頁面是否能正常訪問。

通過ip addr命令看到vip已漂移到了192.168.214.77nginx服務器上,成功實現了熱備。

[root@localhost scripts]# ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

link/ether 52:54:00:1b:a2:11 brd ff:ff:ff:ff:ff:ff

inet 192.168.214.77/24 brd 192.168.214.255 scope global eth0

inet 192.168.214.70/32 scope global eth0

inet6 fe80::5054:ff:fe1b:a211/64 scope link

valid_lft forever preferred_lft forever

3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP>mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 52:54:00:64:47:7d brd ff:ff:ff:ff:ff:ff

inet 192.168.211.77/24 brd 192.168.211.255 scope global eth1

inet6 fe80::5054:ff:fe64:477d/64 scope link

valid_lft forever preferred_lft forever


214.77上查看日誌信息,看到從之前的備機模式已成功進入keepalived主機模式

[root@localhost scripts]# tail -f/var/log/messages

May 17 16:29:06 localhostKeepalived_vrrp[22914]: Using LinkWatch kernel netlink reflector...

May 17 16:29:06 localhost Keepalived_vrrp[22914]:VRRP_Instance(VI_1) Entering BACKUP STATE

May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP sockpool: [ifindex(2), proto(112), unicast(0),fd(10,11)]

May 17 16:29:06 localhostKeepalived_healthcheckers[22912]: Using LinkWatch kernel netlink reflector...

May 17 16:49:47 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Transitionto MASTER STATE

May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) EnteringMASTER STATE

May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) setting protocol VIPs.

May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for192.168.214.70

May 17 16:49:48 localhostKeepalived_healthcheckers[22912]: Netlink reflector reports IP 192.168.214.70added

May 17 16:49:53 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for192.168.214.70



如果想了解更多,請關註我們的公眾號
公眾號ID:opdevos

掃碼關註

技術分享圖片

Nginx+keepalived雙機熱備