1. 程式人生 > >利用saltstack一鍵部署高可用負載均衡叢集

利用saltstack一鍵部署高可用負載均衡叢集

實驗環境:
Server1 172.25.254.1 maseter/minion keepalived/haproxy
Server2 172.25.254.2 minion httpd
Server3 172.25.254.3 minion nginx
Server4 172.25.254.4 minion keepalived/haproxy

配置一臺新的server4進行配置salt-minion:

[[email protected] ~]# yum install salt-minion -y     ## 配置完整yum源
[[email protected]
~]# cd /etc/salt/ [[email protected] salt]# ls cloud cloud.maps.d master minion.d proxy.d cloud.conf.d cloud.profiles.d master.d pki roster cloud.deploy.d cloud.providers.d minion proxy [[email protected] salt]# vim minion 填寫master端的IP master: 172.25.254.1 [
[email protected]
salt]# /etc/init.d/salt-minion start 開啟服務 Starting salt-minion:root:server4 daemon: OK

交換公鑰,新增server4到server1master端

[[email protected] files]# salt-key -A  新增公鑰匹配
The following keys are going to be accepted:
Unaccepted Keys:
server4
Proceed? [n/Y] y
Key for minion server4 accepted.
[
[email protected]
files]# salt-key -L 查詢新增成功 Accepted Keys: server1 server2 server3 server4 Denied Keys: Unaccepted Keys: Rejected Keys:

配置高可用模組keepalived:

[[email protected] ~]# cd /srv/salt/
[[email protected] salt]# ls
_grains  haproxy  httpd  nginx  pkgs  top.sls  users
[[email protected] salt]# mkdir keepalived
[[email protected] salt]# cd keepalived/
[[email protected] keepalived]# ls
[[email protected] keepalived]# vim install.sls
[[email protected] keepalived]# cat install.sls 
kp-install:
  file.managed:
    - name: /mnt/keepalived-2.0.6.tar.gz
    - source: salt://keepalived/files/keepalived-2.0.6.tar.gz
[[email protected] keepalived]# mkdir files
[[email protected] keepalived]# cd files/
[[email protected] files]# ls
[[email protected] files]# pwd
/srv/salt/keepalived/files
[[email protected] files]# ls
keepalived-2.0.6.tar.gz

在server1上推送keepalived

[[email protected] files]# salt server4 state.sls keepalived.install 

推送成功結果:在server4的/mnt/目錄下可以看到推送過來的安裝包

[[email protected] salt]# cd /mnt/
[[email protected] mnt]# ls
keepalived-2.0.6.tar.gz

在server1完善keepalived的安裝指令碼進行推送:

[[email protected] keepalived]# cd ..
[[email protected] salt]# ls
_grains  haproxy  httpd  keepalived  nginx  pkgs  top.sls  users
[[email protected] salt]# cd pkgs/
[[email protected] pkgs]# ls
make.sls
[[email protected] pkgs]# cat make.sls      ## 原始碼編譯一些依賴性的軟體包
make-gcc:
  pkg.installed:
    - pkgs:
      - pcre-devel
      - openssl-devel
      - gcc
[[email protected] pkgs]# cd ..
[[email protected] salt]# cd keepalived/
[[email protected] keepalived]# ls
files  install.sls
[[email protected] keepalived]# vim install.sls 
[[email protected] keepalived]# cat install.sls 
include:    
  - pkgs.make   匯入工具包

kp-install:
  file.managed:
    - name: /mnt/keepalived-2.0.6.tar.gz
    - source: salt://keepalived/files/keepalived-2.0.6.tar.gz
  cmd.run:   進行原始碼編譯不顯示過程資訊
    - name: cd /mnt && tar zxf keepalived-2.0.6.tar.gz && cd keepalived-2.0.6 && ./configure --prefix=/usr/local/keepalived --with-init=SYSV &> /dev/null && make &> /dev/null && make install &> /dev/null
    - creates: /usr/local/keepalived
[[email protected] keepalived]# salt server4 state.sls keepalived.install   進行推送在server4安裝keepalived

安裝過程中在server4可以檢視相應程序

[[email protected] mnt]# ps ax

在server4的對應安裝位置上已經有了keepalived

[[email protected] mnt]# ls
keepalived-2.0.6  keepalived-2.0.6.tar.gz
[[email protected] mnt]# ll /usr/local/keepalived/
total 16
drwxr-xr-x 2 root root 4096 Aug 18 10:42 bin
drwxr-xr-x 5 root root 4096 Aug 18 10:42 etc
drwxr-xr-x 2 root root 4096 Aug 18 10:42 sbin
drwxr-xr-x 5 root root 4096 Aug 18 10:42 share

配置Keepalived的高可用
將server4的keepalived的配置檔案/usr/local/keepalived/etc/keepalived/keepalived.conf和啟動指令碼/usr/local/keepalived/etc/rc.d/init.d/keepalived傳送到server1的/srv/salt/keepalived/files目錄下

[[email protected] mnt]# cd /usr/local/keepalived/etc/rc.d/init.d/
[[email protected] init.d]# scp keepalived server1:/srv/salt/keepalived/files  ## keepalived的啟動指令碼

[[email protected] ~]# cd /usr/local/keepalived/etc/keepalived/
[[email protected] keepalived]# ls
keepalived.conf  samples
[[email protected] keepalived]# scp keepalived.conf server1:/srv/salt/keepalived/files    ## 配置檔案

在server1配置安裝指令碼製作相應的軟連結:

[[email protected] ~ ]# cd  /srv/salt/keepalived
[[email protected] keepalived]# ls
files  install.sls
[[email protected] keepalived]# vim install.sls 
[[email protected] keepalived]# cat install.sls 
include:
  - pkgs.make

kp-install:
  file.managed:
    - name: /mnt/keepalived-2.0.6.tar.gz
    - source: salt://keepalived/files/keepalived-2.0.6.tar.gz
  cmd.run:
    - name: cd /mnt && tar zxf keepalived-2.0.6.tar.gz && cd keepalived-2.0.6 && ./configure --prefix=/usr/local/keepalived --with-init=SYSV &> /dev/null && make &> /dev/null && make install &> /dev/null
    - creates: /usr/local/keepalived

/etc/keepalived:
  file.directory:
    - mode: 755    指令碼的許可權

/etc/sysconfig/keepalived:   製作軟連結
  file.symlink:
    - target: /usr/local/keepalived/etc/sysconfig/keepalived

/sbin/keepalived:     製作軟連結
  file.symlink:
    - target: /usr/local/keepalived/sbin/keepalived
[[email protected] keepalived]# salt server4 state.sls keepalived.install  進行推送


在server4可以看到對應的軟連結代表推送成功:
在這裡插入圖片描述

在serevr1安裝開啟keepalived服務的指令碼:

[[email protected] keepalived]# pwd
/srv/salt/keepalived
[[email protected] keepalived]# vim service.sls 
[[email protected] keepalived]# cat service.sls 
include:
  - keepalived.install    匯入安裝指令碼

/etc/keepalived/keepalived.conf:
  file.managed:
    - source: salt://keepalived/files/keepalived.conf

kp-service:
  file.managed:
    - name: /etc/init.d/keepalived
    - source: salt://keepalived/files/keepalived
    - mode: 755
  service.running:
    - name: keepalived
    - reload: True
    - watch:
      - file: /etc/keepalived/keepalived.conf
[[email protected] keepalived]# cd /srv/pillar/   進去pillar模組
[[email protected] pillar]# ls
top.sls  web
[[email protected] pillar]# cd web/
[[email protected] web]# ls
install.sls
[[email protected] web]# vim install.sls 
[[email protected] web]# cat install.sls 
{% if grains['fqdn'] == 'server2' %}    如果主機名字為server2
webserver: httpd
bind: 172.25.254.2
port: 80
{% elif grains['fqdn'] == 'server3' %}
webserver: nginx
{% endif %}
[[email protected] web]# cd ..
[[email protected] pillar]# ls
top.sls  web
[[email protected] pillar]# mkdir keepalived
[[email protected] pillar]# cd keepalived/
[[email protected] keepalived]# ls
[[email protected] keepalived]# cp ../web/install.sls .

[[email protected] keepalived]# vim install.sls
[[email protected] keepalived]# cat install.sls 
{% if grains['fqdn'] == 'server1' %}
state: MASTER   寫入狀態MASTER還是BACKUP測試高可用
vrid: 38
priority: 100   寫入優先順序
{% elif grains['fqdn'] == 'server4' %}
state: BACKUP
vrid: 38
priority: 50
{% endif %}
[[email protected] keepalived]# ls
install.sls
[[email protected] keepalived]# cd ..
[[email protected] pillar]# ls
keepalived  top.sls  web
[[email protected] pillar]# vim top.sls 
[[email protected] pillar]# cat top.sls 
base:
  '*':
    - web.install
    - keepalived.install
[[email protected] pillar]# cd ..
[[email protected] srv]# cd salt/
[[email protected] salt]# ls
_grains  haproxy  apache  keepalived  nginx  pkgs  top.sls  users
[[email protected] salt]# cd keepalived/
[[email protected] keepalived]# ls
files  install.sls  service.sls

配置keppalived的一鍵安裝並開啟服務加入優先順序:

[[email protected] keepalived]# vim service.sls 
[[email protected] keepalived]# cat service.sls 
include:
  - keepalived.install

/etc/keepalived/keepalived.conf:
  file.managed:
    - source: salt://keepalived/files/keepalived.conf:
    - template: jinja
    - context:
      STATE: {{ pillar['state'] }}
      VRID: {{ pillar['vrid'] }}
      PRIORITY: {{ pillar['priority'] }}

kp-service:
  file.managed:
    - name: /etc/init.d/keepalived
    - source: salt://keepalived/files/keepalived
    - mode: 755
  service.running:
    - name: keepalived
    - reload: True
    - watch:
      - file: /etc/keepalived/keepalived.conf
[[email protected] keepalived]# vim files/keepalived
[[email protected] keepalived]# ls
files  install.sls  service.sls
[[email protected] keepalived]# cd files/
[[email protected] files]# ls
keepalived  keepalived-2.0.6.tar.gz  keepalived.conf
[[email protected] files]# vim keepalived.conf    編輯主配置檔案寫入虛擬IP
[[email protected] keepalived]# salt server4 state.sls keepalived.service   推送service服務:

在server4可以看到對應的配置檔案還有指令碼檢視有相應程序:
在server4檢視已經接管虛擬IP:

[[email protected] keepalived]# ip addr
2: eth0: 
    inet 172.25.254.4/24 brd 172.25.254.255 scope global eth0
    inet 172.25.254.100/32 scope global eth0

在server1編寫top.sls指令碼在對應節點安裝服務:

[[email protected] keepalived]# pwd
/srv/salt/keepalived
[[email protected] keepalived]# cd ..
[[email protected] salt]# ls
_grains  haproxy  apache  keepalived  nginx  pkgs  top.sls  users
[[email protected] salt]# vim top.sls 
[[email protected] salt]# cat top.sls 
base:
  'server1':
    - haproxy.install
    - keepalived.service
  'server4':
    - haproxy.install
    - keepalived.service
  'roles:apache':
    - match: grain
    - apache.service
  'roles:nginx':
    - match: grain
    - nginx.service
[[email protected] salt]# salt '*' state.highstate  
在server4檢視已經有相應的haproxy和keepalived的程序:

[[email protected] keepalived]# ps ax
[[email protected] keepalived]# ip addr  虛擬IP已經轉換到server1
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:52:c1:00 brd ff:ff:ff:ff:ff:ff
    inet 172.25.254.4/24 brd 172.25.254.255 scope global eth0
    inet6 fe80::5054:ff:fe52:c100/64 scope link 
       valid_lft forever preferred_lft forever

在網頁測試後端可以負載均衡:

在這裡插入圖片描述

在這裡插入圖片描述
將server1的keepalived服務關閉:

[[email protected] salt]# /etc/init.d/keepalived stop
Stopping keepalived:                                       [  OK  ]

在網頁再次測試後端依舊可以負載均衡

server4接管虛擬IP成為新的master:

 [[email protected] keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:52:c1:00 brd ff:ff:ff:ff:ff:ff
    inet 172.25.254.4/24 brd 172.25.254.255 scope global eth0
    inet 172.25.254.100/32 scope global eth0
    inet6 fe80::5054:ff:fe52:c100/64 scope link 
       valid_lft forever preferred_lft forever

server1再次開啟keepalived會搶回來master的身份:

[[email protected] salt]# /etc/init.d/keepalived start
Starting keepalived:                                       [  OK  ]
[[email protected] salt]# ip addr   已經接管虛擬IP
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:58:4d:1f brd ff:ff:ff:ff:ff:ff
    inet 172.25.38.1/24 brd 172.25.38.255 scope global eth0
    inet 172.25.38.100/32 scope global eth0
    inet6 fe80::5054:ff:fe58:4d1f/64 scope link 
       valid_lft forever preferred_lft forever

呼叫指令碼控制高可用

[[email protected] files]# cd /opt/
[[email protected] opt]# ls
[[email protected] opt]# vim check_haproxy.sh
[[email protected] opt]# cat check_haproxy.sh   指令碼意思就是當haproxy服務出現故障時就停止keepalived
#!/bin/bash

/etc/init.d/haproxy status  &> /dev/null || /etc/init.d/haproxy restart &> /dev/null

if [ $? -ne 0 ];then
/etc/init.d/keepalived stop &> /dev/null
fi
[[email protected] opt]# chmod +x check_haproxy.sh   給指令碼賦予執行許可權
[[email protected] opt]# /etc/init.d/haproxy status
haproxy (pid  2877) is running...
[[email protected] opt]# echo $?
0
[[email protected] opt]# cd /etc/keepalived/
[[email protected] keepalived]# ls
keepalived.conf
[[email protected] opt]#scp check_haproxy.sh server4:/opt/  將指令碼傳遞到server4,必須新增可執行許可權不然指令碼沒辦法執行

[[email protected] keepalived]# vim /srv/salt/keepalived/files/keepalived.conf   
[[email protected] keepalived]# cat /srv/salt/keepalived/files/keepalived.conf 
! Configuration File for keepalived

vrrp_script check_haproxy {
        script "/opt/check_haproxy.sh"    同樣新增指令碼的呼叫
        interval 2
        weight  2
}

global_defs {
   notification_email {
        [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
  # vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state {{ STATE }}
    interface eth0
    virtual_router_id {{ VRID }}
    priority {{ PRIORITY }}
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100
    }
 track_script{
        check_haproxy
    }
}

進行推送測試:

[[email protected] keepalived]# salt server4 state.sls keepalived.service  

在server1減去haproxy指令碼的執行許可權,指令碼生效相應的keepalived服務會關閉:

[[email protected] ~]# cd /etc/init.d/
[[email protected] init.d]# ls
auditd            halt       keepalived    netconsole  rdisc        rsyslog      saslauthd
blk-availability  haproxy    killall       netfs       restorecond  salt-master  single
crond             ip6tables  lvm2-lvmetad  network     rhnsd        salt-minion  sshd
functions         iptables   lvm2-monitor  postfix     rhsmcertd    sandbox      udev-post
[[email protected] init.d]# chmod -x haproxy 
[[email protected] init.d]# /etc/init.d/keepalived status
keepalived is stopped

MASTER會轉換到serevr4,直接的效果就是server4接管虛擬IP

[[email protected] opt]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:52:c1:00 brd ff:ff:ff:ff:ff:ff
    inet 172.25..254.4/24 brd 172.25.254.255 scope global eth0
    inet 172.25.254.100/32 scope global eth0
    inet6 fe80::5054:ff:fe52:c100/64 scope link 
       valid_lft forever preferred_lft forever

驗證高可用

[[email protected] init.d]# ls
auditd            halt       keepalived    netconsole  rdisc        rsyslog      saslauthd
blk-availability  haproxy    killall       netfs       restorecond  salt-master  single
crond             ip6tables  lvm2-lvmetad  network     rhnsd        salt-minion  sshd
functions         iptables   lvm2-monitor  postfix     rhsmcertd    sandbox      udev-post
[[email protected] init.d]# chmod +x haproxy  重新增加可執行許可權
[[email protected] init.d]# ll haproxy 
-rwxr-xr-x 1 root root 2298 Jul 10  2013 haproxy
[[email protected] init.d]# /etc/init.d/keepalived status
keepalived is stopped
[[email protected] init.d]# /etc/init.d/keepalived start  開啟keepalived服務
Starting keepalived:                                       [  OK  ]
[[email protected] init.d]# ip addr   server1依舊會搶回來MASTER的角色進行接管虛擬IP
[[email protected] init.d]# salt server1 service.start keepalived  開啟keepalived服務
server1:
    True
[[email protected] init.d]# ls
auditd            halt       keepalived    netconsole  rdisc        rsyslog      saslauthd
blk-availability  haproxy    killall       netfs       restorecond  salt-master  single
crond             ip6tables  lvm2-lvmetad  network     rhnsd        salt-minion  sshd
functions         iptables   lvm2-monitor  postfix     rhsmcertd    sandbox      udev-post
[[email protected] init.d]# cd
[[email protected] ~]# salt-cp '*' /etc/passwd /tmp   將/etc/passwd檔案同步到所有主機的/tmp目錄下

在server2檢視已經推送成功,拿server2舉例子,不一一進行檢視後端:

[[email protected] ~]# cd /tmp/
[[email protected] tmp]# ls
passwd  yum.log  yum_save_tx-2018-08-17-09-30GSgtBm.yumtx
[[email protected] ~]# salt '*' cmd.run 'rm -f /tmp/passwd'   呼叫salt命令刪除傳遞過去的passwd檔案 
server4:
server2:
server3:
server1:
[[email protected] ~]# cd /tmp/  在server1檢視已經被刪除
[[email protected] tmp]# ls
yum.log

在server2檢視passwd檔案已經被刪除:

[[email protected] tmp]# ls
yum.log  yum_save_tx-2018-08-17-09-30GSgtBm.yumtx
[[email protected] tmp]# salt server3 state.single pkg.installed httpd  直接呼叫模組命令安裝apache

在serevr3檢視httpd已經被安裝:

[[email protected] ~]# rpm -q httpd
httpd-2.2.15-29.el6_4.x86_64