本節內容涉及的saltstack配置以及各服務的安裝包和配置檔案均打包上傳到了百度雲，可自由下載使用

實驗環境（rhel6.5 x86_64bit virtual machine）

172.25.5.91 salt-master rhel65-lockey1
172.25.5.92 salt-minion rhel65-lockey2 （將自動部署keepalived，haproxy）
172.25.5.93 salt-minion rhel65-lockey3 （將自動部署keepalived，haproxy）
172.25.5.94 salt-minion rhel65-lockey4 （將自動部署nginx）
172.25.5.95 salt-minion rhel65-lockey5 （將自動部署nginx）
 

172.25.5.100 physical machine(for testing)

一、準備工作

1. 做好各主機的解析

vim /etc/hosts

172.25.5.91  rhel65-lockey1
172.25.5.92  rhel65-lockey2
172.25.5.93  rhel65-lockey3
172.25.5.94  rhel65-lockey4
172.25.5.95  rhel65-lockey5

2. 各主機需要關閉selinux以及防火牆設定

3. 在master端執行salt-key -L發現所有minion，並且通過salt-key -A命令執行認證，得到下面結果：

[[email protected] ~]# salt-key -L

4. master端取消以下注釋並重啟服務生效：

file_roots:
  base:
    - /srv/salt

pillar_roots:
  base:
    - /srv/pillar

二、此高可用負載均衡叢集組成

高可用：keepalived
負載均衡：haproxy
後端web server：NGINX

三、Get start

salt自動化部署叢集目錄結構

3.1 pillar目錄

為了結合jinja模板實現一些配置檔案中引數的動態調整（如keepalived中MASTER與BACKUP以及優先順序的值）
 

[[email protected] pillar]# tree

.
├── top.sls
└── webservice
    └── web.sls

1 directory, 2 files

[[email protected] pillar]# cat webservice/web.sls

{% if grains['host'] == 'rhel65-lockey2' %}
state: MASTER
priority: 100
{% elif grains['host'] == 'rhel65-lockey3' %}
state: BACKUP
priority: 50
{% endif %}

[[email protected] pillar]# cat top.sls

base:
  'rhel65-lockey2':
    - webservice.web
  'rhel65-lockey3':
    - webservice.web

執行命令 salt ‘*’ saltutil.refresh_pillar使得選定主機動態引數設定生效

[[email protected] pillar]# salt ‘*’ pillar.items#檢視結果

3.2 _grains目錄

為服務主機新增role設定，用以根據role標籤決定主機需要安裝的軟體

[[email protected] _grains]# pwd
/srv/salt/_grains
[[email protected] _grains]# cat my_grains.py

#!/usr/bin/env python

def my_grains():
    grains =  {}
    grains['roles'] = 'nginx'
    grains['name'] = 'lockey'
    return grains

[[email protected] salt]# salt rhel65-lockey[5,4] saltutil.sync_grains
#為2和4號主機新增nginx的role，後面部署的時候將通過這個標籤匹配進行nginx的安裝

rhel65-lockey4:
    - grains.my_grains
rhel65-lockey5:
    - grains.my_grains

[[email protected] salt]# salt rhel65-lockey[5,4] grains.item roles#檢視結果

rhel65-lockey4:
    ----------
    roles:
        nginx
rhel65-lockey5:
    ----------
    roles:
        nginx

3.3 haproxy目錄:

 haproxy安裝、服務配置檔案、啟動指令碼和安裝包

[[email protected] haproxy]# cat install.sls

include:
  - pkg.nginx-pre
  - user.haproxy

haproxy-install:
  file.managed:
    - name: /mnt/haproxy-1.6.11.tar.gz
    - source: salt://haproxy/files/haproxy-1.6.11.tar.gz

  cmd.run:
    - name: cd /mnt && tar zxf haproxy-1.6.11.tar.gz && cd haproxy-1.6.11 && make TARGET=linux26 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 PREFIX=/usr/local/haproxy && make TARGET=linux26 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 PREFIX=/usr/local/haproxy install
    - creates: /usr/local/haproxy


/etc/haproxy:
  file.directory:
    - mode: 755

/etc/haproxy/haproxy.cfg:
  file.managed:
    - source: salt://haproxy/files/haproxy.cfg

/etc/init.d/haproxy:
  file.managed:
    - source: salt://haproxy/files/haproxy
    - mode: 755

[[email protected] haproxy]# cat service.sls

include:
  - haproxy.install

haproxy-service:
  service.running:
    - name: haproxy
    - enable: true
    - reload: true
    - watch:
      - file: /etc/haproxy/haproxy.cfg

3.4 keepalived目錄

keepalived安裝、服務配置檔案、啟動指令碼和安裝包

[[email protected] keepalived]# cat install.sls

include:
  - pkg.nginx-pre

keepalived-install:
  file.managed:
    - name: /mnt/keepalived-1.3.5.tar.gz
    - source: salt://keepalived/files/keepalived-1.3.5.tar.gz

  cmd.run:
    - name: cd /mnt && tar zxf keepalived-1.3.5.tar.gz && cd keepalived-1.3.5 && ./configure --prefix=/usr/local/keepalived --with-init=SYSV && make && make install
    - creates: /usr/local/keepalived

/etc/sysconfig/keepalived:
  file.managed:
    - source: salt://keepalived/files/keepalived-sys
    - mode: 644
    - user: root

/etc/keepalived:
  file.directory:
    - mode: 755

/sbin/keepalived:
  file.symlink:
    - target: /usr/local/keepalived/sbin/keepalived

/etc/keepalived/keepalived.conf:
  file.managed:
    - source: salt://keepalived/files/keepalived.conf
    - mode: 644
    - user: root
    - group: root
    - template: jinja###注意jinja模板的用法
    - context:
       state: {{ pillar['state'] }}
       priority: {{ pillar['priority'] }}

/etc/init.d/keepalived:
  file.managed:
    - source: salt://keepalived/files/keepalived
    - mode: 755
    - require:
      - file: /etc/sysconfig/keepalived
      - file: /etc/keepalived/keepalived.conf

[[email protected] keepalived]# cat service.sls

include:
  - keepalived.install

keepalived-service:
  service.running:
    - name: keepalived
    - enable: true
    - reload: true
    - watch:
      - file: /etc/keepalived/keepalived.conf

3.5 nginx目錄

nginx安裝以及服務配置檔案和啟動指令碼、安裝包

[[email protected] nginx]# cat install.sls

include:
  - pkg.nginx-pre

nginx-source-install:
  file.managed:
   - name: /mnt/nginx-1.12.1.tar.gz
   - source: salt://nginx/files/nginx-1.12.1.tar.gz

  cmd.run:
   - name: cd /mnt && tar zxf nginx-1.12.1.tar.gz && cd nginx-1.12.1 && sed -i.bak 's/#define NGINX_VER          "nginx\/" NGINX_VERSION/#define NGINX_VER          "nginx"/g' src/core/nginx.h && sed -i.bak 's/CFLAGS="$CFLAGS -g"/#CFLAGS="$CFLAGS -g"/g' auto/cc/gcc && ./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-http_stub_status_module && make && make install
   - creates: /usr/local/nginx

[[email protected] nginx]# cat service.sls

include:
  - nginx.install
  - user.nginx

/usr/local/nginx/conf/nginx.conf:
  file.managed:
    - source: salt://nginx/files/nginx.conf
    - mode: 644

/etc/init.d/nginx:
  file.managed:
    - source: salt://nginx/files/nginx
    - mode: 755

nginx-service:
  service.running:
    - name: nginx
    - enable: true
    - reload: true
    - require:
      - file: /etc/init.d/nginx
    - watch:
      - file: /usr/local/nginx/conf/nginx.conf

3.6 pkg目錄

為各軟體安裝解決依賴包問題，需要包含在最先執行的安裝檔案中

[[email protected] pkg]# cat nginx-pre.sls

pkg-init:
  pkg.installed:
    - pkgs:
      - gcc
      - gcc-c++
      - zlib-devel
      - openssl-devel
      - pcre-devel

3.7 top.sls

頂級檔案，用來進行叢集的分節點軟體安裝

[[email protected] salt]# cat top.sls

base:
  'rhel65-lockey2':
    - keepalived.service
    - haproxy.service
  'rhel65-lockey3':
    - keepalived.service
    - haproxy.service
  'roles:nginx':
    - match: grain
    - nginx.service

3.8 user目錄

建立部分服務執行的使用者和組（如haproxy，nginx）

[[email protected] salt]# cat user/nginx.sls

nginx:
  user.present:
    - uid: 800
    - shell: /sbin/nologin
    - home: /usr/local/nginx
    - createhome: false

[[email protected] salt]# cat user/haproxy.sls

haproxy:
  group.present:
    - gid: 200
  user.present:
    - uid: 200
    - gid: 200
    - shell: /sbin/nologin
    - home: /usr/local/haproxy
    - createhome: false

各服務的配置檔案以及啟動指令碼就不貼出了，下面提出幾點需要注意的地方：

1.jinja模板呼叫的格式以及對應的檔案中的寫法

     - template: jinja#此部分對應keepalived.conf
        - context:
           state: {{ pillar['state'] }}
           priority: {{ pillar['priority'] }}

keepalived.conf 中呼叫時寫法如下

state {{ state }}
priority {{ priority }}

2.對於pillar和grains檔案寫成後，必須要對特定的主機進行標籤的重新整理，保證最後執行時能夠對應進行軟體安裝：

salt rhel65-lockey[4,5] saltutil.sync_grains
salt ‘*’ saltutil.refresh_pillar

3. 對於haproxy的配置檔案，務必指定後端主機以及監聽：

frontend public
        bind          www.lockey.com *:80
        default_backend dynamic

backend dynamic
        balance         roundrobin
        server          web1 172.25.5.94:80 cookie s1 check inter 1000
        server          web2 172.25.5.95:80 cookie s2 check inter 1000

4.top檔案的書寫

[[email protected] salt]# cat top.sls

base:
  'rhel65-lockey2':#2和3主機安裝keepalived和haproxy
    - keepalived.service
    - haproxy.service
  'rhel65-lockey3':
    - keepalived.service
    - haproxy.service
  'roles:nginx':#具有nginx標籤的主機（4和5）安裝nginx服務
    - match: grain
    - nginx.service

5. 確保後端主機的web服務埠80未被佔用，以及selinux和防火牆

四、進行整體自動安裝測試然後執行

4.1 首先測試一下，如果有問題解決後再執行安裝

[[email protected] salt]# salt ‘*’ state.highstate test=true
注意：如果提示keepalived/haproxy/nginx服務無法啟動的錯誤是正常的，因為啟動指令碼還未推送過去

4.2 測試順利則執行推送開始自動安裝：

 [[email protected] salt]# salt ‘*’ state.highstate

五、叢集可用性測試

1. 檢視vip是否在keepalived定義的MASTER上

停掉master的keepalived服務之後發生ip漂移

因為叢集部署成功之後是通過VIP(172.25.5.99)提供服務的，我在物理機中加了一條解析記錄然後通過curl命令進行負載均衡測試：
vim /etc/hosts

172.25.5.99 www.lockey.com

成功結果如下：