多機搭建haproxy+rabbitmq叢集
環境說明：
主機1：192.168.0.234 mq01 DISK
主機2：192.168.0.235 mq02 RAM
主機3：192.168.0.236 mq03 RAM
主機4：測試機器，安裝了jdk-8u151-linux-x64，
作業系統：CentOS release 6.9 (Final)
核心版本：2.6.32-696.10.1.el6.x86_64
rabbitmq版本：3.6.11
拓撲圖：
步驟：
1，各個主機上下載安裝rabbitmq（步驟略）
2，配置3臺機器上本地主機名解析
vim /etc/hosts
192.168.0.234 mq01
192.168.0.235 mq02
192.168.0.236 mq03
3，設定3個機器上的節點間統一認證的Erlang Cookie
登入到主機1上傳輸配置檔案：
scp /var/lib/rabbitmq/.erlang.cookie [email protected]:/var/lib/rabbitmq/
scp /var/lib/rabbitmq/.erlang.cookie [email protected]:/var/lib/rabbitmq/
在3個節點上分別執行（確保 Cookie file /var/lib/rabbitmq/.erlang.cookie must be accessible by owner only）：
chmod 600 /var/lib/rabbitmq/.erlang.cookie
4，在3臺主機上使用-detached執行各個節點
在各個主機上啟動rabbitmq-server
rabbitmqctl stop
rabbitmq-server -detached
[[email protected] scripts]# rabbitmqctl cluster_status
[[email protected] scripts]# rabbitmqctl cluster_status
5，建立並部署叢集
主機2上的[email protected]：
rabbitmqctl stop_app
rabbitmqctl join_cluster --ram [email protected]
rabbitmqctl start_app
主機3上的[email protected]：
rabbitmqctl stop_app
rabbitmqctl join_cluster --ram [email protected]
rabbitmqctl start_app
然後在3臺主機上分別執行：
主機1：rabbitmqctl cluster_status
Cluster status of node [email protected]
[{nodes,[{disc,[[email protected]]},{ram,[[email protected],[email protected]]}]},
{running_nodes,[[email protected],[email protected],[email protected]]},
{cluster_name,<<"[email protected]">>},
{partitions,[]},
{alarms,[{[email protected],[]},{[email protected],[]},{[email protected],[]}]}]
主機2：rabbitmqctl cluster_status
Cluster status of node [email protected]
[{nodes,[{disc,[[email protected]]},{ram,[[email protected],[email protected]]}]},
{running_nodes,[[email protected],[email protected],[email protected]]},
{cluster_name,<<"[email protected]">>},
{partitions,[]},
{alarms,[{[email protected],[]},{[email protected],[]},{[email protected],[]}]}]
主機3：rabbitmqctl cluster_status
Cluster status of node [email protected]
[{nodes,[{disc,[[email protected]]},{ram,[[email protected],[email protected]]}]},
{running_nodes,[[email protected],[email protected],[email protected]]},
{cluster_name,<<"[email protected]">>},
{partitions,[]},
{alarms,[{[email protected],[]},{[email protected],[]},{[email protected],[]}]}]
6，配置映象佇列
[email protected]作為備份節點，磁碟節點
在主機2上執行操作：
[[email protected] scripts]# rabbitmqctl -n [email protected] set_policy mytest '.*' '{"ha-mode": "nodes", "ha-params": ["[email protected]",], "ha-sync-mode": "automatic"}'
Setting policy "mytest" for pattern ".*" to "{\"ha-mode\": \"nodes\", \"ha-params\": [\"[email protected]\",], \"ha-sync-mode\": \"automatic\"}" with priority "0"
[[email protected] scripts]# rabbitmqctl -n [email protected] set_policy -p test mytest '.*' '{"ha-mode": "nodes", "ha-params": ["[email protected]",], "ha-sync-mode": "automatic"}'
Setting policy "mytest" for pattern ".*" to "{\"ha-mode\": \"nodes\", \"ha-params\": [\"[email protected]\",], \"ha-sync-mode\": \"automatic\"}" with priority "0"
檢視策略：
主機1：
[[email protected] scripts]# rabbitmqctl list_policies
Listing policies
/ mytest all .* {"ha-mode":"nodes","ha-params":["[email protected]"],"ha-sync-mode":"automatic"} 0
主機2：
[[email protected] scripts]# rabbitmqctl list_policies
Listing policies
/ mytest all .* {"ha-mode":"nodes","ha-params":["[email protected]"],"ha-sync-mode":"automatic"} 0
主機3：
[[email protected] scripts]# rabbitmqctl list_policies
Listing policies
/ mytest all .* {"ha-mode":"nodes","ha-params":["[email protected]"],"ha-sync-mode":"automatic"} 0
7，在3臺主機上啟動rabbitmq的web管理外掛management
[[email protected] scripts]# rabbitmq-plugins enable rabbitmq_management
[[email protected] scripts]# rabbitmq-plugins enable rabbitmq_management
[[email protected] scripts]# rabbitmq-plugins enable rabbitmq_management
8，配置haproxy作為叢集的代理
配置檔案/etc/haproxy/haproxy.cfg
global
log 127.0.0.1 local3 info
maxconn 5120
gid 507
daemon
quiet
nbproc 4
pidfile /var/run/haproxy.pid
defaults
log global
mode tcp
option tcplog
option dontlognull
retries 3
option redispatch
maxconn 5000
contimeout 5s
clitimeout 120s
srvtimeout 120s
listen rmqcluster
bind 0.0.0.0:5672
mode tcp
option tcpka
log 127.0.0.1 local3 info
balance roundrobin
server mq01 192.168.0.234:5672 weight 4 check inter 2000 rise 2 fall 1 backup
server mq02 192.168.0.235:5672 weight 4 check inter 2000 rise 2 fall 1
server mq03 192.168.0.236:5672 weight 4 check inter 2000 rise 2 fall 1
listen rmqmgt
bind 0.0.0.0:15672
mode tcp
option tcpka
log 127.0.0.1 local3 info
balance roundrobin
server mq01 192.168.0.234:15672 weight 4 check inter 2000 rise 2 fall 1 backup
server mq02 192.168.0.235:15672 weight 4 check inter 2000 rise 2 fall 1
server mq03 192.168.0.236:15672 weight 4 check inter 2000 rise 2 fall 1
listen stats
bind 0.0.0.0:8080
mode http
log 127.0.0.1 local3 err
stats enable
stats uri /status
stats refresh 5s
stats auth admin:A9724hxJ
9，關閉叢集
disk節點mq01：
rabbitmqctl stop_app
此時beam.smp程序還在執行；
ram節點mq02：
rabbitmqctl stop_app
此時beam.smp程序還在執行；
ram節點Mq03：
rabbitmqctl stop_app
此時beam.smp程序還在執行；
10，啟動叢集
必須先把磁碟節點啟動起來。一個磁碟節點都沒有執行的時候，直接啟動記憶體節點會報錯如下，出現下面報錯之後上面“9，關閉叢集”中說的beam.smp程序也會掛掉：
[[email protected] ~]# rabbitmqctl start_app
Starting node [email protected]
Error: unable to connect to node [email protected]: nodedown
DIAGNOSTICS
===========
attempted to contact: [[email protected]]
* connected to epmd (port 4369) on mq03
* epmd reports: node 'rabbit' not running at all
no other nodes on mq03
* suggestion: start the node
current node details:
- node name: '[email protected]'
- home dir: /var/lib/rabbitmq
- cookie hash: TI04uDcfMAPrk+JQ82hdCw==
解決辦法：
先啟動磁碟節點，然後再在ram節點主機上的執行/etc/init.d/rabbbitmq-server start來啟動節點，然後再start_app。