騰訊訊息佇列CMQ部署與驗證
阿新 • • 發佈:2021-01-09
### 環境
| IP | 備註 |
| ------------- | ------------- |
| 192.168.1.66 | node1 前置機 |
| 192.168.1.110 | node2 |
| 192.168.1.202 | node3 |
### 架構圖
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109175126709-593542179.png)
### 元件介紹
| 元件 | 監聽埠 |
| ---------- | ----------------------- |
| access | 12000/120001 |
| pushserver | 50012 |
| nameserver | 80 |
| broker | 11011/11012/11013/11014 |
| barad | 11112 |
| ctsdb | 9201 |
| oss | 80 |
| master | 9992 |
| moni | 9999 |
| accessGW | 80 |
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109175144068-1203292320.png)
CMP依賴VIP:
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109175157372-70005186.png)
### 準備工作
```
3臺8*16伺服器
安裝文件:
物料包(元件物料包和python指令碼) :https://cmq-1255613487.cos.ap-chengdu.myqcloud.com/install_cmq/CMQ-ALL.zip
```
### 安裝[192.168.1.66] 前端元件:
```
#安裝
#[root@node_66 ~]# ls
CMQ-1.0.2-軟體包.tar.gz
mkdir -p /opt/tmp
tar xf CMQ-1.0.2-軟體包.tar
cp -r v1.0.0/cmq_console/precomponent/* /opt/tmp/
#安裝cmq_console元件
cd /opt/tmp/
#執行部署[填寫內網IP]
sh pre_component_install.sh 192.168.1.66 192.168.1.66
#---------------------------------------------------------------------------
#安裝 auth access 元件
#將 cmq_console/auth-access 目錄下所有檔案拷貝到 /opt/tmp 目錄下,並進入此目錄
cp -r /root/v1.0.0/cmq_console/auth-access/* /opt/tmp
sh access_auto_install.sh 192.168.1.66 192.168.1.66
#提示:
#backup script to /opt/script
#cp: cannot stat ‘./*.md’: No such file or directory 報錯忽略
#安裝cmq_backserver
```
cp -r /root/v1.0.0/cmq_backserver /opt/tmp
cd /opt/tmp/cmq_backserver
```
## 指令碼內容 vim 1.sh
#-------------------------------------------------------------------#
cat >1.sh<sendmsg.txt
其中queueName=test 就是佇列名稱:queue
所以需要改為
# 再修改batchsendmsg.txt的queueName和appid
# 執行壓測,nameserver-ip進行替換 ,命令不能執行請先安裝工具: yum install -y httpd-tools
# 壓測時應該選擇 barad 的IP 進行壓測,grep barad /opt/tmp/cmq_backserver/config.txt
ab -k -n 500000 -c 500 -p sendmsg.txt -T application/json "http://192.168.1.202/v2/index.php"
appid查詢:
http://192.168.1.66/cmq-manager/queue?rid=1&userAppId=1
appId=1
Name=queue
```
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109175949596-225168496.png)
清空佇列:
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109180004845-1931665964.png)
### 解除安裝ACCESS
```
######################################################
#!/bin/sh
# Host Login Info
user=`sed '/^user=/!d;s/.*=//' $1 | sed 's/\r//g'`
passwd=`sed '/^passwd=/!d;s/.*=//' $1 | sed 's/\r//g'`
sshPort=`sed '/^sshPort=/!d;s/.*=//' $1 |sed 's/\r//g'`
# DB Info
dbhost=`sed '/^dbhost=/!d;s/.*=//' $1 | sed 's/\r//g'`
dbport=`sed '/^dbport=/!d;s/.*=//' $1 | sed 's/\r//g'`
dbuser=`sed '/^dbuser=/!d;s/.*=//' $1 | sed 's/\r//g'`
dbpasswd=`sed '/^dbpasswd=/!d;s/.*=//' $1 | sed 's/\r//g'`
# Component IP List
cmq_nameserver=`sed '/^routeProxy=/!d;s/.*=//' $1 | sed 's/\r//g'`
cmq_access=`sed '/^access=/!d;s/.*=//' $1 | sed 's/\r//g'`
cmq_moni=`sed '/^monitor=/!d;s/.*=//' $1 | sed 's/\r//g'`
#cmq_moni2=`sed '/^monitor2=/!d;s/.*=//' $1 | sed 's/\r//g'`
cmq_master=`sed '/^master=/!d;s/.*=//' $1 | sed 's/\r//g'`
#cmq_master2=`sed '/^master2=/!d;s/.*=//' $1 | sed 's/\r//g'`
cmq_oss=`sed '/^oss=/!d;s/.*=//' $1 | sed 's/\r//g'`
cmq_publisher=`sed '/^pushServer=/!d;s/.*=//' $1 | sed 's/\r//g'`
cmq_broker=`sed '/^broker=/!d;s/.*=//' $1 | sed 's/\r//g'`
# Barad Info
barad=`sed '/^barad=/!d;s/.*=//' $1 | sed 's/\r//g'`
ctsdb=`sed '/^ctsdb=/!d;s/.*=//' $1 | sed 's/\r//g'`
jvmMem=`sed '/^jvmMem=/!d;s/.*=//' $1 | sed 's/\r//g'`
indexTtl=`sed '/^indexTtl=/!d;s/.*=//' $1 | sed 's/\r//g'`
metricTtl=`sed '/^metricTtl=/!d;s/.*=//' $1 | sed 's/\r//g'`
yum=$barad" "$ctsdb
baradList=${barad// /,}
ctsdbList=${ctsdb// /,}
# Some Vip
nameserverVip=`sed '/^nameserverVip=/!d;s/.*=//' $1 | sed 's/\r//g'`
baradIp=`sed '/^baradIp=/!d;s/.*=//' $1 | sed 's/\r//g'`
ctsdbIp=`sed '/^ctsdbIp=/!d;s/.*=//' $1 | sed 's/\r//g'`
ctsdbVport=`sed '/^ctsdbVport=/!d;s/.*=//' $1 | sed 's/\r//g'`
# Directory Of Component
targetDir=`sed '/^targetDir=/!d;s/.*=//' $1 | sed 's/\r//g'`
packageTmpDir='/opt/cmqTmp/'
packageDir=`pwd`'/package'
for moduleName in cmq_oss cmq_moni cmq_master cmq_broker cmq_access cmq_nameserver cmq_publisher yum barad ctsdb;
#for moduleName in cmq_access;
#for moduleName in barad;
#for moduleName in cmq_publisher;
#for moduleName in cmq_moni;
#for moduleName in ctsdb;
#for moduleName in cmq_master;
#for moduleName in cmq_nameserver;
#for moduleName in cmq_oss;
#for moduleName in cmq_broker;
do
iplist=`eval echo '$'"$moduleName"`
for host in $iplist;
do
./onlineUninstall.sh $host $user $passwd $packageDir $packageTmpDir $targetDir$moduleName $moduleName $sshPort $dbhost $dbport $dbuser $dbpasswd $cmq_moni $cmq_master $baradIp $nameserverVip $jvmMem $indexTtl $metricTtl $baradList $ctsdbList $ctsdbIp $ctsdbVport
done
done
######################################################
註釋: for moduleName in cmq_oss cmq_moni cmq_master cmq_broker cmq_access cmq_nameserver cmq_publisher yum barad ctsdb; 行
取消註釋: #for moduleName in cmq_access; 行 改為 for moduleName in cmq_access;
修改結果:
#for moduleName in cmq_oss cmq_moni cmq_master cmq_broker cmq_access cmq_nameserver cmq_publisher yum barad ctsdb;
for moduleName in cmq_access;
#執行解除安裝操作:
sh uninstall_all.sh config.txt
```
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109180021109-277151851.png)
### 錯誤問題重現:
```
#解除安裝了一個元件然後檢視日誌和執行結果
cd /root/v1.0.0/SDK/sdk/simple_demo
python ./cmq_sample_topic.py
```
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109180036131-540014741.png)
進入日誌`/data/cmq_nameserver/log`
```
192.168.1.202伺服器進入目錄監控日誌:
cd /data/cmq_access/log/
[root@node_202 log]# tailf /data/cmq_nameserver/log/*
2021/01/09 17:28:29 requestId: 8649737934421390396, clientIP: 192.168.1.66, scheme: http, arg: clientRequestId: 1231231231; appId:1; Action: PublishMessage; topicName: topic;
2021/01/09 17:28:29 ResponseVipVport, url:http://192.168.1.202:12001, req.host: 192.168.1.202
2021/01/09 17:28:29 ResponseVipVport, clinent_ip: 192.168.1.66, ipHash: 3232279594, proxy_idx: 194
2021/01/09 17:28:29 ResponseVipVport, addr: http://192.168.1.202:12001 has found in vipProxyArr
2021/01/09 17:28:29 $$$, ResponseVipVport, Dial error: dial tcp 192.168.1.202:12001: connect: connection refused
2021/01/09 17:28:29 http: proxy error: dial tcp 192.168.1.202:12001: connect: connection refused
2021/01/09 17:28:29 ResponseVipVport finish request, requestId: 8649737934421390396
```
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109180048613-1161145349.png)
### 恢復元件:
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109180103795-528128728.png)
```
註釋:
# ./cmq_db_init.sh $dbhost $dbport $dbuser $dbpasswd
# for moduleName in cmq_oss cmq_moni cmq_master cmq_broker cmq_access cmq_nameserver cmq_publisher yum barad ctsdb;
第一行註釋是為了不重新初始化資料庫,避免資料丟失
第二行不重灌所有元件
啟用:
for moduleName in cmq_access;
```
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109180116437-842090045.png)
```
#執行安裝元件:
sh install_all.sh config.txt
驗證傳送訊息:
cd /root/v1.0.0/SDK/sdk/simple_demo
python ./cmq_sample_topic.py
```
### 驗證恢復後的元件是否正常
![](https://img2020.cnblogs.com/blog/1362790/202101/1362790-20210109180129512-3157937