1. 程式人生 > >Nginx線上服務狀態下平滑升級或新增模組的詳細操作記錄

Nginx線上服務狀態下平滑升級或新增模組的詳細操作記錄

今天,產品那邊發來需求,說有個 APP 的 IOS 版本下載包需要新增 https 協議,在景安購買了免費的 SSL 證書。當我往 nginx 上新增 ssl 時,發現伺服器上的 nginx 居然沒編譯 SSL 模組!

看了下舊版本 nginx 的 configure 選項:

1 linux-gz215:# /usr/local/sbin/nginx -V
2 nginx version: nginx/1.0.11
3 built by gcc 4.1.2 20070115 (prerelease) (SUSE Linux)
4 configure arguments: --prefix=/usr/local/nginx

可能是出於最小化安裝的考慮,就只有一個 prefix 引數,而版本也挺低的,乾脆就升級一下好了!由於伺服器處於線上服務狀態,為了避免升級帶來的不良影響,我決定給 nginx 來個平滑升級,結果發現還真是如絲般順滑。。。

下面記錄一下平滑升級和新增模組的過程。

一、半自動平滑升級

所謂半自動,其實就是在最後遷移的時候使用原始碼自帶的升級命令:make upgrade 來自動完成。

①、按需編譯新版本的 nginx

根據需求,常規編譯新版本 nginx,不過只要執行到 make 就打住,不要 make install!

#下載1.5.7版本,並解壓
cd /usr/local/src
wget http://nginx.org/download/nginx-1.6.0.tar.gz
tar zxvf nginx-1.6.0.tar.gz
cd nginx-1.6.0

#根據實際需要新增的模組,先準備所需檔案(其實只需要解壓即可,全部安裝,後面編譯就可以不指定路徑了):
#1. 安裝pcre:
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.34.tar.gz 
tar -zxvf pcre-8.34.tar.gz
cd pcre-8.34
./configure && make && make install

#2. 安裝zlib:
cd /usr/local/src
wget http://zlib.net/zlib-1.2.8.tar.gz
tar -zxvf zlib-1.2.8.tar.gz
cd zlib-1.2.8
./configure && make && make install

#3. 安裝openssl:
cd /usr/local/src
wget http://www.openssl.org/source/openssl-1.0.1c.tar.gz
tar -zxvf openssl-1.0.1c.tar.gz
cd openssl-1.0.1c
./configure && make && make install

#加上所需引數開始編譯:
./configure --user=www --group=www \
--prefix=/usr/local/nginx \
--with-http_ssl_module \
--with-openssl=/usr/local/src/openssl-1.0.1c \ #對應openssl原始碼解壓後的路徑,下同(pcre,zlib)
--with-http_stub_status_module \
--with-pcre \
--with-pcre=/usr/local/src/pcre-8.21 \
--with-zlib=/usr/local/src/zlib-1.2.8

#執行make編譯,但是不要執行make install

make

②、重新命名 nginx 舊版本二進位制檔案,即 sbin 目錄下的 nginx(期間 nginx 並不會停止服務!):

linux-gz215:/usr/local/src/nginx-1.6.0 # mv /usr/local/nginx/sbin/nginx /usr/local/nginx/sbin/nginx.old

③、然後拷貝一份新編譯的二進位制檔案:

linux-gz215:/usr/local/src/nginx-1.6.0 # cp objs/nginx /usr/local/nginx/sbin/

④、在原始碼目錄執行 make upgrade 開始升級:

linux-gz215:/usr/local/src/nginx-1.6.0 # make upgrade
 
#下面是make upgrade命令的列印資訊:
/usr/local/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
kill -USR2 `cat /usr/local/nginx/logs/nginx.pid`
sleep 1
test -f /usr/local/nginx/logs/nginx.pid.oldbin
kill -QUIT `cat /usr/local/nginx/logs/nginx.pid.oldbin`
 
#最後確認一下nginx程序,可以發現有2個主程序,並且有正在關閉的程序(shutting down):
linux-gz215:/usr/local/src/nginx-1.6.0 # ps aux | grep nginx
root 969 0.0 0.3 8260 1844 ? Ss Dec09 0:01 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
www 4196 0.1 2.5 19112 12872 ? S 14:52 0:00 nginx: worker process is shutting down
www 4260 0.1 2.5 19112 12872 ? S 14:52 0:00 nginx: worker process is shutting down
www 4257 0.1 2.5 19112 12872 ? S 14:52 0:00 nginx: worker process is shutting down
root 4663 0.0 0.3 5488 1900 ? S 14:58 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
 
#過一段時間後,再次確認nginx程序,可以發現老程序已自動退出了(存在一段時間是因為舊程序還有未結束的服務)
root 969 0.0 0.3 8260 1844 ? Ss Dec09 0:01 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
www       4665  0.1  2.4  16508 12444 ?        S    14:58   0:01 nginx: worker process

完成後,最後確認一下 nginx -V :

linux-gz215:~ # /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.6.0
built by gcc 4.1.2 20070115 (prerelease) (SUSE Linux)
TLS SNI support enabled
configure arguments: --user=www --group=www --prefix=/usr/local/nginx --with-http_ssl_module --with-openssl=/usr/local/src/openssl-1.0.1c --with-http_stub_status_module --with-pcre --with-pcre=/usr/local/src/pcre-8.21 --with-zlib=/usr/local/src/zlib-1.2.8

正常了,平滑升級成功!

二、純手動平滑升級

純手動模式,指的是在最後做遷移的時候,全部使用手動命令來搞定,避免編譯可能存在不一致的引數啥的。

實際上,在 make 之後,我們可以檢視 nginx 原始碼目錄下的 Makefile 內容如下:

default:        build

clean:
        rm -rf Makefile objs

build:
        $(MAKE) -f objs/Makefile
        $(MAKE) -f objs/Makefile manpage

install:
        $(MAKE) -f objs/Makefile install

upgrade:
        /usr/local/nginx/sbin/nginx -t

        kill -USR2 `cat /usr/local/nginx/logs/nginx.pid`
        sleep 1
        test -f /usr/local/nginx/logs/nginx.pid.oldbin

        kill -QUIT `cat /usr/local/nginx/logs/nginx.pid.oldbin`

所以,說白了純手動就是執行 upgrade 標籤下的命令列而已,實際上只要確認 Makefile 下的命令路徑都是正確的,用命令自動遷移是沒有任何問題的。

總是有人會不放心的,喜歡手動一步一步的搞定,我也來整理下純手動步驟:

①~③和半自動一樣,按常規步驟先編譯 nginx,不過只執行到 make 就打住,然後將舊的 sbin 下的 nginx 檔案移走,再將編譯得到的 objs 目錄下的 nginx 檔案放到原來的 sbin 目錄。

④、測試新版本的 nginx 是否正常:

[[email protected]_Server nginx-1.6.0]# /usr/local/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok #OK,沒有問題!
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

⑤、給舊 nginx 傳送平滑遷移訊號(若不清楚 pid 路徑就用可用命令(2)):

#可用命令(1):
kill -USR2 `cat /usr/local/nginx/logs/nginx.pid`

#可用命令(2):
kill -USR2 `ps aux | grep "nginx: master process" | grep -v grep | awk '{print $2}'`

Ps:後面其實就是舊 nginx 的 pid,所以先用 ps aux 找到正在執行的 nginx 主程序 pid,再執行 kill -USR2 PID 值亦可。

⑥、等待舊版本 Nginx 的 pid 變為 oldbin(執行如下命令檢視是否生成)

test -f /usr/local/nginx/logs/nginx.pid.oldbin && echo OK!

⑦、  從容關閉舊版本的 Nginx 程序

kill –WINCH `cat /usr/local/nginx/log/nginx.oldbin`

此時,舊的工作程序就都會慢慢隨著任務執行完畢而退出,新版的 Nginx 的工作程序會逐漸取代舊版工作程序。

⑧、此時,不過載配置啟動舊工作程序(個人感覺是為了將任務完全切換到新的 nginx 上)

kill –HUP `cat /url/local/nginx/log/nginx.oldbin`

⑨、結束工作程序,完成此次升級操作:

kill –QUIT `cat /usr/local/nginx/log/nginx.oldbin`

⑩、最後,驗證 nginx 是否升級成功:

linux-gz215:~ # /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.6.0 #沒問題
built by gcc 4.1.2 20070115 (prerelease) (SUSE Linux)
TLS SNI support enabled
configure arguments: --user=www --group=www --prefix=/usr/local/nginx --with-http_ssl_module --with-openssl=/usr/local/src/openssl-1.0.1c --with-http_stub_status_module --with-pcre --with-pcre=/usr/local/src/pcre-8.21 --with-zlib=/usr/local/src/zlib-1.2.8

特意測試了下純手動的做法,下面是我的操作記錄,僅供參考:

linux-gz215:/usr/local/nginx # cd sbin/
linux-gz215:/usr/local/nginx/sbin # ll
總計 6828
-rwxr-xr-x 1 root root 6975582 2014-12-23 16:44 nginx
linux-gz215:/usr/local/nginx/sbin # mv nginx nginx.old
linux-gz215:/usr/local/nginx/sbin # 
linux-gz215:/usr/local/nginx/sbin # cp /usr/local/src/nginx-1.5.7/objs/
autoconf.err        nginx               ngx_auto_config.h   ngx_modules.c       src/              
Makefile            nginx.8             ngx_auto_headers.h  ngx_modules.o       
linux-gz215:/usr/local/nginx/sbin # cp /usr/local/src/nginx-1.5.7/objs/nginx .
linux-gz215:/usr/local/nginx/sbin # ll
總計 13656
-rwxr-xr-x 1 root root 6975582 2014-12-23 16:57 nginx
-rwxr-xr-x 1 root root 6975582 2014-12-23 16:44 nginx.old
linux-gz215:/usr/local/nginx/sbin # /usr/local/nginx/sbin/nginx -t -c /usr/local/nginx/conf/nginx.conf
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
linux-gz215:/usr/local/nginx/sbin # ps aux | grep nginx 
root     18152  0.0  0.0   9264  2588 ?        S    16:45   0:00 nginx: master process ./nginx
nobody   18331  1.0  0.0  13360  5488 ?        S    16:52   0:04 nginx: worker process
nobody   18332  1.2  0.0  13360  5488 ?        S    16:52   0:05 nginx: worker process
nobody   18333  0.6  0.0  13360  5488 ?        S    16:52   0:02 nginx: worker process
nobody   18334  0.8  0.0  13360  5488 ?        S    16:52   0:03 nginx: worker process
nobody   18335  0.4  0.0  13360  5488 ?        S    16:52   0:01 nginx: worker process
nobody   18336  0.1  0.0  13360  5488 ?        S    16:52   0:00 nginx: worker process
nobody   18337  0.3  0.0  13828  5844 ?        S    16:52   0:01 nginx: worker process
nobody   18338  0.2  0.0  13360  5488 ?        S    16:52   0:01 nginx: worker process
root     18473  0.0  0.0   4952   796 pts/1    S+   16:58   0:00 grep nginx
linux-gz215:/usr/local/nginx/sbin # kill -USR2 18152
linux-gz215:/usr/local/nginx/sbin # ps aux | grep nginx
root     18152  0.0  0.0   9264  2588 ?        S    16:45   0:00 nginx: master process ./nginx
nobody   18331  0.9  0.0  13360  5488 ?        S    16:52   0:04 nginx: worker process
nobody   18332  1.2  0.0  13360  5488 ?        S    16:52   0:05 nginx: worker process
nobody   18333  0.5  0.0  13360  5488 ?        S    16:52   0:02 nginx: worker process
nobody   18334  0.8  0.0  13360  5488 ?        S    16:52   0:03 nginx: worker process
nobody   18335  0.4  0.0  13360  5488 ?        S    16:52   0:01 nginx: worker process
nobody   18336  0.2  0.0  13792  5840 ?        S    16:52   0:01 nginx: worker process
nobody   18337  0.5  0.0  13464  5504 ?        S    16:52   0:02 nginx: worker process
nobody   18338  0.2  0.0  13360  5488 ?        S    16:52   0:01 nginx: worker process
root     18474  0.0  0.0   9124  2460 ?        S    16:59   0:00 nginx: master process ./nginx
nobody   18475  5.0  0.0  13364  5424 ?        S    16:59   0:00 nginx: worker process
nobody   18476  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18477  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18478  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18479  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18480  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18481  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18482  0.0  0.0  13136  5032 ?        S    16:59   0:00 nginx: worker process
root     18484  0.0  0.0   4960   812 pts/1    S+   16:59   0:00 grep nginx
linux-gz215:/usr/local/nginx/sbin # kill -WINCH 18152
linux-gz215:/usr/local/nginx/sbin # ps aux | grep nginx
root     18152  0.0  0.0   9264  2588 ?        S    16:45   0:00 nginx: master process ./nginx
nobody   18334  0.7  0.0  13360  5488 ?        S    16:52   0:03 nginx: worker process is shutting down
nobody   18337  0.5  0.0  13360  5488 ?        S    16:52   0:02 nginx: worker process is shutting down
root     18474  0.0  0.0   9124  2460 ?        S    16:59   0:00 nginx: master process ./nginx
nobody   18475  2.3  0.0  13672  5724 ?        S    16:59   0:01 nginx: worker process
nobody   18476  0.0  0.0  13136  5240 ?        S    16:59   0:00 nginx: worker process
nobody   18477  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18478  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18479  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18480  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18481  0.0  0.0  13136  5240 ?        S    16:59   0:00 nginx: worker process
nobody   18482  0.0  0.0  13136  5240 ?        S    16:59   0:00 nginx: worker process
root     18486  0.0  0.0   4956   796 pts/1    S+   16:59   0:00 grep nginx
linux-gz215:/usr/local/nginx/sbin # ps aux | grep nginx
root     18152  0.0  0.0   9264  2588 ?        S    16:45   0:00 nginx: master process ./nginx
root     18474  0.0  0.0   9124  2460 ?        S    16:59   0:00 nginx: master process ./nginx
nobody   18475  2.8  0.0  13792  5908 ?        S    16:59   0:01 nginx: worker process
nobody   18476  0.0  0.0  13136  5240 ?        S    16:59   0:00 nginx: worker process
nobody   18477  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18478  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18479  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18480  0.0  0.0  13136  5040 ?        S    16:59   0:00 nginx: worker process
nobody   18481  0.0  0.0  13136  5240 ?        S    16:59   0:00 nginx: worker process
nobody   18482  0.0  0.0  13136  5240 ?        S    16:59   0:00 nginx: worker process
root     18488  0.0  0.0   4956   796 pts/1    S+   16:59   0:00 grep nginx
linux-gz215:/usr/local/nginx/sbin # kill -HUP 18152
linux-gz215:/usr/local/nginx/sbin # ps aux | grep nginx
root     18152  0.0  0.0   9264  2588 ?        S    16:45   0:00 nginx: master process ./nginx
root     18474  0.0  0.0   9124  2460 ?        S    16:59   0:00 nginx: master process ./nginx
nobody   18475  3.1  0.0  13256  5376 ?        S    16:59   0:02 nginx: worker process
nobody   18476  0.0  0.0  13256  5336 ?        S    16:59   0:00 nginx: worker process
nobody   18477  0.8  0.0  13420  5532 ?        S    16:59   0:00 nginx: worker process
nobody   18478  0.2  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18479  0.2  0.0  13580  5656 ?        S    16:59   0:00 nginx: worker process
nobody   18480  0.0  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18481  0.3  0.0  13412  5532 ?        S    16:59   0:00 nginx: worker process
nobody   18482  0.0  0.0  13256  5320 ?        S    16:59   0:00 nginx: worker process
nobody   18570  2.0  0.0  13276  5380 ?        S    17:00   0:00 nginx: worker process
nobody   18571  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18572  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18573  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18574  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18575  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18576  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18577  0.0  0.0  13276  5164 ?        S    17:00   0:00 nginx: worker process
root     18579  0.0  0.0   4960   812 pts/1    S+   17:00   0:00 grep nginx
linux-gz215:/usr/local/nginx/sbin # kill -QUIT 18152
linux-gz215:/usr/local/nginx/sbin # ps aux | grep nginx
root     18474  0.0  0.0   9124  2460 ?        S    16:59   0:00 nginx: master process ./nginx
nobody   18475  2.2  0.0  13256  5376 ?        S    16:59   0:02 nginx: worker process
nobody   18476  0.0  0.0  13280  5400 ?        S    16:59   0:00 nginx: worker process
nobody   18477  1.5  0.0  13256  5376 ?        S    16:59   0:01 nginx: worker process
nobody   18478  0.1  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18479  0.2  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18480  0.0  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18481  0.2  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18482  0.0  0.0  13256  5320 ?        S    16:59   0:00 nginx: worker process
nobody   18570  3.2  0.0  13644  5672 ?        S    17:00   0:00 nginx: worker process
nobody   18571  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18572  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18573  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18574  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18575  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18576  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18577  0.6  0.0  13360  5412 ?        S    17:00   0:00 nginx: worker process
root     18582  0.0  0.0   4956   804 pts/1    R+   17:00   0:00 grep nginx
linux-gz215:/usr/local/nginx/sbin # ps aux | grep nginx
root     18474  0.0  0.0   9124  2460 ?        S    16:59   0:00 nginx: master process ./nginx
nobody   18475  2.2  0.0  13256  5376 ?        S    16:59   0:02 nginx: worker process
nobody   18476  0.2  0.0  13452  5520 ?        S    16:59   0:00 nginx: worker process
nobody   18477  1.5  0.0  13256  5376 ?        S    16:59   0:01 nginx: worker process
nobody   18478  0.1  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18479  0.2  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18480  0.0  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18481  0.2  0.0  13256  5376 ?        S    16:59   0:00 nginx: worker process
nobody   18482  0.0  0.0  13256  5320 ?        S    16:59   0:00 nginx: worker process
nobody   18570  3.2  0.0  13644  5672 ?        S    17:00   0:01 nginx: worker process
nobody   18571  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18572  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18573  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18574  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18575  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18576  0.0  0.0  13276  5172 ?        S    17:00   0:00 nginx: worker process
nobody   18577  0.5  0.0  13360  5412 ?        S    17:00   0:00 nginx: worker process
root     18584  0.0  0.0   4956   812 pts/1    S+   17:00   0:00 grep nginx
linux-gz215:/usr/local/nginx/sbin #

為了驗證平滑升級確實不影響線上業務,我特意在升級的時候,利用 ab 命令一直在傳送請求:

ab -n1000000 -c10 http://domain.com/

直到升級完成,使用 ctrl +C 終止並檢視 ab 結果,可以發現幾十萬次的請求全部成功,沒有失敗!證明平滑升級的可行性!