1. 程式人生 > >使用lua-nginx模組實現請求解析與排程

使用lua-nginx模組實現請求解析與排程

**系統版本及需求**: ``OS``:CentOS 7.7.1908 ``OpenResty``:1.15.8.2 [TOC] # 描述 lua-nginx-module模組是什麼: > It is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty. > > By leveraging Nginx's subrequests, this module allows the integration of the powerful Lua threads (known as Lua "coroutines") into the Nginx event model. > > Unlike Apache's mod_lua and Lighttpd's mod_magnet, Lua code executed using this module can be 100% non-blocking on network traffic as long as the Nginx API for Lua provided by this module is used to handle requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services. OpenResty的核心元件,將lua執行緒整合到nginx模型中,且不會阻塞網路流量。 [lua-nginx-module專案詳細內容](https://github.com/openresty/lua-nginx-module) 可以通過編譯將其安裝為Nginx Module。本文直接安裝OpenResty,通過lua指令碼主要實現以下兩個目標: - 將一份請求轉發給多個後端,但僅用其中一個後端回覆請求。 - 分析一份請求的引數內容,根據規則將請求分發到不同的後端。 通過以上兩個目標,也很容易衍生出其他的可能性,例如通過此模組實現根據請求使用者的特徵將其排程到不同的伺服器:以此來達到目標(比如灰度、就近訪問、黑白名單等);根據轉發多後端特性,實現完全的真實環境壓力測試等。 # 安裝配置 ## 安裝openresty 通過原始碼編譯安裝,具體步驟如下: ```bash yum install -y pcre-devel openssl-devel gcc curl mkdir -p /data/pkg/ && cd /data/pkg/ wget https://openresty.org/download/openresty-1.15.8.2.tar.gz tar xf openresty-1.15.8.2.tar.gz cd openresty-1.15.8.2 ./configure --with-file-aio --with-http_ssl_module --with-http_realip_module --with-http_sub_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_stub_status_module make -j$nproc make install ``` 編譯時Nginx的大多數選項都支援。如上啟動了一些Module,具體根據你的需求選擇。 預設安裝路徑``/usr/local/openresty``,想更改路徑通過--prefix=/path指定。 ## 使用示例 建立一個存放指令碼的目錄: ```bash cd /usr/local/openresty mkdir nginx/conf/lua ``` 建立一個lua測試指令碼: vim nginx/conf/lua/hello.lua ```lua local action = ngx.var.request_method if(action == "POST") then ngx.say("Method: POST; Hello world") elseif(action == "GET") then ngx.say("Method: GET; Welcome to the web site") end ``` 在Server段配置中啟用lua指令碼: vim nginx/conf/nginx.conf 在nginx.conf中新增加一個server段,請求路徑以/開頭的則使用lua指令碼進行處理。 ```conf server { listen 0.0.0.0:8080; location / { root html; index index.html index.htm; } location ~* ^/(.*)$ { content_by_lua_file "conf/lua/hello.lua"; # lua script location } } ``` 測試效果: 使用./bin/openresty -t命令檢查配置無誤,然後使用./bin/openresty命令啟動服務。 POST和GET請求方式返回不同的相應內容 ```bash curl 127.0.0.1:8080 # 返回資訊 Method: GET; Welcome to the web site curl -d "" 127.0.0.1:8080 # 返回資訊 Method: POST; Hello world ``` # HTTP請求複製 環境準備妥當,現在通過lua指令碼程式配合openresty實現對於HTTP請求的複製。 當一個請求來到openresty服務時,把此請求轉發給後端的server1、server2等等,但只是用server1或server2的應答訊息回覆這個請求。 一個簡單的示例圖: ![HTTP請求複製](https://img2020.cnblogs.com/blog/2286432/202102/2286432-20210228213500632-2099875475.png) **需求**:將請求同時傳送到/prod和/test路徑後的真實後端,但只使用/prod的後端回覆client的請求 lua程式碼 vim nginx/conf/lua/copyRequest.lua ```lua function req_copy() local resp_prod, resp_test = ngx.location.capture_multi { {"/prod" .. ngx.var.request_uri, arry}, {"/test" .. ngx.var.request_uri, arry}, } if resp_prod.status == ngx.HTTP_OK then local header_list = {"Content-Type", "Content-Encoding", "Accept-Ranges"} for _, i in ipairs(header_list) do if resp_prod.header[i] then ngx.header[i] = resp_prod.header[i] end end ngx.say(resp_prod.body) else ngx.say("Upstream server error : " .. resp_prod.status) end end req_copy() ``` nginx新建的server段配置如下,且引入vhosts目錄下配置檔案,nginx/conf/nginx.conf: ```conf server { listen 0.0.0.0:8080; location / { root html; index index.html index.htm; } # 匹配lua檔案中/prod+原請求的uri(/copy/*) location ^~ /prod/ { # 路徑重寫後,/prod後端伺服器收到的路徑為客戶端請求路徑去掉開頭/copy/ rewrite /prod/copy/(.*)$ /$1 break; proxy_pass http://127.0.0.1:8081; } # 匹配lua檔案中/test+原請求的uri(/copy/*) location ^~ /test/ { # 路徑重寫後,/prod後端伺服器收到的路徑為客戶端請求路徑去掉開頭/copy/ rewrite /test/copy/(.*)$ /$1 break; proxy_pass http://127.0.0.1:8082; } location ^~ /copy/ { content_by_lua_file "conf/lua/copyRequest.lua"; } } include vhosts/*.conf; ``` 建立兩個後端主機,模擬代表不同環境: ```conf # nginx/conf/vhosts/prod.conf server { listen 8081; server_name localhost; access_log logs/prod_server.log; location / { return 200 "Welcome to prod server"; } location /api/v1 { return 200 "API V1"; } } # nginx/conf/vhosts/test.conf server { listen 8082; server_name localhost; access_log logs/test_server.log; location / { return 200 "Welcome to test server"; } } ``` 配置更新後重載服務,然後測試訪問: ```sh > curl 127.0.0.1:8080/copy/ Welcome to prod server > curl 127.0.0.1:8080/copy/api/v1 API V1 # /prod和/test後端的服務都收到了請求 # 檢視日誌;tail -2 nginx/logs/prod_server.log 127.0.0.1 - - [01/Mar/2020:01:41:12 +0800] "GET / HTTP/1.0" 200 22 "-" "curl/7.29.0" 127.0.0.1 - - [01/Mar/2020:01:41:15 +0800] "GET /api/v1 HTTP/1.0" 200 6 "-" "curl/7.29.0" # 檢視日誌;tail -2 nginx/logs/test_server.log 127.0.0.1 - - [01/Mar/2020:01:41:12 +0800] "GET / HTTP/1.0" 200 22 "-" "curl/7.29.0" 127.0.0.1 - - [01/Mar/2020:01:41:15 +0800] "GET /api/v1 HTTP/1.0" 200 22 "-" "curl/7.29.0" ``` 模擬場景圖示: ![lua-nginx.copyRequest](https://img2020.cnblogs.com/blog/2286432/202102/2286432-20210228213540641-1350456455.png) 1. client訪問nginx服務監聽的``IP:8080/copy/``路徑 2. lua指令碼處理收到的請求,代訪問``/prod和/test``路徑 3. ``/prod和/test``路徑在本地被代理到真實後端 4. 真實後端接收到請求並返回 5. lua指令碼僅處理``/prod(resp_prod)``後端伺服器返回的內容(見lua指令碼中程式碼) 6. 將/prod後端伺服器返回的內容返回給client # HTTP報文解析 **需求**:將請求報文引數中的userid在某個區間的請求排程到指定的後端服務上。 建立nginx/conf/lua/requestBody.lua程式碼: ```lua -- post body提交方式為application/x-www-form-urlencoded的內容獲取方法 function urlencodedMethod() local postBody = {} for key, val in pairs(args) do postBody[key] = val end local uid = postBody["userid"] postBody = nil return tonumber(uid) end -- get請求方式為xx.com/?userid=x其params的獲取方式 function uriParameterMethod() local getParameter = {}, key, val for key, val in pairs(args) do if type(val) == "table" then getParameter[key] = table.concat(val) else getParameter[key] = val end end local uid = getParameter["userid"] getParameter = nil return tonumber(uid) end -- 獲取post body提交的方式;multipart/from-data在此示例中沒有實現對其內容的處理 function contentType() local conType = ngx.req.get_headers()["Content-Type"] local conTypeTable = {"application/x-www-form-urlencoded", "multipart/form-data"} local receiveConType, y if(type(conType) == "string") then for y = 1, 2 do local word = conTypeTable[y] local from, to, err = ngx.re.find(conType, word, "jo") if from and to then receiveConType = string.sub(conType, from, to) end end else receiveConType = nil end return receiveConType end -- 迴圈出一些需要的header返回給客戶端 function iterHeaders(resp_content) local header_list = {"Content-Type", "Content-Encoding", "Accept-Ranges","Access-Control-Allow-Origin", "Access-Control-Allow-Methods","Access-Control-Allow-Headers", "Access-Control-Allow-Credentials"} for _, i in ipairs(header_list) do if(resp_content.header[i]) then ngx.header[i] = resp_content.header[i] end end return resp_content end -- 將userid大於等於1小於等於10的請求傳送給/prod路徑的後端 -- 將userid大於等於11小於等於20的請求傳送給/test路徑的後端 -- 將userid非以上兩種的同時傳送給/prod和/test路徑的後端,使用/prod後端回覆請求 function requestTo(uid) local resp, resp_noReply if(uid >= 1 and uid <= 10) then resp = ngx.location.capture_multi { {"/prod".. ngx.var.request_uri, arry}, } elseif(uid >= 11 and uid <= 20) then resp = ngx.location.capture_multi { {"/test".. ngx.var.request_uri, arry}, } else resp, resp_noReply = ngx.location.capture_multi { {"/prod" .. ngx.var.request_uri, arry}, {"/test" .. ngx.var.request_uri, arry}, } end local res if(resp.status == ngx.HTTP_OK) then res = iterHeaders(resp) else res = "Upstream server err : " .. reps_content.status end ngx.say(res.body) end -- 處理主函式 function main_func() ngx.req.read_body() local action = ngx.var.request_method if(action == "POST") then arry = {method = ngx.HTTP_POST, body = ngx.req.read_body()} args = ngx.req.get_post_args() elseif(action == "GET") then args = ngx.req.get_uri_args() arry = {method = ngx.HTTP_GET} end local u if(action == "POST") then if args then local getContentType = contentType() if(getContentType == "application/x-www-form-urlencoded") then u = urlencodedMethod() end end elseif(action == "GET") then if args then u = uriParameterMethod() end end if(u == nil) then ngx.say("Request parameter cannot be empty