tornado6與python3.7 非同步新姿勢
廢話不多說,直接上程式碼
__auth__ = "aleimu" __doc__ = "學習tornado6.0+ 版本與python3.7+" import time import asyncio import tornado.gen import tornado.web import tornado.ioloop import tornado.httpserver# tornado的HTTP伺服器實現 from tornado.options import define, options from tornado.httpclient import HTTPClient, AsyncHTTPClient from requests import get settings = {'debug': True} url = "http://127.0.0.1:5000/"# 這是另個服務,請求5s後返回結果 # RuntimeError: Cannot run the event loop while another loop is running # 解釋:HTTPClient內部寫 loop.run_xxx,因為那是啟動event loop的命令,通常只再最最最外面用一次,之後的程式碼都應假設 loop 已經在運轉了。 def synchronous_fetch(url): print("synchronous_fetch") try: http_client = HTTPClient() time.sleep(5) response = http_client.fetch(url) print(response.body) except Exception as e: print("Error: " + str(e)) return str(e) http_client.close() return response.body # 替代synchronous_fetch的同步請求,沒有內建loop.run_xxx def synchronous_get(url): response = get(url) time.sleep(5) print("synchronous_fetch") return response.text # 簡單的模擬非同步操作,這裡之後應該替換成各種非同步庫的函式 async def sleep(): print("start sleep") await asyncio.sleep(5) print("end sleep") # 非同步請求 async def asynchronous_fetch(url): http_client = AsyncHTTPClient() response = await http_client.fetch(url) print("asynchronous_fetch") return response.body # 測試 class MainHandler(tornado.web.RequestHandler): def get(self): self.write("Hello, world:%s" % self.request.request_time()) self.finish() print("not finish!") return # 同步阻塞 class synchronous_fetcher(tornado.web.RequestHandler): def get(self): self.write("%s,%s" % (synchronous_fetch(url), self.request.request_time())) # 同步阻塞 class synchronous_geter(tornado.web.RequestHandler): def get(self): self.write("%s,%s" % (synchronous_get(url), self.request.request_time())) # 非同步阻塞,我以為curl "127.0.0.1:8888/1" 總耗時希望為5s,可是是25s,看來非同步沒搞好,以下的函式都是基於此改進的 class asynchronous_fetcher_1(tornado.web.RequestHandler): async def get(self): body = await asynchronous_fetch(url) for i in range(3): print("skip %s" % i) await tornado.gen.sleep(5) time.sleep(5) print("end request") self.write("%s,%s" % (body, self.request.request_time())) # curl "127.0.0.1:8888/1" # b'{\n"data": "123"\n}\n',25.026000022888184 # 非同步阻塞,效果同上,這裡只是證明 tornado.gen.sleep(5)和asyncio.sleep(5) 效果一致 class asynchronous_fetcher_2(tornado.web.RequestHandler): async def get(self): body = await asynchronous_fetch(url)# 關注協程完成後返回的結果 for i in range(3): print("skip %s" % i) await sleep() time.sleep(5) print("end request") self.write("%s,%s" % (body, self.request.request_time())) # curl "127.0.0.1:8888/2" # b'{\n"data": "123"\n}\n',25.039999961853027 # 非同步非阻塞-將部分非同步操作放入組中,實現loop管理 class asynchronous_fetcher_3(tornado.web.RequestHandler): async def get(self): body = await asynchronous_fetch(url) await asyncio.wait([sleep() for i in range(3)]) print("end request") self.write("%s,%s" % (body, self.request.request_time())) # curl "127.0.0.1:8888/3" # b'{\n"data": "123"\n}\n',10.001000165939331 # 非同步非阻塞-將所有非同步操作放入組中,實現loop管理 class asynchronous_fetcher_4(tornado.web.RequestHandler): async def get(self): task_list = [sleep() for i in range(3)] task_list.append(asynchronous_fetch(url)) body = await asyncio.wait(task_list)# 將所有非同步操作的結果返回,但是是無序的,要是需要返回結果的話解析起來比較麻煩 print("end request:", body) # print(type(body), len(body),type(body[0]),len(body[0]),type(body[0])) self.write("%s,%s" % ([x.result() for x in body[0] if x.result() is not None][0], self.request.request_time())) # curl "127.0.0.1:8888/4" # b'{\n"data": "123"\n}\n',5.006999969482422 def make_app(): return tornado.web.Application([ (r"/", MainHandler), (r"/1", asynchronous_fetcher_1), (r"/2", asynchronous_fetcher_2), (r"/3", asynchronous_fetcher_3), (r"/4", asynchronous_fetcher_4), (r"/5", synchronous_fetcher), (r"/6", synchronous_geter), ], **settings) if __name__ == "__main__": print("server start!") app = make_app() server = tornado.httpserver.HTTPServer(app) server.bind(8888) server.start(1)# forks one process per cpu,windows上無法fork,這裡預設為1 tornado.ioloop.IOLoop.current().start()
總結
1.Tornado使用單執行緒事件迴圈,寫的不好,會阻塞的非常嚴重,比如synchronous_geter 2.flask+celery可以完成常見的非同步任務 3.await語法只能出現在通過async修飾的函式中 4.可以看到tornado.gen.coroutine, tornado.concurrent.run_on_executor,tornado.web.asynchronous,tornado.gen.coroutine等這些裝飾器都不在經常使用了,都由async和await代替
參考文件:
https://zhuanlan.zhihu.com/p/27258289# Python async/await入門指南 http://www.tornadoweb.org/en/stable/guide/intro.html# 這個官網 https://www.osgeo.cn/tornado/guide/intro.html#Tornado 1.0 - Tornado 6.0的更新說明,以及6.0版本的中文文件,適合英語不好的人閱讀 https://www.osgeo.cn/tornado/releases/v5.0.0.html#在Python 3上, IOLoop 總是包裝asyncio事件迴圈。
On Python 3, IOLoop is always a wrapper around the asyncio event loop.
這是我重新複習tornado的原因,tornado放棄了之前自己實現的tornado.ioloop,全面擁抱asyncio的event_loop.這個改動是非常大的,
而且閱讀tornado的原始碼可以發現其中大部分函式都支援了型別檢驗,和返回值提示,值得閱讀.