1. 程式人生 > >Python-Django框架的select_related 和 prefetch_related函式對 QuerySet 查詢的優化

Python-Django框架的select_related 和 prefetch_related函式對 QuerySet 查詢的優化

概念:
  • select_related()當執行它的查詢時它沿著外來鍵關係查詢關聯的物件資料。它會生成一個複雜的查詢並引起效能的消耗,但是在以後使用外來鍵關係時將不需要資料庫查詢。
  • prefetch_related()返回的也是QuerySet,它將在單個批處理中自動檢索每個指定查詢的物件。這具有與select_related類似的目的,兩者都被設計為阻止由訪問相關物件而導致的資料庫查詢的泛濫,但是策略是完全不同的。
  • select_related通過建立SQL連線並在SELECT語句中包括相關物件的欄位來工作。因此,select_related在同一資料庫查詢中獲取相關物件。然而,為了避免由於跨越“多個'
    關係而導致的大得多的結果集,select_related限於單值關係 -外來鍵一對一關係
  • prefetch_related,另一方面,為每個關係單獨查詢,並在Python中“加入”。這允許它預取多對多多對一物件,除了外來鍵一對一關係,它們不能使用select_related來完成
按例如下:
  • 首先我們在setting.py檔案裡設定SQL查詢語句:
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console'
:{ 'level':'DEBUG', 'class':'logging.StreamHandler', }, }, 'loggers': { 'django.db.backends': { 'handlers': ['console'], 'propagate': True, 'level':'DEBUG', }, } }
建立一個model.py
from django.db import models


class
Province(models.Model):
"""省""" name = models.CharField(max_length=10) def __unicode__(self): return self.name class City(models.Model): """市""" name = models.CharField(max_length=5) province = models.ForeignKey(Province) def __unicode__(self): return self.name class Person(models.Model): """人""" firstname = models.CharField(max_length=10) lastname = models.CharField(max_length=10) visitation = models.ManyToManyField(City, related_name="visitor") hometown = models.ForeignKey(City, related_name="birth") living = models.ForeignKey(City, related_name="citizen") def __unicode__(self): return self.firstname + self.lastname
  • 三張表裡都插入了少量的資料(需要自己新增資料)
一:select_related()
  • 對於一對一欄位(OneToOneField)外來鍵欄位(ForeignKey),可以使用select_related 來對QuerySet進行優化。
  • 接下來我們看案例:
>>> City.objects.all()
#  列印結果是:有四條資料 <QuerySet [<City: Hcity>, <City: Ccity>, <City: Ecity>, <City: Qcity>]>

# 下面我們來實現外來鍵欄位的操作:
>>> citys = City.objects.all()
>>> for city in citys:
...     print city.province
"""
BeiJing
BeiJing
BeiJing
BeiJing
"""
  • 下面我們開啟SQL查詢的語句:(可以看到查詢了5次資料庫)
(0.000) SELECT `apps_city`.`id`, `apps_city`.`name`, `apps_city`.`province_id` FROM `apps_city`; args=()
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
BeiJing
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
BeiJing
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
BeiJing
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
BeiJing
  • 下面我們用select_related()來檢視一下查詢的次數
>>> citys = City.objects.select_related().all()
>>> for city in citys:
...     print city.province
# 看列印結果:同樣返回的4條資料
"""
BeiJing
BeiJing
BeiJing
BeiJing
"""
  • 接下來我們看SQL語句的查詢:(只有一條,仔細觀察INNER JOIN想起了連表查詢吧)
    顯然大大的減少了SQL查詢的次數
(0.000) SELECT `apps_city`.`id`, `apps_city`.`name`, `apps_city`.`province_id`, `apps_province`.`id`, `apps_province`.`name` FROM `apps_city` INNER JOIN `apps_province` ON (`apps_ci
ty`.`province_id` = `apps_province`.`id`); args=()
接下來我們給select_related() 新增:*fields 引數
  • select_related() 接受可變長引數,每個引數是需要獲取的外來鍵(父表的內容)的欄位名,以及外來鍵的外來鍵的欄位名外來鍵的外來鍵的外來鍵…。若要選擇外來鍵的外來鍵需要使用兩個下劃線“__”來連線。
  • 我們舉個例子:
>>> persons = Person.objects.select_related("living__province").get(pk=1)
>>> persons.living.province
# 輸出結果: <Province: BeiJing>
  • 接下來看出發的SQL語句:
(0.000) SELECT `apps_person`.`id`, `apps_person`.`firstname`, `apps_person`.`lastname`, `apps_person`.`hometown_id`, `apps_person`.`living_id`, `apps_city`.`id`, `apps_city`.`name`,
 `apps_city`.`province_id`, `apps_province`.`id`, `apps_province`.`name` FROM `apps_person` INNER JOIN `apps_city` ON (`apps_person`.`living_id` = `apps_city`.`id`) INNER JOIN `apps
_province` ON (`apps_city`.`province_id` = `apps_province`.`id`) WHERE `apps_person`.`id` = 1; args=(1,)
  • 以上可以看出來,Django使用了2次INNER JOIN來完成請求,取到了city表和province表的內容,並新增到結果表的相應列,在呼叫查詢persons.living的時候也不必再次進行SQL查詢。
  • 如果未指定外來鍵則不會被新增到結果中。
>>> persons.hometown.province
(0.001) SELECT `apps_city`.`id`, `apps_city`.`name`, `apps_city`.`province_id` FROM `apps_city` WHERE `apps_city`.`id` = 1; args=(1,)
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
<Province: BeiJing>
  • 同時,如果不指定外來鍵,就會進行兩次查詢。如果深度更深,查詢的次數就越多。
  • Diango1.7開始,select_related()函式的作用方式改變了。在1.7版本以前select_related()只能這麼做:
>>> persons = Person.objects.select_related("hometown__province","living__province").get(pk=1)
  • 看SQL執行的查詢:

(0.003) SELECT `apps_person`.`id`, `apps_person`.`firstname`, `apps_person`.`lastname`, `apps_person`.`hometown_id`, `apps_person`.`living_id`, `apps_city`.`id`, `apps_city`.`name`,
 `apps_city`.`province_id`, `apps_province`.`id`, `apps_province`.`name`, T4.`id`, T4.`name`, T4.`province_id`, T5.`id`, T5.`name` FROM `apps_person` INNER JOIN `apps_city` ON (`app
s_person`.`hometown_id` = `apps_city`.`id`) INNER JOIN `apps_province` ON (`apps_city`.`province_id` = `apps_province`.`id`) INNER JOIN `apps_city` T4 ON (`apps_person`.`living_id`
= T4.`id`) INNER JOIN `apps_province` T5 ON (T4.`province_id` = T5.`id`) WHERE `apps_person`.`id` = 1; args=(1,)
# 從下面我們可以看出來,通過外來鍵獲取到的資料,不會進行資料庫的查詢:
>>> persons.living.province
<Province: BeiJing>
>>> persons.hometown.province
<Province: BeiJing>
  • 但是在1.7以上版本,可以像QuerySet的其他函式一樣進行操作:
>>> persons = Person.objects.select_related("living__province").select_related("hometown__province").get(pk=1)
  • 看SQL語句的查詢:
(0.001) SELECT `apps_person`.`id`, `apps_person`.`firstname`, `apps_person`.`lastname`, `apps_person`.`hometown_id`, `apps_person`.`living_id`, `apps_city`.`id`, `apps_city`.`name`,
 `apps_city`.`province_id`, `apps_province`.`id`, `apps_province`.`name`, T4.`id`, T4.`name`, T4.`province_id`, T5.`id`, T5.`name` FROM `apps_person` INNER JOIN `apps_city` ON (`app
s_person`.`hometown_id` = `apps_city`.`id`) INNER JOIN `apps_province` ON (`apps_city`.`province_id` = `apps_province`.`id`) INNER JOIN `apps_city` T4 ON (`apps_person`.`living_id`
= T4.`id`) INNER JOIN `apps_province` T5 ON (T4.`province_id` = T5.`id`) WHERE `apps_person`.`id` = 1; args=(1,)
>>> persons.living.province
<Province: BeiJing>
>>> persons.hometown.province
<Province: BeiJing>
>>>
  • 建議大家使用1.7以上版本
二:prefetch_related()
  • 下面我來們操作一下:
>>> persons = Person.objects.prefetch_related("visitation__province").get(pk=1)
  • 查詢SQL語句:
(0.007) SELECT (`apps_person_visitation`.`person_id`) AS `_prefetch_related_val_person_id`, `apps_city`.`id`, `apps_city`.`name`, `apps_city`.`province_id` FROM `apps_city` INNER JO
IN `apps_person_visitation` ON (`apps_city`.`id` = `apps_person_visitation`.`city_id`) WHERE `apps_person_visitation`.`person_id` IN (1); args=(1,)
>>> persons.visitation
<django.db.models.fields.related_descriptors.ManyRelatedManager object at 0x0000000003FCAAC8>
  • prefetch_related()的操作和 select_related()大概是相同的, 只是prefetch_related()是操作,多對多的表格。