Python-Django框架的select_related 和 prefetch_related函式對 QuerySet 查詢的優化
阿新 • • 發佈:2018-12-03
概念:
select_related()
當執行它的查詢時它沿著外來鍵關係查詢關聯的物件資料。它會生成一個複雜的查詢並引起效能的消耗,但是在以後使用外來鍵關係時將不需要資料庫查詢。prefetch_related()
返回的也是QuerySet,它將在單個批處理中自動檢索每個指定查詢的物件。這具有與select_related類似的目的,兩者都被設計為阻止由訪問相關物件而導致的資料庫查詢的泛濫,但是策略是完全不同的。select_related
通過建立SQL連線並在SELECT語句
中包括相關物件的欄位來工作。因此,select_related在同一資料庫查詢中獲取相關物件。然而,為了避免由於跨越“多個'
單值關係
-外來鍵
和一對一關係
。prefetch_related
,另一方面,為每個關係單獨查詢,並在Python中“加入”。這允許它預取多對多
和多對一
物件,除了外來鍵
和一對一關係
,它們不能
使用select_related來完成
。
按例如下:
- 首先我們在setting.py檔案裡設定SQL查詢語句:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console' :{
'level':'DEBUG',
'class':'logging.StreamHandler',
},
},
'loggers': {
'django.db.backends': {
'handlers': ['console'],
'propagate': True,
'level':'DEBUG',
},
}
}
建立一個model.py
from django.db import models
class Province(models.Model):
"""省"""
name = models.CharField(max_length=10)
def __unicode__(self):
return self.name
class City(models.Model):
"""市"""
name = models.CharField(max_length=5)
province = models.ForeignKey(Province)
def __unicode__(self):
return self.name
class Person(models.Model):
"""人"""
firstname = models.CharField(max_length=10)
lastname = models.CharField(max_length=10)
visitation = models.ManyToManyField(City, related_name="visitor")
hometown = models.ForeignKey(City, related_name="birth")
living = models.ForeignKey(City, related_name="citizen")
def __unicode__(self):
return self.firstname + self.lastname
- 三張表裡都插入了少量的資料(需要自己新增資料)
一:select_related()
- 對於
一對一欄位(OneToOneField)
和外來鍵欄位(ForeignKey)
,可以使用select_related 來對QuerySet進行優化。 - 接下來我們看案例:
>>> City.objects.all()
# 列印結果是:有四條資料 <QuerySet [<City: Hcity>, <City: Ccity>, <City: Ecity>, <City: Qcity>]>
# 下面我們來實現外來鍵欄位的操作:
>>> citys = City.objects.all()
>>> for city in citys:
... print city.province
"""
BeiJing
BeiJing
BeiJing
BeiJing
"""
- 下面我們開啟SQL查詢的語句:(可以看到查詢了5次資料庫)
(0.000) SELECT `apps_city`.`id`, `apps_city`.`name`, `apps_city`.`province_id` FROM `apps_city`; args=()
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
BeiJing
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
BeiJing
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
BeiJing
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
BeiJing
- 下面我們用
select_related()
來檢視一下查詢的次數
>>> citys = City.objects.select_related().all()
>>> for city in citys:
... print city.province
# 看列印結果:同樣返回的4條資料
"""
BeiJing
BeiJing
BeiJing
BeiJing
"""
- 接下來我們看SQL語句的查詢:(只有一條,仔細觀察
INNER JOIN
想起了連表查詢吧)
顯然大大的減少了SQL查詢的次數
(0.000) SELECT `apps_city`.`id`, `apps_city`.`name`, `apps_city`.`province_id`, `apps_province`.`id`, `apps_province`.`name` FROM `apps_city` INNER JOIN `apps_province` ON (`apps_ci
ty`.`province_id` = `apps_province`.`id`); args=()
接下來我們給select_related() 新增:*fields 引數
- select_related() 接受
可變長引數
,每個引數是需要獲取的外來鍵(父表的內容)的欄位名
,以及外來鍵的外來鍵的欄位名
、外來鍵的外來鍵的外來鍵…
。若要選擇外來鍵的外來鍵
需要使用兩個下劃線“__”
來連線。 - 我們舉個例子:
>>> persons = Person.objects.select_related("living__province").get(pk=1)
>>> persons.living.province
# 輸出結果: <Province: BeiJing>
- 接下來看出發的SQL語句:
(0.000) SELECT `apps_person`.`id`, `apps_person`.`firstname`, `apps_person`.`lastname`, `apps_person`.`hometown_id`, `apps_person`.`living_id`, `apps_city`.`id`, `apps_city`.`name`,
`apps_city`.`province_id`, `apps_province`.`id`, `apps_province`.`name` FROM `apps_person` INNER JOIN `apps_city` ON (`apps_person`.`living_id` = `apps_city`.`id`) INNER JOIN `apps
_province` ON (`apps_city`.`province_id` = `apps_province`.`id`) WHERE `apps_person`.`id` = 1; args=(1,)
- 以上可以看出來,Django使用了2次
INNER JOIN
來完成請求,取到了city表和province表的內容,並新增到結果表的相應列,在呼叫查詢persons.living
的時候也不必再次進行SQL查詢。 - 如果未指定外來鍵則不會被新增到結果中。
>>> persons.hometown.province
(0.001) SELECT `apps_city`.`id`, `apps_city`.`name`, `apps_city`.`province_id` FROM `apps_city` WHERE `apps_city`.`id` = 1; args=(1,)
(0.000) SELECT `apps_province`.`id`, `apps_province`.`name` FROM `apps_province` WHERE `apps_province`.`id` = 2; args=(2,)
<Province: BeiJing>
- 同時,如果
不指定外來鍵
,就會進行兩次查詢。如果深度更深,查詢的次數就越多。 - 從
Diango1.7
開始,select_related()函式的作用方式改變了。在1.7版本以前
select_related()只能這麼做:
>>> persons = Person.objects.select_related("hometown__province","living__province").get(pk=1)
- 看SQL執行的查詢:
(0.003) SELECT `apps_person`.`id`, `apps_person`.`firstname`, `apps_person`.`lastname`, `apps_person`.`hometown_id`, `apps_person`.`living_id`, `apps_city`.`id`, `apps_city`.`name`,
`apps_city`.`province_id`, `apps_province`.`id`, `apps_province`.`name`, T4.`id`, T4.`name`, T4.`province_id`, T5.`id`, T5.`name` FROM `apps_person` INNER JOIN `apps_city` ON (`app
s_person`.`hometown_id` = `apps_city`.`id`) INNER JOIN `apps_province` ON (`apps_city`.`province_id` = `apps_province`.`id`) INNER JOIN `apps_city` T4 ON (`apps_person`.`living_id`
= T4.`id`) INNER JOIN `apps_province` T5 ON (T4.`province_id` = T5.`id`) WHERE `apps_person`.`id` = 1; args=(1,)
# 從下面我們可以看出來,通過外來鍵獲取到的資料,不會進行資料庫的查詢:
>>> persons.living.province
<Province: BeiJing>
>>> persons.hometown.province
<Province: BeiJing>
- 但是在1.7以上版本,可以像QuerySet的其他函式一樣進行操作:
>>> persons = Person.objects.select_related("living__province").select_related("hometown__province").get(pk=1)
- 看SQL語句的查詢:
(0.001) SELECT `apps_person`.`id`, `apps_person`.`firstname`, `apps_person`.`lastname`, `apps_person`.`hometown_id`, `apps_person`.`living_id`, `apps_city`.`id`, `apps_city`.`name`,
`apps_city`.`province_id`, `apps_province`.`id`, `apps_province`.`name`, T4.`id`, T4.`name`, T4.`province_id`, T5.`id`, T5.`name` FROM `apps_person` INNER JOIN `apps_city` ON (`app
s_person`.`hometown_id` = `apps_city`.`id`) INNER JOIN `apps_province` ON (`apps_city`.`province_id` = `apps_province`.`id`) INNER JOIN `apps_city` T4 ON (`apps_person`.`living_id`
= T4.`id`) INNER JOIN `apps_province` T5 ON (T4.`province_id` = T5.`id`) WHERE `apps_person`.`id` = 1; args=(1,)
>>> persons.living.province
<Province: BeiJing>
>>> persons.hometown.province
<Province: BeiJing>
>>>
建議大家使用1.7以上版本
二:prefetch_related()
- 下面我來們操作一下:
>>> persons = Person.objects.prefetch_related("visitation__province").get(pk=1)
- 查詢SQL語句:
(0.007) SELECT (`apps_person_visitation`.`person_id`) AS `_prefetch_related_val_person_id`, `apps_city`.`id`, `apps_city`.`name`, `apps_city`.`province_id` FROM `apps_city` INNER JO
IN `apps_person_visitation` ON (`apps_city`.`id` = `apps_person_visitation`.`city_id`) WHERE `apps_person_visitation`.`person_id` IN (1); args=(1,)
>>> persons.visitation
<django.db.models.fields.related_descriptors.ManyRelatedManager object at 0x0000000003FCAAC8>
- prefetch_related()的操作和 select_related()大概是相同的, 只是
prefetch_related()
是操作,多對多
的表格。