1. 程式人生 > >django源碼分析——靜態文件staticfiles中間件

django源碼分析——靜態文件staticfiles中間件

asi tween 全局搜索 sts number 緩存 請求 store wsgi

本文環境python3.5.2,django1.10.x系列

1.在上一篇文章中已經分析過handler的處理過程,其中load_middleware就是將配置的中間件進行初始化,然後調用相應的設置方法。
django框架提供的認證,回話保持,靜態文件調試處理等都是通過以中間件的形式來處理。
2.本節就分析一下django框架提供的staticfiles中間件,該中間件分別實現了三個框架的命令,分別為collectstatic,findstatic,runserver。
其中,runserver方法是使用框架的開發者在本地調試使用的方法,使用該方式替換django.core中的runserver是為了使開發時,Django框架能夠在本地調試時處理靜態文件,這樣更有利於提升本地開發的效率。
3.下面就一起分析一下該runserver的執行過程。

分析

1.該代碼位於django/contrib/staticfiles/目錄下,首先來看management/commands/runserver.py

class Command(RunserverCommand):                                                                # 繼承自核心包的runserver
    help = "Starts a lightweight Web server for development and also serves static files."

    def add_arguments(self, parser):
        super(Command, self).add_arguments(parser)
        parser.add_argument(
            ‘--nostatic‘, action="store_false", dest=‘use_static_handler‘, default=True,
            help=‘Tells Django to NOT automatically serve static files at STATIC_URL.‘,
        )                                                                                       # 新增是否使用默認靜態文件處理handler
        parser.add_argument(
            ‘--insecure‘, action="store_true", dest=‘insecure_serving‘, default=False,
            help=‘Allows serving static files even if DEBUG is False.‘,
        )                                                                                       # 是否使用server處理靜態文件,這樣調試的環境可以訪問靜態文件

    def get_handler(self, *args, **options):
        """
        Returns the static files serving handler wrapping the default handler,
        if static files should be served. Otherwise just returns the default
        handler.
        """
        handler = super(Command, self).get_handler(*args, **options)                            # 獲取core中的runserver處理對象
        use_static_handler = options[‘use_static_handler‘]                                      # 是否使用靜態handler處理,默認使用
        insecure_serving = options[‘insecure_serving‘]                                          # 是否使用靜態handler處理靜態文件,默認不使用
        if use_static_handler and (settings.DEBUG or insecure_serving):                         # 如果使用靜態handler,並且在調試或者設置使用靜態handler處理則使用靜態handler
            return StaticFilesHandler(handler)
        return handler

首先該runserver是為了實現在處理接口的同時,處理靜態文件,所以Command繼承了core核心中的RunserverCommand類,這樣只需要在已有的基礎上,改寫是該類處理靜態文件即可。
該類又增加了兩個參數,nostatic表示不自動處理靜態文件,insecure表示就算不是調試模式Django框架也要處理靜態文件。
當調用get_handler的時候,先判斷是否配置了自動處理靜態文件,或者是否開啟了insecure模式,如果自動處理靜態文件,並且調試為true或者開啟了自動處理靜態文件,就StaticFilesHandler(handler)處理返回。
我們分析一下StaticFilesHandler(handler)
該類位於staticfiles/handlers.py中

from django.conf import settings
from django.contrib.staticfiles import utils
from django.contrib.staticfiles.views import serve
from django.core.handlers.wsgi import WSGIHandler, get_path_info
from django.utils.six.moves.urllib.parse import urlparse
from django.utils.six.moves.urllib.request import url2pathname


class StaticFilesHandler(WSGIHandler):                                          # 繼承自wsgi
    """
    WSGI middleware that intercepts calls to the static files directory, as
    defined by the STATIC_URL setting, and serves those files.
    """
    # May be used to differentiate between handler types (e.g. in a
    # request_finished signal)
    handles_files = True

    def __init__(self, application):
        self.application = application                                          # 傳入處理handler
        self.base_url = urlparse(self.get_base_url())                           # 解析配置的靜態文件路徑
        super(StaticFilesHandler, self).__init__()

    def get_base_url(self):
        utils.check_settings()                                                  # 檢查靜態文件相關配置是否正確
        return settings.STATIC_URL                                              # 返回配置中的靜態文件

    def _should_handle(self, path):
        """
        Checks if the path should be handled. Ignores the path if:

        * the host is provided as part of the base_url
        * the request‘s path isn‘t under the media path (or equal)
        """
        return path.startswith(self.base_url[2]) and not self.base_url[1]       # 路徑是否以靜態路徑開頭,並且配置文件沒有給出靜態文件的Host

    def file_path(self, url):
        """
        Returns the relative path to the media file on disk for the given URL.
        """
        relative_url = url[len(self.base_url[2]):]                              # 獲取文件的相對路徑
        return url2pathname(relative_url)

    def serve(self, request):
        """
        Actually serves the request path.
        """
        return serve(request, self.file_path(request.path), insecure=True)      # 啟動server處理靜態文件

    def get_response(self, request):
        from django.http import Http404

        if self._should_handle(request.path):                                       # 如果是靜態文件路徑則使用server處理
            try:
                return self.serve(request)
            except Http404 as e:
                if settings.DEBUG:
                    from django.views import debug
                    return debug.technical_404_response(request, e)
        return super(StaticFilesHandler, self).get_response(request)

    def __call__(self, environ, start_response):
        if not self._should_handle(get_path_info(environ)):                         # 先判斷請求url是否是靜態文件路徑
            return self.application(environ, start_response)                        # 如果不是靜態文件路徑,則正常處理
        return super(StaticFilesHandler, self).__call__(environ, start_response)    # 如果是靜態文件路徑則調用父方法處理

該類繼承自WSGIHandler,此時當調用該handler的call方法時,會調用該類的__call__,會先獲取environ中的請求路徑,判斷該url是否是配置文件中靜態文件路徑開頭。
如果是靜態文件路徑開頭則使用傳入的handler直接處理不執行一下步驟,如果是靜態文件路徑則調用該了的父類的處理方法。
只不過處理過程調用該類的get_response方法,該方法的主要作用是:

    def get_response(self, request):
        from django.http import Http404

        if self._should_handle(request.path):                                       # 如果是靜態文件路徑則使用server處理
            try:
                return self.serve(request)
            except Http404 as e:
                if settings.DEBUG:
                    from django.views import debug
                    return debug.technical_404_response(request, e)
        return super(StaticFilesHandler, self).get_response(request)

如果給url是靜態文件路徑則調用self.server方法處理,否則調用父類正常的get_response方法。
當調用self.server方法時,就使用了導入的django.contrib.staticfiles.views中的server方法處理。
分析該server的內容如下:

def serve(request, path, insecure=False, **kwargs):
    """
    Serve static files below a given point in the directory structure or
    from locations inferred from the staticfiles finders.

    To use, put a URL pattern such as::

        from django.contrib.staticfiles import views

        url(r‘^(?P<path>.*)$‘, views.serve)

    in your URLconf.

    It uses the django.views.static.serve() view to serve the found files.
    """
    if not settings.DEBUG and not insecure:                                     # 再次檢查配置是否為調試模式,是否設置框架處理靜態文件
        raise Http404
    normalized_path = posixpath.normpath(unquote(path)).lstrip(‘/‘)             # 解析url並分解出路徑,並去除最左邊的/
    absolute_path = finders.find(normalized_path)                               # 查找靜態文件,如果查找到文件就返回文件的絕對地址
    if not absolute_path:
        if path.endswith(‘/‘) or path == ‘‘:
            raise Http404("Directory indexes are not allowed here.")
        raise Http404("‘%s‘ could not be found" % path)
    document_root, path = os.path.split(absolute_path)                          # 返回匹配上的文件夾,與文件    
    return static.serve(request, path, document_root=document_root, **kwargs)   # 處理該靜態文件的response

再次檢查是否是處理靜態文件,如果不是則直接報錯404,否則調用finders去查找該靜態文件,我們繼續查看finders.find方法。
當找到該靜態文件時候,調用static.server處理該靜態文件。

def find(path, all=False):
    """
    Find a static file with the given path using all enabled finders.

    If ``all`` is ``False`` (default), return the first matching
    absolute path (or ``None`` if no match). Otherwise return a list.
    """
    searched_locations[:] = []                                          
    matches = []
    for finder in get_finders():                                        # 獲取配置的finder類
        result = finder.find(path, all=all)                             # 通過finder來查找靜態文件
        if not all and result:                                          # 如果不是全部查找,找到對應文件就返回數據
            return result
        if not isinstance(result, (list, tuple)):                       # 如果是全部查找,而result不是列表或元組,則手動轉換
            result = [result]
        matches.extend(result)                                          # 將查找到的結果,加入到列表中
    if matches:                                                         # 如果有結果就返回
        return matches
    # No match.
    return [] if all else None                                          # 如果全局查找就返回空列表,否則返回None

get_finders()該函數返回finder對象,我們查看該方法

def get_finders():
    for finder_path in settings.STATICFILES_FINDERS:                    # 獲取配置文件中的查找文件對象,默認配置在conf/global_settings.py中 
        yield get_finder(finder_path)                                   # 獲取finder對象   django.contrib.staticfiles.finders.FileSystemFinder,django.contrib.staticfiles.finders.AppDirectoriesFinder


@lru_cache.lru_cache(maxsize=None)
def get_finder(import_path):
    """
    Imports the staticfiles finder class described by import_path, where
    import_path is the full Python path to the class.
    """
    Finder = import_string(import_path)                                         # 通過配置的路徑,導入finder
    if not issubclass(Finder, BaseFinder):                                      # 檢查導入Finder是否是BaseFinder子類
        raise ImproperlyConfigured(‘Finder "%s" is not a subclass of "%s"‘ %
                                   (Finder, BaseFinder))
    return Finder()                                                             # 返回finder實例

如果配置文件中沒有配置,則使用conf/global_settings.py中的配置文件,配置的兩個類就位於該FileSystemFinder和AppDirectoriesFinder兩個類。
FileSystemFinder主要是查找文件系統的靜態文件,AppDirectoriesFinder主要是查找位於app應用中的靜態文件。
其中FileSystemFinder分析如下:

class FileSystemFinder(BaseFinder):
    """
    A static files finder that uses the ``STATICFILES_DIRS`` setting
    to locate files.
    """
    def __init__(self, app_names=None, *args, **kwargs):
        # List of locations with static files
        self.locations = []
        # Maps dir paths to an appropriate storage instance
        self.storages = OrderedDict()
        if not isinstance(settings.STATICFILES_DIRS, (list, tuple)):            # 檢查配置文件中的靜態文件處理是否是列表或者元組
            raise ImproperlyConfigured(
                "Your STATICFILES_DIRS setting is not a tuple or list; "
                "perhaps you forgot a trailing comma?")
        for root in settings.STATICFILES_DIRS:                                  # 獲取配置的文件路徑
            if isinstance(root, (list, tuple)):                                 # 如果配置的靜態路徑是列表或者元組
                prefix, root = root                                             # 獲取前綴與路徑
            else: 
                prefix = ‘‘                                                     # 如果不是列表或元組則為‘‘
            if settings.STATIC_ROOT and os.path.abspath(settings.STATIC_ROOT) == os.path.abspath(root):   # 判斷靜態文件static是否與media重合
                raise ImproperlyConfigured(
                    "The STATICFILES_DIRS setting should "
                    "not contain the STATIC_ROOT setting")
            if (prefix, root) not in self.locations:                            # 如果解析出來的前綴與路徑不再loactions中則添加進去
                self.locations.append((prefix, root))
        for prefix, root in self.locations:                                     # 遍歷locations
            filesystem_storage = FileSystemStorage(location=root)               # 給每個值生成一個FileSystemStorage實例
            filesystem_storage.prefix = prefix
            self.storages[root] = filesystem_storage                            # 將生成實例保存進字典中
        super(FileSystemFinder, self).__init__(*args, **kwargs)

    def find(self, path, all=False):
        """
        Looks for files in the extra locations
        as defined in ``STATICFILES_DIRS``.
        """
        matches = []
        for prefix, root in self.locations:                                     # 根據locations的值匹配
            if root not in searched_locations:                                  # 如果root不再全局搜索路徑中,則添加到搜索路徑中
                searched_locations.append(root)
            matched_path = self.find_location(root, path, prefix)               # 查找文件
            if matched_path:                                                    # 如果找到
                if not all:                                                     # 如果不是查找全部則找到第一個就返回
                    return matched_path
                matches.append(matched_path)                                    # 如果查找全部則添加到返回數組中
        return matches

    def find_location(self, root, path, prefix=None):
        """
        Finds a requested static file in a location, returning the found
        absolute path (or ``None`` if no match).
        """
        if prefix:                                                              # 是否有前綴
            prefix = ‘%s%s‘ % (prefix, os.sep)                                  # 添加前綴加系統的分隔符, ‘/‘
            if not path.startswith(prefix):                                     # 如果路徑不是前綴開頭則直接返回
                return None     
            path = path[len(prefix):]                                           # 獲取除去前綴的路徑
        path = safe_join(root, path)                                            # 獲取最終的文件路徑
        if os.path.exists(path):
            return path

    def list(self, ignore_patterns):
        """
        List all files in all locations.
        """
        for prefix, root in self.locations:                                     # 獲取所有文件的設置的靜態文件處理
            storage = self.storages[root]
            for path in utils.get_files(storage, ignore_patterns):
                yield path, storage

主要是查找配置的靜態文件查找目錄,匹配當前是否找到文件。
AppDirectoriesFinder主要是查找配置在app中的靜態文件。

class AppDirectoriesFinder(BaseFinder):
    """
    A static files finder that looks in the directory of each app as
    specified in the source_dir attribute.
    """
    storage_class = FileSystemStorage
    source_dir = ‘static‘                                                       # 源文件夾

    def __init__(self, app_names=None, *args, **kwargs):
        # The list of apps that are handled
        self.apps = []                                                          # 需要查找文件的應用
        # Mapping of app names to storage instances
        self.storages = OrderedDict()                                           # 存儲需要查找的應用
        app_configs = apps.get_app_configs()                                    # 獲取所有app的配置
        if app_names:                                                           # 如果有傳入值,
            app_names = set(app_names)
            app_configs = [ac for ac in app_configs if ac.name in app_names]    # 將app_configs設置為在默認配置中的項目
        for app_config in app_configs:                                          # 遍歷篩選出來的應用配置
            app_storage = self.storage_class(
                os.path.join(app_config.path, self.source_dir))                 # 將應用下面的static目錄初始化一個app_storage對象
            if os.path.isdir(app_storage.location):                             # 檢查生成的靜態文件夾是否存在
                self.storages[app_config.name] = app_storage                    # 根據配置應用的名稱對應,app_storage對象
                if app_config.name not in self.apps:                            # 如果app沒在app列表中,則將該應用的名稱添加到列表
                    self.apps.append(app_config.name)
        super(AppDirectoriesFinder, self).__init__(*args, **kwargs)

    def list(self, ignore_patterns):
        """
        List all files in all app storages.
        """
        for storage in six.itervalues(self.storages):                           # 叠代列表中的應用下的app_storage實例
            if storage.exists(‘‘):  # check if storage location exists          # 檢查app_storage實例是否存在當前目錄
                for path in utils.get_files(storage, ignore_patterns):          # 獲取返回的路徑
                    yield path, storage                                         # 返回當前路徑,與app_storage實例

    def find(self, path, all=False):
        """
        Looks for files in the app directories.
        """
        matches = []
        for app in self.apps:                                                   # 查找app中的文件
            app_location = self.storages[app].location                          # 獲取app的絕對路徑
            if app_location not in searched_locations:                          # 如果當前路徑不在搜索路徑中則添加到全局搜索列表中
                searched_locations.append(app_location)
            match = self.find_in_app(app, path)                                 # 在app中的路徑中查找
            if match:                                                           # 如果匹配
                if not all:                                                     # 如果不是全局搜索,則立馬返回第一個匹配的
                    return match
                matches.append(match)                                           # 如果是全局搜索則添加到返回列表中
        return matches                                                          # 返回所有匹配的數據

    def find_in_app(self, app, path):
        """
        Find a requested static file in an app‘s static locations.
        """
        storage = self.storages.get(app)                                        # 獲取app_storage實例
        if storage: 
            # only try to find a file if the source dir actually exists
            if storage.exists(path):                                            # 檢查當前文件是否存在
                matched_path = storage.path(path)                               # 返回匹配後的文件路徑
                if matched_path:
                    return matched_path

當調用find時會調用find_in_app方法,該方法中的每個實例都是FileSystemStorage,storage.path(path)調用該方法

    def path(self, name):
        return safe_join(self.location, name)

查找當前文件夾路徑,當找到時就返回。
當通過這兩種方式找到文件時,返回文件時,

    document_root, path = os.path.split(absolute_path)                          # 返回匹配上的文件夾,與文件    
    return static.serve(request, path, document_root=document_root, **kwargs)   # 處理該靜態文件的response

執行到該方法,django.views.static.server的代碼為

def serve(request, path, document_root=None, show_indexes=False):
    """
    Serve static files below a given point in the directory structure.

    To use, put a URL pattern such as::

        from django.views.static import serve

        url(r‘^(?P<path>.*)$‘, serve, {‘document_root‘: ‘/path/to/my/files/‘})

    in your URLconf. You must provide the ``document_root`` param. You may
    also set ``show_indexes`` to ``True`` if you‘d like to serve a basic index
    of the directory.  This index view will use the template hardcoded below,
    but if you‘d like to override it, you can create a template called
    ``static/directory_index.html``.
    """
    path = posixpath.normpath(unquote(path)).lstrip(‘/‘)                                
    fullpath = safe_join(document_root, path)                                           # 獲取文件的全路徑
    if os.path.isdir(fullpath):                                                         # 判斷是否是文件夾
        if show_indexes:                                                                # 如果顯示文件的樹結構則顯示
            return directory_index(path, fullpath)                              
        raise Http404(_("Directory indexes are not allowed here."))
    if not os.path.exists(fullpath):                                                    # 如果不存在則報錯
        raise Http404(_(‘"%(path)s" does not exist‘) % {‘path‘: fullpath})
    # Respect the If-Modified-Since header.
    statobj = os.stat(fullpath)                                                         # 獲取文件的狀態
    if not was_modified_since(request.META.get(‘HTTP_IF_MODIFIED_SINCE‘),               # 判斷該文件是否已經客戶端緩存過期,如果還在緩存期就直接返回
                              statobj.st_mtime, statobj.st_size):
        return HttpResponseNotModified()
    content_type, encoding = mimetypes.guess_type(fullpath)                             # 獲取文件的文件類型,獲取文件的編碼格式
    content_type = content_type or ‘application/octet-stream‘                           # 設置返回文件的文件類型
    response = FileResponse(open(fullpath, ‘rb‘), content_type=content_type)            # 將文件讀入到緩存流中,並返回response
    response["Last-Modified"] = http_date(statobj.st_mtime)                             # 添加最後的文件modified的時間
    if stat.S_ISREG(statobj.st_mode):                                                   # 是否是一般文件
        response["Content-Length"] = statobj.st_size                                    # 設置返回文件的長度
    if encoding: 
        response["Content-Encoding"] = encoding                                         # 如果返回有文件的編碼格式就設置文件的編碼格式
    return response  

當找到文件後,獲取到文件路徑後,FileResponse來進行文件返回
FileResponse的代碼為

class FileResponse(StreamingHttpResponse):
    """
    A streaming HTTP response class optimized for files.
    """
    block_size = 4096

    def _set_streaming_content(self, value):                                        # 重寫父類的設置stream方法
        if hasattr(value, ‘read‘):                                                  # 如果有read方法
            self.file_to_stream = value                                             # 將file_to_stream設值
            filelike = value                                                        # 
            if hasattr(filelike, ‘close‘):                                          # 如果有close方法,添加到完成時關閉
                self._closable_objects.append(filelike)
            value = iter(lambda: filelike.read(self.block_size), b‘‘)               # 叠代讀文件的block_size大小的文件,直到讀為空為止
        else:
            self.file_to_stream = None
        super(FileResponse, self)._set_streaming_content(value)                     # 調用父類方法處理value

 我們查看StreamingHttpResponse

class StreamingHttpResponse(HttpResponseBase):
    """
    A streaming HTTP response class with an iterator as content.

    This should only be iterated once, when the response is streamed to the
    client. However, it can be appended to or replaced with a new iterator
    that wraps the original content (or yields entirely new content).
    """

    streaming = True

    def __init__(self, streaming_content=(), *args, **kwargs):
        super(StreamingHttpResponse, self).__init__(*args, **kwargs)
        # `streaming_content` should be an iterable of bytestrings.
        # See the `streaming_content` property methods.
        self.streaming_content = streaming_content                                  # 設置stream調用streaming_content.setter方法

    @property
    def content(self):
        raise AttributeError(
            "This %s instance has no `content` attribute. Use "
            "`streaming_content` instead." % self.__class__.__name__
        )

    @property
    def streaming_content(self):
        return map(self.make_bytes, self._iterator)

    @streaming_content.setter
    def streaming_content(self, value):
        self._set_streaming_content(value)                                          # 調用_set_streaming_content

    def _set_streaming_content(self, value):
        # Ensure we can never iterate on "value" more than once.
        self._iterator = iter(value)                                                # 設置可叠代對象
        if hasattr(value, ‘close‘):                                                 # 如果對象有close方法則在叠代結束後關閉
            self._closable_objects.append(value)

    def __iter__(self):
        return self.streaming_content                                               # 叠代streaming_content

    def getvalue(self):
        return b‘‘.join(self.streaming_content)

通過將response生成一個可叠代對象,將返回的數據進行分塊發送,文件塊大小為4096,此時就將文件內容分塊發送出去,此時一個靜態文件的響應過程完成。

django源碼分析——靜態文件staticfiles中間件