1. 程式人生 > >negroni-gzip原始碼分析

negroni-gzip原始碼分析

gibhub地址:針對negroni的gzip

作業內容:支援了gzip的伺服器小程式

這是一個為Negroni設計的gzip壓縮處理中介軟體,需要用到已有的compress中的gzip。原始碼共有一百多行並不多,下面記錄一下對gzip原始碼的分析與理解。

首先gzip是幹啥用的?gzip是一種壓縮方式。瀏覽器和web伺服器之間為了減少傳輸鏈路上的檔案的大小,瀏覽器和伺服器在傳送資料的時候會將資料以某種方式進行壓縮,使用什麼壓縮方式則記錄在報文頭裡面。這裡的gzip便是一種壓縮方式。並不是所有的瀏覽器和伺服器都能支援所有的壓縮方式。比如說瀏覽器支援gzip壓縮方式,即瀏覽器能夠解析gzip壓縮後的內容,那麼他會在請求中傳送Accept-Encoding請求報頭值為"gzip"  表明瀏覽器支援gzip這種壓縮方式,web伺服器根據讀取Accept-Encoding請求報頭的值來判斷瀏覽器是否接受壓縮的內容,伺服器發現瀏覽器能夠解析gzip壓縮後的內容之後就會對要傳送的資料進行gzip壓縮再發送到客戶端,同時設定Content-Encoding實體報頭值為gzip以告知瀏覽器實體正文采用了gzip的壓縮編碼。

那麼這裡的negroni-gzip就是一箇中間件,用來使regroni搭建的伺服器能夠支援gzip壓縮。

原始碼分析:

  • const資料內容
    // These compression constants are copied from the compress/gzip package.
    const (
    	encodingGzip = "gzip"
    
    	headerAcceptEncoding  = "Accept-Encoding"
    	headerContentEncoding = "Content-Encoding"
    	headerContentLength   = "Content-Length"
    	headerContentType     = "Content-Type"
    	headerVary            = "Vary"
    	headerSecWebSocketKey = "Sec-WebSocket-Key"
    
    	BestCompression    = gzip.BestCompression
    	BestSpeed          = gzip.BestSpeed
    	DefaultCompression = gzip.DefaultCompression
    	NoCompression      = gzip.NoCompression
    )
    

    上面的資料代表了接下來的程式碼中各種const常量代表的值。裡面使用了compress/gzip裡面的值,其中NoCompression = 0,BestSpeed = 1,BestCompression = 9,DefaultCompression = -1。這些值代表壓縮的level,不能超BestCompression。

  • // gzipResponseWriter is the ResponseWriter that negroni.ResponseWriter is
    // wrapped in.
    type gzipResponseWriter struct {
    	w *gzip.Writer
    	negroni.ResponseWriter
    	wroteHeader bool
    }

    結構體gzipResponseWriter,wroteHeader代表response(即響應內容)是否已經編碼 。

  • // Check whether underlying response is already pre-encoded and disable
    // gzipWriter before the body gets written, otherwise encoding headers
    func (grw *gzipResponseWriter) WriteHeader(code int) {
    	headers := grw.ResponseWriter.Header()
    	if headers.Get(headerContentEncoding) == "" {
    		headers.Set(headerContentEncoding, encodingGzip)
    		headers.Add(headerVary, headerAcceptEncoding)
    	} else {
    		grw.w.Reset(ioutil.Discard)
    		grw.w = nil
    	}
    
    	// Avoid sending Content-Length header before compression. The length would
    	// be invalid, and some browsers like Safari will report
    	// "The network connection was lost." errors
    	grw.Header().Del(headerContentLength)
    
    	grw.ResponseWriter.WriteHeader(code)
    	grw.wroteHeader = true
    }
    

    WriteHeader函式,如果要傳送給客戶端的響應內容未預編碼,則採用gzip壓縮方式壓縮後再發送到客戶端,同時設定Content-Encoding實體報頭值為gzip。否則在寫之前令gzipWriter失效,使得它對任何寫呼叫無條件成功。

  • // Write writes bytes to the gzip.Writer. It will also set the Content-Type
    // header using the net/http library content type detection if the Content-Type
    // header was not set yet.
    func (grw *gzipResponseWriter) Write(b []byte) (int, error) {
    	if !grw.wroteHeader {
    		grw.WriteHeader(http.StatusOK)
    	}
    	if grw.w == nil {
    		return grw.ResponseWriter.Write(b)
    	}
    	if len(grw.Header().Get(headerContentType)) == 0 {
    		grw.Header().Set(headerContentType, http.DetectContentType(b))
    	}
    	return grw.w.Write(b)
    }

    向gzip.Writer中寫入位元組流。如果報頭沒寫的話就寫報頭,如果不是使用的gzip壓縮的話就用ResponseWriter寫。如果頭的Content-Type還沒有被設定,則用net/http庫中的型別檢測來完成設定。 

  • type gzipResponseWriterCloseNotifier struct {
    	*gzipResponseWriter
    }

     一個簡單的資料型別定義。

  • func (rw *gzipResponseWriterCloseNotifier) CloseNotify() <-chan bool {
    	return rw.ResponseWriter.(http.CloseNotifier).CloseNotify()
    }
    
    func newGzipResponseWriter(rw negroni.ResponseWriter, w *gzip.Writer) negroni.ResponseWriter {
    	wr := &gzipResponseWriter{w: w, ResponseWriter: rw}
    
    	if _, ok := rw.(http.CloseNotifier); ok {
    		return &gzipResponseWriterCloseNotifier{gzipResponseWriter: wr}
    	}
    
    	return wr
    }

    gzipResponseWriter,用於寫入gzip編碼後的資料。

  • // handler struct contains the ServeHTTP method
    type handler struct {
    	pool sync.Pool
    }

    一個sync.Pool物件就是一組臨時物件的集合,Pool用於儲存那些被分配了但是沒有被使用,而未來可能會使用的值,以減小垃圾回收的壓力。

  • // Gzip returns a handler which will handle the Gzip compression in ServeHTTP.
    // Valid values for level are identical to those in the compress/gzip package.
    func Gzip(level int) *handler {
    	h := &handler{}
    	h.pool.New = func() interface{} {
    		gz, err := gzip.NewWriterLevel(ioutil.Discard, level)
    		if err != nil {
    			panic(err)
    		}
    		return gz
    	}
    	return h
    }

    Gzip返回一個handler來處理在ServeHTTP中的壓縮,需要呼叫gzip庫的NewWriterLevel方法。

  • // ServeHTTP wraps the http.ResponseWriter with a gzip.Writer.
    func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) {
    	// Skip compression if the client doesn't accept gzip encoding.
    	if !strings.Contains(r.Header.Get(headerAcceptEncoding), encodingGzip) {
    		next(w, r)
    		return
    	}
    
    	// Skip compression if client attempt WebSocket connection
    	if len(r.Header.Get(headerSecWebSocketKey)) > 0 {
    		next(w, r)
    		return
    	}
    
    	// Retrieve gzip writer from the pool. Reset it to use the ResponseWriter.
    	// This allows us to re-use an already allocated buffer rather than
    	// allocating a new buffer for every request.
    	// We defer g.pool.Put here so that the gz writer is returned to the
    	// pool if any thing after here fails for some reason (functions in
    	// next could potentially panic, etc)
    	gz := h.pool.Get().(*gzip.Writer)
    	defer h.pool.Put(gz)
    	gz.Reset(w)
    
    	// Wrap the original http.ResponseWriter with negroni.ResponseWriter
    	// and create the gzipResponseWriter.
    	nrw := negroni.NewResponseWriter(w)
    	grw := newGzipResponseWriter(nrw, gz)
    
    	// Call the next handler supplying the gzipResponseWriter instead of
    	// the original.
    	next(grw, r)
    
    	gz.Close()
    }

    處理handler中壓縮請求的函式:如果客戶端不支援gzip編碼則跳過並不壓縮。如果客戶端在嘗試WebSocket連線時,也會不壓縮。接下來從pool中遍歷writer,如果之後遇到的錯誤,就通過defer的方法,返回pool,用ResponseWriter重置,這讓我們可以再利用已經被分配的buffer,而不是為每一個單獨的請求開闢新的buffer。最後用negroni.ResponseWriter打包原來的ResponseWriter,並建立一個新的gzipResponseWriter,並且呼叫下一個handler。最後關閉gz。