1. 程式人生 > >Python3 用 requests 2.x 下載大文件

Python3 用 requests 2.x 下載大文件

Python requests 大文件 爬蟲

【環境】

  • OS:Windows 10 x64

  • Python:3.6.5 x64

  • requests:2.18.4


【代碼】

# encoding=utf-8
# author: walker
# date: 2018-06-11
# summary: 使用 requests 下載大文件

import time
import requests

# 下載一個大文件
def DownOneFile(srcUrl, localFile):
    print('%s\n --->>>\n  %s' % (srcUrl, localFile))
    
    startTime = time.time()
    with requests.get(srcUrl, stream=True) as r:
        contentLength = int(r.headers['content-length'])
        line = 'content-length: %dB/ %.2fKB/ %.2fMB'
        line = line % (contentLength, contentLength/1024, contentLength/1024/1024)
        print(line)
        downSize = 0
        with open(localFile, 'wb') as f:
            for chunk in r.iter_content(8192):
                if chunk:
                    f.write(chunk)
                downSize += len(chunk)
                line = '%d KB/s - %.2f MB, 共 %.2f MB'
                line = line % (downSize/1024/(time.time()-startTime), downSize/1024/1024, contentLength/1024/1024)
                print(line, end='\r')
                if downSize >= contentLength:
                    break
        timeCost = time.time() - startTime
        line = '共耗時: %.2f s, 平均速度: %.2f KB/s'
        line = line % (timeCost, downSize/1024/timeCost)
        print(line)

if __name__ == '__main__':
    srcUrl = r'http://cachefly.cachefly.net/100mb.test'
    localFile = r'D:\Python3Project\test\out\out.test'
    DownOneFile(srcUrl, localFile)


【效果圖】

技術分享圖片


【相關閱讀】

  • requests 官方文檔:http://docs.python-requests.org


*** walker ***


Python3 用 requests 2.x 下載大文件