實例1：京東商品頁面的爬取

import requests

url="https://item.jd.com/2967929.html"

try:

r=requests.get(url)

r.raise_for_status()

r.encoding=r.apparent_encoding

print(r.text[:1000])

except:

print("爬取失敗")

實例2：亞馬遜商品頁面的爬取

import requests

url="https://www.amazon.cn/gp/product/B01M8L5Z3Y"

try:

kv={‘user-agent‘:‘Mozilla/5.0‘}

r=requests.get(url,headers=kv)

r.raise_for_status()

r.encoding=r.apparent_encoding

print(r.text[1000:2000])

except:

print("爬取失敗")

實例3：百度/360搜索關鍵字提交

import requests

keyword="Python"

try:

kv={‘q‘:keyword}

r=requests.get("http://www.baidu.com/s",params=kv)

print(r.request.url)

r.raise_for_status()

print(len(r.text))

except:

print("爬取失敗")

實例4：網絡圖片的爬取和存儲

import requests

import os

url="http://image.nationalgeographic.com.cn/2017/0211/20170211061910157.jpg"

root="D://pics"

try:

if not os.path.exists(root):

os.mkdir(root)

if not os.path.exists(path):

r=requests.get(url)

with open(path,‘wb‘) as f:

f.write(r.content)

f.close()

print("文件保存成功")

else:

print("文件已存在")

except:

print("爬取失敗")

實例5：IP地址歸屬地的自動查詢

import requests

url="http://m.ip138.com/ip.asp?ip="

try:

r=requests.get(url+‘202.204.80.112‘)

r.raise_for_status()

r.encoding=r.apparent_encoding

print(r.text[-500:])

except:

print("爬取失敗")

Python網絡爬蟲與信息提取-Requests庫網絡爬去實戰