1. 程式人生 > >Python爬蟲之模擬登陸知乎

Python爬蟲之模擬登陸知乎

在chrome瀏覽器下抓取登陸過程的包(注意把Preserve log勾上):


表單的結構主要包括_xsrf, password, phone_num

我們要找到_xsrf的值,重新載入zhihu.com之後我們可以發現Response裡面有_xsrf


我們就可以把_xsrf的值讀取出來

然後set一下cookies,就可以模擬登陸知乎了。

import requests
from bs4 import BeautifulSoup

zhihu_url = 'http://www.zhihu.com'
headers = {
        'Referer': 'http://www.zhihu.com/',
        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/55.0.2883.87 Chrome/55.0.2883.87 Safari/537.36'}
#get xsrf
xsrf = BeautifulSoup(requests.get(zhihu_url, headers = headers).content, 'html.parser').find('input')['value']

#get cookies
data = {
        '_xsrf': xsrf,
        'phone_num': '***********',
        'password': '**********'}

loginurl = 'https://www.zhihu.com/login/phone_num'
cookies = requests.post(loginurl, data = data, headers = headers).cookies

#login
url = 'https://www.zhihu.com/question/55940910'
html = requests.get(url, headers = headers, cookies = cookies)
soup = BeautifulSoup(html.content, 'html.parser')
titles = soup.select(r'.QuestionHeader-title')
print('title: ', titles[0].text)