selenium自動化測試工具python筆試面試專案實戰7 書籍下載

摘要： 下載smtebooks的IT書單 smtebooks是最好用的英文IT書籍網站，書籍涵蓋Programming & IT, Business, Magazines, Ebooks, History, Medical, Art, Non-fiction, Academic, Text...

下載smtebooks的IT書單

smtebooks是最好用的英文IT書籍網站，書籍涵蓋Programming & IT, Business, Magazines, Ebooks, History, Medical, Art, Non-fiction, Academic, Textbooks, Cooking, SEO, Science & Math, Travel & Tourism等多個方面。

現在請爬取 ofollow,noindex">https://smtebooks.net/category/programming-it 的所有書籍。

• 參考答案

#!/usr/bin/python3
# -*- coding: utf-8 -*-
# 討論釘釘免費群21745728 qq群144081101 567351477
# CreateDate: 2018-10-22
import time
import re
from selenium import webdriver
import pandas as pd

def find_address(browser):
valids = []
url_base = 'https://smtebooks.net/Category/programming-it?page='
for page in range(1,3):
time.sleep(2)
browser.get(url_base + str(page))
books = re.findall(r'<a\s+href="(/book/\S+)">(.*?)</a>', browser.page_source)
print(books)
if not books:
break
valids = valids + books
return valids

browser = webdriver.Chrome()
results = find_address(browser)

df = pd.DataFrame(results)
df.to_csv('address.csv')
browser.quit()

該書籍列表比較大，已經下載好儲存在： smtebooks IT類書籍列表 2018-10-22.csv