1. 程式人生 > >selenium+python爬取數據跳轉網頁

selenium+python爬取數據跳轉網頁

tns pen 得出 items search ems find option http

項目要做一個四個層級欄的數據抓取,而且點擊查詢後數據會在新跳出的網頁。

源碼如下 註釋解釋

from selenium import webdriver
import selenium
#from time import sleep as sp

url=http://202.127.42.157/moazzys/nongqing.aspx
site=webdriver.Chrome()
site.get(url)

#data={}
mainhandle=site.current_window_handle#主頁面句柄  每個瀏覽器標簽頁都有一個句柄
years=site.find_elements_by_xpath(
//select[@id="ContentPlaceHolder1_year"]/option) co_year=len(years) for d in range(co_year): years[d].click() provs=site.find_elements_by_xpath(//select[@id="ContentPlaceHolder1_prov"]/option) co_prov=len(provs) for c in range(co_prov): provs[c].click() items
=site.find_elements_by_xpath(//select[@id="ContentPlaceHolder1_item"]/option) co_item=len(items) for b in range(co_item): items[b].click() types=site.find_elements_by_xpath(//select[@id="ContentPlaceHolder1_type"]/option) co_type=len(types)
for a in range(co_type): types[a].click() #點擊進入新窗口 site.find_element_by_id(ContentPlaceHolder1_btnsearch).click() handles = site.window_handles for handle in handles:# 輪流得出標簽頁的句柄 切換窗口 因為只有兩個標簽頁實際是假for循環 if handle!=mainhandle: site.switch_to_window(handle) #獲得數據 try: raw=site.find_element_by_xpath(//tr/td[@align="right"]) print (raw.text) #data.append(raw.text) except Exception as e: print("無該數據") #獲得數據 site.close() #關閉當前標簽頁 site.switch_to_window(mainhandle)#回到原來標簽頁 types=site.find_elements_by_xpath(//select[@id="ContentPlaceHolder1_type"]/option) types[a].click() items=site.find_elements_by_xpath(//select[@id="ContentPlaceHolder1_item"]/option) items[b].click() provs=types=site.find_elements_by_xpath(//select[@id="ContentPlaceHolder1_prov"]/option) provs[c].click() year=types=site.find_elements_by_xpath(//select[@id="ContentPlaceHolder1_year"]/option) years[d].click() site.quit()

selenium+python爬取數據跳轉網頁