Scrapy框架安裝配置

1.建立一個虛擬環境,在虛擬環境中進行操作

安裝:
pip install virtualenv
​
建立:
到對應目錄下:
virtualenv 虛擬環境名稱

  進入虛擬環境:

D:\virtu\spiderenv\Scripts>activate

2.安裝twiste依賴,

https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

到該網站下載對應python版本對應位數的twisted,下載後

pip install 路徑\twisted包名

3.安裝pywin32

下載對應python版本的安裝包後安裝:

pip install 路徑\pywin32包名

或者直接安裝

安裝後在虛擬環境中執行python,輸入import win32com ,如果沒有報錯,則說明安裝成功

D:\virtu\spiderenv\Scripts>python
Python 3.6.7 (v3.6.7:6ec5cf24b7, Oct 20 2018, 13:35:33) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import win32com
 

4.安裝lxml

pip install lxml

5.安裝Scrapy

pip install scrapy

6.驗證安裝是否成功,如下情況則為安裝成功

(spiderenv) D:\virtu\spiderenv\Scripts>scrapy
Scrapy 1.5.1 - no active project
​
Usage:
  scrapy <command> [options] [args]
​
Available commands:
  bench         Run quick benchmark test
  fetch         Fetch a URL using the Scrapy downloader
  genspider     Generate new spider using pre-defined templates
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy
​
  [ more ]      More commands available when run from project directory
​
Use "scrapy <command> -h" to see more info about a command