1. 程式人生 > >記一次Linux下安裝pyspider的過程

記一次Linux下安裝pyspider的過程

首先執行

pip install pyspider

此時系統提示

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-Lau0Qp/pycurl/
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

這是我pip版本的問題,執行

sudo python -m pip install --upgrade pip
升級pip

繼續執行

 sudo pip install pyspider

來安裝pyspider,此時報錯:

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-szUHrj/pycurl/

原因是pyspider的依賴庫未安裝,需要執行

sudo apt-get install python python-dev python-distribute python-pip libcurl4-openssl-dev libxml2-dev libxslt1-dev python-lxml

命令來安裝以下支援類庫

本來興沖沖的以為可以正常安裝pyspider了,可繼續執行 sudo pip install pyspider時系統報錯資訊為:

 compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-B8gZjb/pycurl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-aDBSCP-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-B8gZjb/pycurl/

原因是因為安裝工具包時需要抓取網頁因而要處理 https,而處理 https 又依賴加解密演算法(即 cryptography 包),而 cryptography 又依賴傅立葉變換的演算法以及相應的編譯環境。Ubuntu 16.04 預設沒有安裝 libffi-dev 和 libssl-dev,gcc 也不一定安裝,而目標安裝包又沒有將相關軟體包記到依賴列表裡,因此需要先手動安裝:

sudo apt-get install libssl-dev libffi-dev build-essential 

此時再執行 sudo pip install pyspider 就OK了!

因此Linux下安裝要執行以下幾個命令,(劃重點!)

sudo python -m pip install --upgrade pip
sudo apt-get install python python-dev python-distribute python-pip libcurl4-openssl-dev libxml2-dev libxslt1-dev python-lxml
sudo apt-get install libssl-dev libffi-dev build-essential
sudo pip install pyspider