centos7 安裝 pyspider 出現的一系列問題及解決方案集合
先安裝python3 和 pip3
wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz
安裝zlib-devel包(後面安裝pip需要用到,這裏先下載，後面就不用重復編譯)
yum install zlib-devel
tar -xvf Python-3.6.5.tgz
yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel
./configure --prefix=/usr/local/python3/ --with-zlib=/usr/include --enable-loadable-sqlite-extensions
make && make install
ln -s /usr/local/python3/bin/python3 /usr/local/bin/python3
pip3 安裝
wget --no-check-certificate https://pypi.python.org/packages/source/p/pip/pip-8.0.2.tar.gz#md5=3a73c4188f8dbad6a1e6f6d44d117eeb
tar -zxvf pip-8.0.2.tar.gz
cd pip-8.0.2
sudo python3 setup.py build
sudo python3 setup.py install
sudo ln -s /usr/local/python3.4/bin/pip3 /usr/bin/pip3
出現的錯誤
Collecting pycurl (from pyspider)
Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "/tmp/pip-build-ffrwekm7/pycurl/setup.py", line 223, in configure_unix
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "/usr/local/python3/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/local/python3/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: ‘curl-config‘: ‘curl-config‘
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-ffrwekm7/pycurl/setup.py", line 913, in <module>
ext = get_extension(sys.argv, split_extension_source=split_extension_source)
File "/tmp/pip-build-ffrwekm7/pycurl/setup.py", line 582, in get_extension
ext_config = ExtensionConfiguration(argv)
File "/tmp/pip-build-ffrwekm7/pycurl/setup.py", line 99, in __init__
self.configure()
File "/tmp/pip-build-ffrwekm7/pycurl/setup.py", line 227, in configure_unix
raise ConfigurationError(msg)
__main__.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory: ‘curl-config‘: ‘curl-config‘
在安裝 pyspider 的時候我就遇到了這個問題， pyspider 依賴 pycurl 這個庫，而 pycurl 要求系統中存在相對應的庫。(如果系統自動安裝我們要先卸載)
Collecting pycurl
Using cached https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz
Complete output from command python setup.py egg_info:
Using curl-config (libcurl 7.29.0)
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-5y2xk7rz/pycurl/setup.py", line 913, in <module>
ext = get_extension(sys.argv, split_extension_source=split_extension_source)
File "/tmp/pip-install-5y2xk7rz/pycurl/setup.py", line 582, in get_extension
ext_config = ExtensionConfiguration(argv)
File "/tmp/pip-install-5y2xk7rz/pycurl/setup.py", line 99, in __init__
self.configure()
File "/tmp/pip-install-5y2xk7rz/pycurl/setup.py", line 316, in configure_unix
specify the SSL backend manually.‘‘‘)
__main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using. Please see PycURL documentation for how to specify the SSL backend manually.
---------------------------------------------------------------------
pip remove pycurl
`yum install libcurl-devel`
export PYCURL_SSL_LIBRARY=openssl >> ~/.bashrc
source ~/.bashrc
pip3 install pycurl
--------------------------------------------------------------------------
ubuntu的方案
sudo apt-get install libssl-dev libcurl4-openssl-dev python-dev
pip3 install pyspider
引用頁面
https://blog.csdn.net/blueheart20/article/details/81626516
https://github.com/binux/pyspider
http://www.cnblogs.com/yunlongaimeng/p/9947228.html
centos7 安裝 pyspider 出現的一系列問題及解決方案集合