1. 程式人生 > >windows下安裝scrapy問題解決辦法

windows下安裝scrapy問題解決辦法

 Scrapy是爬蟲界大名鼎鼎的存在,在Linux下安裝非常順利,但是在Windows下確實非常多的問題發生,這裡記錄存在的各類問題以及如何解決。

1.   安裝Scrapy失敗
     直接在windows下安裝scrapy大概率會是失敗的,主要原因是各類依賴包未必全部安裝,故依據具體的問題而定。
     >> pip install scrapy
2.  openssl 
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\Scripts\scrapy-script.py", line 5, in <module>
    sys.exit(scrapy.cmdline.execute())
  File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\cmdline.py", line 128, in execute
    cmds = _get_commands_dict(settings, inproject)
  File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\cmdline.py", line 46, in _get_commands_dict
    cmds = _get_commands_from_module('scrapy.commands', inproject)
  File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\cmdline.py", line 29, in _get_commands_from_module
    for cmd in _iter_command_classes(module):
  File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\cmdline.py", line 19, in _iter_command_classes
    for module in walk_modules(module_name):
  File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\utils\misc.py", line 71, in walk_modules
    submod = import_module(fullpath)
  File "C:\ProgramData\Anaconda3\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\commands\version.py", line 6, in <module>
    import OpenSSL
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\__init__.py", line 8, in <module>
    from OpenSSL import rand, crypto, SSL
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\rand.py", line 12, in <module>
    from OpenSSL._util import (
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "C:\ProgramData\Anaconda3\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py", line 12, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: DLL load failed: 作業系統無法執行 %1。
解決辦法:
 >>   pip3 uninstall pyopenssl
 >>  pip3 uninstall cryptography
 >>  pip3 install pyopenssl
  >>  pip3 install cryptography

   scrapy在Linux安裝非常順利,但是在Windows上卻問題多多,一波三折,個人建議,如果玩開發,還是推薦linux或者mac。