Python中安装Scrapy时遇到错误如何解决？

32 位的 WINDOWS7，PYTHON3.6 环境下要安装 Scrapy-1.4.0-py2.py3-none-any.whl ，这个 whl 文件已经下载并放到 python 目录下，输入命令为 pip install Scrapy-1.4.0-py2.py3-none-any.whl
安装了很多包后，出现了以下的出错提示，请问这是什么原因导致的，如何解决呢？感谢指点！

Requirement already satisfied: pyparsing in d:\python\python3\lib\site-packages (from packaging>=16.8->setuptools->zope.interfac
e>=4.0.2->Twisted>=13.1.0->Scrapy==1.4.0)
Installing collected packages: Twisted, Scrapy
Running setup.py install for Twisted … error
Exception:
Traceback (most recent call last):
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\compat_init_.py”, line 73, in console_to_str
return s.decode(sys.stdout.encoding)
UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xd5 in position 2: invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\commands\install.py”, line 342, in run
prefix=options.prefix_path,
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\req\req_set.py”, line 784, in install
**kwargs
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\req\req_install.py”, line 878, in install
spinner=spinner,
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\utils_init_.py”, line 676, in call_subprocess
line = console_to_str(proc.stdout.readline())
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\compat_init_.py”, line 75, in console_to_str
return s.decode(‘utf_8’)
UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xd5 in position 2: invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\commands\install.py”, line 385, in run
requirement_set.cleanup_files()
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\req\req_set.py”, line 729, in cleanup_files
req.remove_temporary_source()
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\req\req_install.py”, line 977, in remove_temporary_source
rmtree(self.source_dir)
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\retrying.py”, line 49, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\retrying.py”, line 212, in call
raise attempt.get()
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\retrying.py”, line 247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\six.py”, line 686, in reraise
raise value
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\retrying.py”, line 200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\utils_init_.py”, line 102, in rmtree
onerror=rmtree_errorhandler)
File “d:\python\python3\lib\shutil.py”, line 494, in rmtree
return _rmtree_unsafe(path, onerror)
File “d:\python\python3\lib\shutil.py”, line 384, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File “d:\python\python3\lib\shutil.py”, line 384, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File “d:\python\python3\lib\shutil.py”, line 384, in _rmtree_unsafe
rmtree_unsafe(fullname, onerror)
[Previous line repeated 1 more times]
File “d:\python\python3\lib\shutil.py”, line 389, in rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\utils_init.py", line 114, in rmtree_errorhandler
func(path)
Permissi onError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: ‘c:\TEMP\pip-build-n_4iqc5q\Twisted\src\twisted\i
nternet\iocpreactor\iocpsupport\iocpsupport.c’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\basecommand.py”, line 215, in main
status = self.run(options, args)
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\commands\install.py”, line 385, in run
requirement_set.cleanup_files()
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\utils\build.py”, line 38, in exit
self.cleanup()
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\utils\build.py”, line 42, in cleanup
rmtree(self.name)
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\retrying.py”, line 49, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\retrying.py”, line 212, in call
raise attempt.get()
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\retrying.py”, line 247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\six.py”, line 686, in reraise
raise value
File “C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip_vendor\retrying.py”, line 200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\utils_init.py", line 102, in rmtree
onerror=rmtree_errorhandler)
File “d:\python\python3\lib\shutil.py”, line 494, in rmtree
return _rmtree_unsafe(path, onerror)
File “d:\python\python3\lib\shutil.py”, line 384, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File “d:\python\python3\lib\shutil.py”, line 384, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File “d:\python\python3\lib\shutil.py”, line 384, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
[Previous line repeated 2 more times]
File “d:\python\python3\lib\shutil.py”, line 389, in rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Users\YY\AppData\Roaming\Python\Python36\site-packages\pip\utils_init.py", line 114, in rmtree_errorhandler
func(path)
Permissi onError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'c:\TEMP\pip-build-n_4iqc5q\Twisted\src\twisted\i
nternet\iocpreactor\iocpsupport\iocpsupport.c’
Python中安装Scrapy时遇到错误如何解决？


11 回复

安装姿势错了~~ win 下不是这样安装的 。。。。 要么需要第三方集成包 要么换系统安装 scrapy


遇到Scrapy安装错误，先别慌，这玩意儿依赖比较多，最常见的就是缺少C编译环境或者Twisted库装不上。

核心解决思路就两步：1）搞定编译环境；2）用预编译的轮子（wheel）绕过编译。

具体操作：

  1. 先确保有Python和pip：

    python --version
pip --version

  2. 在Windows上，最容易出问题：

    • 首选方案：直接去 Unofficial Windows Binaries for Python Extension Packages 这个神站，下载 Twisted 和 Scrapy 对应的 .whl 文件。比如你用的是 Python 3.10，64位系统，就找 Twisted‑xxx‑cp310‑cp310‑win_amd64.whl 和 Scrapy‑xxx‑py3‑none‑any.whl。
    • 下载后，用 pip 本地安装：
      pip install 下载的/Twisted文件路径.whl
pip install 下载的/Scrapy文件路径.whl
    • 备选方案：安装微软的构建工具。去 Microsoft C++ Build Tools 下载安装“使用C++的桌面开发”工作负载。或者更简单，装个 Visual Studio Build Tools，勾选“C++ 生成工具”。装完重启终端再试 pip install scrapy。

  3. 在macOS/Linux上： 通常需要安装 openssl 和开发工具。用Homebrew（macOS）或apt/yum（Linux）装一下：

    • macOS:
      brew install openssl
export LDFLAGS="-L$(brew --prefix openssl)/lib"
export CPPFLAGS="-I$(brew --prefix openssl)/include"
pip install scrapy
    • Linux (如Ubuntu):
      sudo apt-get install python3-dev build-essential libssl-dev libffi-dev
pip install scrapy

  4. 通用终极大招： 如果上面都麻烦，或者网络不好，直接用 conda 安装，它帮你管理二进制依赖，省心：

    conda install -c conda-forge scrapy

一句话总结：Windows下直接找预编译的whl文件最省事，其他系统装好开发库基本就能搞定。

根据报错先安装各个依赖，最后安装 scrapy

我自己似乎是先单独安装 lxml 和 twisted 的 whl，装好以后再安装 scrapy 的 whl 装好的

lxml 我已经安装完毕了，twisted 我是从 https://pypi.python.org/pypi 上找的 Twisted-17.5.0.tar.bz2 ，解压后，用 pip install setup.py 来安装时提示：
Could not find a version that satisfies the requirement setup.py (from versions: )
No matching distribution found for setup.py

难道这个版本不适用于 32 位的 WINDOWS 7 么？ 那这样就找不到其他版本了，我是 INTEL 的芯片。

请问哪里可以下载第三方集成包呢？感谢！

还有用 32 位机器的的 Python 开发者 的 ahhh
（路过帮顶，不是很熟悉 Windows 的那套

要不还是在 http://www.lfd.uci.edu/~gohlke/pythonlibs/ 这个网页上找 whl 安装吧，tar.bz2 压缩包安装我也经常出问题

谢谢这个网址的 twisted 可以顺利安装了

您说的 ahhh 是集成包么？哪里可以下载呢？谢谢

啊哈哈哈 （逃

回到顶部