[solved]: ImportError: No module named w3lib.http

ERROR:

Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 7, in
    from scrapy.cmdline import execute
  File "/Library/Python/2.7/site-packages/scrapy/__init__.py", line 48, in
    from scrapy.spiders import Spider
  File "/Library/Python/2.7/site-packages/scrapy/spiders/__init__.py", line 10, in
    from scrapy.http import Request
  File "/Library/Python/2.7/site-packages/scrapy/http/__init__.py", line 8, in
    from scrapy.http.headers import Headers
  File "/Library/Python/2.7/site-packages/scrapy/http/headers.py", line 2, in
    from w3lib.http import headers_dict_to_raw
ImportError: No module named w3lib.http

Solution:

If you have pip installed, on Mac & Linux:

Try:

pip install w3lib

if you have installed it with root before:

sudo pip install w3lib

Stack Trace:

$ pip install w3lib

Collecting w3lib

  Using cached https://files.pythonhosted.org/packages/37/94/40c93ad0cadac0f8cb729e1668823c71532fd4a7361b141aec535acb68e3/w3lib-1.19.0-py2.py3-none-any.whl

Requirement already satisfied: six>=1.4.1 in /Users/admin/anaconda3/lib/python3.6/site-packages (from w3lib) (1.12.0)

Installing collected packages: w3lib

Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/Users/admin/anaconda3/lib/python3.6/site-packages/w3lib'

Consider using the `--user` option or check the permissions.

Rahuls-MacBook-Pro:ZomatoData admin$

Rahuls-MacBook-Pro:ZomatoData admin$ sudo pip install w3lib

Password:

The directory '/Users/admin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

The directory '/Users/admin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Collecting w3lib

  Downloading https://files.pythonhosted.org/packages/37/94/40c93ad0cadac0f8cb729e1668823c71532fd4a7361b141aec535acb68e3/w3lib-1.19.0-py2.py3-none-any.whl

Requirement already satisfied: six>=1.4.1 in /Users/admin/anaconda3/lib/python3.6/site-packages (from w3lib) (1.12.0)

Installing collected packages: w3lib

Successfully installed w3lib-1.19.0