Playground for scrapy (scraping and web crawling framework) using Jupyter notebooks

• Python 3.7
• Scrapy 1.6.0
• beautifulsoup4 4.7.1
• lxml 4.3.4
• twisted 19.2.0
• pyOpenSSL 19.0.0

Using conda

All packages can be installed via conda/ANACONDA NAVIGATOR through conda-forge. You can also use conda with the provided environment which is the preferred way.

Using pip

You can also use pip to install scrapy, twisted, lxml and pyOpenSSL

pip3 install Twisted==14.0 lxml pyOpenSSL Scrapy

Use this conda environment for easy installation and testing.