A python tool to scrape data from requested url using Scrapy.

• The tool scrapes data from defined url using only scrapy
• The tool can be called like this: scrapy crawl rwjstspider.
• The tool outputs a json and xlsx file.

• Clone repository with HTTPS

   https://github.com/priye-1/scrapy-data-mining.git

• Setup virtual environment with python venv

  python -m venv .venv

• Activate the Virtual environment

  {path to desired directory} source .venv/bin/activate

• Install requirements

  # use dev or production requirments depending on location
  pip install -r > requirements.txt
  

• Cd into Working directory

  cd pst_ag_project