• Course : DRIO-4302C Data Engineering
• February 3, 2019
• Students : Vincent Barbosa Vaz, William Cardoso
• Teacher : Daniel Courivaud, Raphaël Courivaud

This project is for informational and educational purposes, do not use it for business purposes.

• IMDb scraping with Scrapy
• Flask application
• MongoDB database
• Docker
  • docker-compose.yml
  • Dockerfile
  • run mongodb inside Docker
  • run elasticsearch inside Docker

Crawling/scraping of IMDb for series, with Scrapy.

Save data into MongoDB database.

Create a Flask web-app to display the data.

The user likes series he loves (through Elasticsearch), the app match the bests series to watch.

Clone it :

git clone https://github.com/v-barbosavaz/DRIO4302C

cd DRIO4302C
docker-compose up -d

cd DRIO4302C
pipenv shell
pipenv run python run.py