4-store-mongodb.md

💽 Step 4 - Store the Products

How to avoid to scrape again and again the same data

Table of Contents

• 🎯 Objective
• 🏗 Prerequisites
• 👩‍💻 Just tell me what to do
• 📦 Suggested node modules
• 🛣️ Related Theme and courses

🎯 Objective

Store products in a database with node.js to create, read, update or delete data...

🏗 Prerequisites

1. Be sure to have a clean working copy.

This means that you should not have any uncommitted local changes.

❯ cd /path/to/workspace/clear-fashion
❯ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

2. Pull the master branch to update your local with the new remote changes

❯ git remote add upstream [email protected]:92bondstreet/clear-fashion.git
## or ❯ git remote add upstream https://github.com/92bondstreet/clear-fashion
❯ git fetch upstream
❯ git pull upstream master

👩‍💻 Just tell me what to do

1. Create a free account on MongoDB Atlas, Database as a Service (DBaaS) Provider.

2. Create a MongoDB Cluster

3. Connect your node.js server script

const {MongoClient} = require('mongodb');
const MONGODB_URI = 'mongodb+srv:https://<user>:<password>@<cluster-url>?retryWrites=true&writeConcern=majority';
const MONGODB_DB_NAME = 'clearfashion';
...

const client = await MongoClient.connect(MONGODB_URI, {'useNewUrlParser': true});
const db =  client.db(MONGODB_DB_NAME)

...

4. Insert the Products into this database

  const products = [];

  ...

  const collection = db.collection('products');
  const result = collection.insertMany(products);

  console.log(result);

5. Create at least 3 methods to find products according query.

These 5 methods should

• Find all products related to a given brands
• Find all products less than a price
• Find all products sorted by price
• Find all products sorted by date
• Find all products scraped less than 2 weeks

  const brand = 'loom';
  ...

  const collection = db.collection('products');
  const products = await collection.find({brand}).toArray();;

  console.log(products);

6. Commit your modification

❯ cd /path/to/workspace/clear-fashion
❯ git add -A && git commit -m "feat(new-products): insert all products"

(why following a commit message convention?)

7. Commit early, commit often
8. Don't forget to push before the end of the workshop

❯ git push origin master

Note: if you catch an error about authentication, add your ssh to your github profile.

If you need some helps on git commands, read git - the simple guide

📦 Suggested node modules

• dotenv - Loads environment variables from .env for nodejs projects
• mongodb - Mongo DB Native NodeJS Driver

🛣️ Related Theme and courses

• 📡 Theme 3 - Data is the new Gold