Skip to content

A Web Scraping program to scrap Laptop prices and their specifications

Notifications You must be signed in to change notification settings

VipinindKumar/Web-scraping-price

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Web-scraping-price

Running the program:

python3 scraper.py

Dependencies:

  • amazon.py
  • flipkart.py
  • newegg.py

Outputs:

  • amazon-laptops.csv
  • flipkart-laptops.csv
  • newegg-laptops.csv

ToDo:

  • Factorise the code to use in multiple sites and urls

  • Find more sites to scrap the data

  • Create scraper functions for all these sites

  • Use Pandas to manipulate the data

  • Extract useful information from the data

  • Automate the script to run periodically

  • make ratings a comparable float

  • fix when no discount, current price is differently formed

  • Use title to extract their specs

  • Sorting the data accordingly to select best option available

  • Write script for Flipkart

  • Find other good sites available to buy new/old laptops

  • put urlopen in try/except or while loop

  • add link to the product in csv

  • use next page as a url too, to extract information

  • Report the best options from sorted data

  • Automate to run everyday/periodically

  • Add Refurbished and renewed labels

  • get the last page number from the page and create list of urls to scrape

  • stop newegg after certain page number

  • stop flipkart after certain page number

  • robot check problem

About

A Web Scraping program to scrap Laptop prices and their specifications

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages