Skip to content

sribalakumar/justdialscrapper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

#Python Scraper and D3 visualization using Play 1.2.5

This is a Play 1.2.5 app and the scraper written in python resides in app/controllers/JustDialScrap.py

TO RUN SCRAPER

repohome#>cd app/controllers/

repohome#>python JustDialScrap.py

City: User Input I

Category: User Input II

Example:

repohome#>python JustDialScrap.py

city: chennai

Category: cabs

Above commands runs the scraper and puts the scraped data to Chennai-Cabs.csv file and also generates Chennai-Cabs.json file which has the json data from JustDials top 5 pages(max) verified vs nonverified count for the city Chennai and category cabs.

TO RUN PLAY

repohome#> path to play 1.2.5/play run

which runs the webapp with default play dev settings on port 9000.

TO VIEW THE CHART open broswer url: localhost:9000

Json files generated by scraper is read by Play controller(app/controllers/Application.java) and then passed via Groovy templating to D3 chart scripts which is inside /app/views/Application/index.html. The play app uses D3 Pie-Chart to visualize the verified vs nonverified count.