Skip to content

πŸ“’ Scrape multiple subreddits and store them in a JSON format

Notifications You must be signed in to change notification settings

viktorholk/subreddit-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

4 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Subreddit Scraper

πŸ“’ Scrape multiple subreddits and store them in a JSON format
πŸ”– Edit the posts with the built-in editor

How to use

Scraper

Clone the repository on your computer and open up a terminal. Execute the program with python app.py, this will scrape the subreddits in the list of the source code.
Feel free to change these subreddits to whatever.
You can also type python app.py [count] (example python app 15) - This will scrape 15 posts in each of the subreddits and store them in the json.

Editor

To edit all the files you just scraped type python app -e pr python app --editor to open the editor.
image of editor Here you can press SPACE to edit the content of the post, blacklist it with ESC which will remove it and prevent appending on other scrapes, U to open the url of the body (if it's a url) and Q to quit the editor.

About

πŸ“’ Scrape multiple subreddits and store them in a JSON format

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages