Skip to content

A social media scraper written in Rust. Used for the data on my personal site.

Notifications You must be signed in to change notification settings

ryanmcgrath/activity-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 

Repository files navigation

Activity Scraper

My personal website makes use of data from my GitHub, Twitter, and Dribbble accounts. It's a static site that's republished every few minutes, at which point it scrapes the most recent activity from Twitter, various pieces of data from GitHub and Dribbble, and stores them in a file that the static engine picks up for rendering.

I figure this might be fun or useful for some other people. It's written in Rust, because... well, I enjoy writing in Rust. Nothing about this particularly requires Rust, so if you're not a fan, you may enjoy rewriting in a different language.

Running

You need to make sure the following environment variables are set - you may want to edit the source to change the RYMC_ prefix:

# Dribbble
export RYMC_DRIBBBLE_API_KEY=''

# Twitter
export RYMC_TWITTER_CONSUMER_KEY=""
export RYMC_TWITTER_CONSUMER_SECRET=""
export RYMC_TWITTER_OAUTH_TOKEN=""
export RYMC_TWITTER_OAUTH_SECRET=""

# GitHub
export RYMC_GITHUB_ACCESS_TOKEN=""

export RYMC_ACTIVITY_PATH="/path/to/where/to/store"

Then:

# Install Rust...
cargo build
./target/debug/social

License

Do what you want with it! I make no claims of support or anything on a project like this.

About

A social media scraper written in Rust. Used for the data on my personal site.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages