Skip to content

Web scraping tool built on Ruby. This web scraper pulls data/information on articles written by students from the Medium website, helping alumni to quickly get access to useful articles for their convenience.

License

Notifications You must be signed in to change notification settings

ClaytonSiby/Web_Scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Microverse Capstone Project: Web_Scraper

This web scraper is built using Ruby Programming language. It is a capstone project that is part of the Microverse curriculum, developed to showcase the author's ability to develop a project based on Ruby & any given business logic.

Project Design:

Project output screenshot.

screenshot

This project is a Web Scraper that takes information from Medium Website. It scrapes some articles written by Microverse students & any other articles that have anything to do with Microverse in generall. The scraper takes the information retrieved from the website & uses Nokogiri gem to store everything as an object. I then took this object & retrieve each article information and display it in a way that's more readable & organised. The articles are displayed in discending order based on the number of claps ( which in this case is the popularity that you see when you run the code. )

Built With

  • Ruby
  • Rspec (testing)
  • HTTParty
  • Nokogiri
  • Rubocop
  • Colorize
  • Pry (for debugging)

SetUp

To start contributing to this project run:
git clone https://github.com/ClaytonSiby/Web_Scraper.git

Now create a feature branch & open a pull request based on development (develop) branch.

After cloning the project, you should cd into the (web_scraper) directory where the code is stored & nevigate into bin older using cd bin command on the terminal, then run: bundle install to install the necessary dependencies & lastly run: ruby main.rb to see the scraper in action.

To Run Rspec:

Open your terminal & run rspec & the program will run tests for two files (scraper.rb & structure_data.rb).

Tools Used:

  • Visual Studio Code

Author

👤 Clayton Siby

🤝 Contributing

Contributions, issues and feature requests are welcome!

Feel free to check the issues page.

Please show your support

Give a ⭐️ if you like this project!

Acknowledgments

  • Microverse.org
  • StackOverflow
  • tutorialspoint.com

📝 License

This project is MIT licensed.

About

Web scraping tool built on Ruby. This web scraper pulls data/information on articles written by students from the Medium website, helping alumni to quickly get access to useful articles for their convenience.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages