Project created by Steven Jack Chung
It was based on a Capstone project for the Ruby Module in Microverse
- ⚒️ Scraper
- 📐 About The Project
- 📝 Usage Instructions
Built With
- 🖥️ Setup for starting the Scraper
- 💻 Run Scraper
- ⚗️ Test Code with Rspec
- 📶 Live Preview
- 👥 Author
- 🤝 Contributing
- ⭐ Show your support
Built WithOriginally, a scraper is a tool used to extract something on a surface, for example, a chewing gum from the floor. However, in this case we are using a Web Scraper. A Web Scraper or Web Crawler, is used to extract bits of information from a website to analize them.
I created this basic web scraper to get information from products in an e-commerce website. I used https://helmboots.com for this project.
The scraper has a basic interface that runs in the terminal.
The goals of this project is to show mastery in:
- OOP (Object Oriented Programming) using Ruby.
- Create and structure a Ruby project from previous knowledge.
- Managing the files with correct usage to make code maintainable.
- Git Flow and Github Flow creating branches for different features and for version control.
- Rspec for testing parts of the code.
Once you run the scraper on the terminal, wait for the scrape to finish and select what you want to do when it shows prompt for the menu in the terminal.
For example:
Added 50 products.
1. Show all Products
2. Products by Price
3. Exit
What would you like to do?
1
...
- Option 1. Will show a formatted list with all the products and prices as well as if the product is on sale or not.
- Option 2. Will ask you to input a minimum price and it will show results starting from the minimum price.
For example:
Please input your minimum price.
250
Product: ...
Price: $250
Sale: Yes
Product: ...
Price: $300
- After finishing showing the results it will prompt the menu once again so you can choose what you want to do.
- Option 3. Will exit the program.
1. Show all Products
2. Products by Price
3. Exit
What would you like to do?
3
Have a nice day! :D
Enjoy!!
To get a local copy up and running follow these simple example steps.
Computer running Mac OS, Windows, or Linux Ubuntu. Internet browser: Firefox, Google Chrome, Opera...
Have Ruby installed you can see this link for more detailed instructions Install Ruby
You MUST have Ruby installed in your system. If you don't have it installed you can go this page Install Ruby
After installing ruby, you can download the repo or clone it (paste on the terminal) git clone https://github.com/jcy2704/scraper.git
Open the terminal and install bundler with gem install bundler. After installing bundler, navigate to the folder containing the files you've cloned and enter bundle install to install the necessary gems to run the scraper.
You will need to execute the scraper file located inside the bin folder. If you are using Window, follow next step.
Open the terminal and navigate to the folder using cd path-to-folder containing the scraper files. Once you are inside, your terminal should look like this: User//scraper/. You can type or paste this on terminal bin/scraper.rb to run the program.
Most Mac and some of Linux computers come with ruby installed, but still need to verify by typing in your terminal ruby --version if not please go back to Installation section.
After verifying you have ruby installed and completing the Installation section, open the terminal navigate using cd path-to-folder containing the tic-tac-toe game and run in the terminal bin/scraper.rb to run the game.
To be able to test your code with Rspec, you should have Ruby and Rspec installed in your computer. To install rspec type in your terminal gem install rspec. After installing rspec, just got to the project folder and run rspec in the terminal.
Enjoy our project!!
👤 Steven Jack Chung
- GitHub: @jcy2704
- Twitter: @yiak_
- LinkedIn: Steven Jack Chung
- Gmail: Steven Jack Chung
Contributions, issues, and feature requests are welcome!
Feel free to check the issues page.
Give a ⭐️ if you like this project!