A simple solution made to get wheather information from Australian cities in the Australian Government WebSite. using a basic WebScraper and provide the information through an REST API to a web interface.

This project was made as an example to show how to build a simple WebScraper, showing how to struct data on the solution and how to provide a service with the information that we get.

This project was made only to share code skills and teaching about programming bots, there is no lucrative porpose on this project and can not be used to any business interests.

• Build in Visual Studio 2019
• .NET Core 5 (C#)
• Angular 6
• Angular Material
• Nunit (for unit tests)
• AWS RDS (Relational Data Service)

• Create solution and base crawler
• Add a API layer provide the information as services
• Add a Web Interface and consume our APIs
• Build a global notification and alerts (Credits to @Cristux for the amazing template)
• Clean unnecessary code
• Improve interface resources (implemented Angular Material SDK)
• Basic search on information table by city names
• Weather Filter by Territory
• Create a event log for the API and Erros
  • Created a CustomLog service to log events in Console and updating a file.log on local machine
• Add a Unit Test Layer
  • API Layer
  • CustomLog Layer
  • WebCrawler Layer
  • Implement Git Action to auto test on commits
• Add Data Exportation feature for angular frontend (Credits to @Vikash5529 for the example)
• Better Home Page display
• Generate Dashborads and Reports

• Create a RDS (DataBase) to store historical data
• Host API Layer
• Create Git Action to deploy API
• Host Web frontend
• Create Git Action to deploy FrontEnd

• Hosting APIs on AWS

• WebScraper

The API service was configured to run in your localhost whitout the need to be published in some virtual directory, just build the API project in the solution then go to compiled files and run the .exe to start up the API local server.

...WeatherChecker\WeatherChecker.API\bin\Debug\net5.0\WeatherChecker.API.exe

After this you can change the base URL at the file main.ts

export function getBaseUrl() {
  //return document.getElementsByTagName('base')[0].href;
  return "https://localhost:5000/";
}