Skip to content

This repository houses an Energy Forecasting API that uses Machine Learning to predict daily and monthly energy consumption from historical data. It's designed as a practical demonstration of a Machine Learning Engineering workflow, from initial analysis to a deployable API packaged with Docker.

Notifications You must be signed in to change notification settings

labrijisaad/Monthly-Daily-Energy-Forecasting-Docker-API

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Monthly & Daily Energy Forecasting Docker API

This image, generated with DALL-E, depicts tradition and tomorrow dance under a future-woven sky.

📘 Introduction

This project is all about predicting energy use 🔌. It began with a simple CSV file 📁, and I worked through steps like analyzing the data (EDA), using machine learning for predictions 🤖, and finally creating a Dockerized API 🐳. This journey represents a common task for Machine Learning Engineers: transforming data analysis into a usable API. I developed two prediction models: one for the next day 📆 and another for the next 30 days 🗓️, both based on historical data.

🎯 Objectives

Here's what I aimed to achieve:

  • Understand and Prepare a raw CSV dataset 📁.
  • Build models to forecast energy use for the upcoming day and month 🔮.
  • Package these models into an API, making it straightforward for others to use, encapsulated in Docker 📦.
  • Allow using data with the Docker-contained API through Docker volumes.

Preparation Before Building the API 🛠️

Prior to constructing the API within the src folder, I put together and described my work in Jupyter notebooks 📓, found in the notebooks directory. These notebooks explain my steps:

🧜‍♂️ Development Workflow

Below is a Mermaid Diagram outlining the Development Workflow.

graph LR
    classDef processNode fill:#4F5D75,stroke:#EF8354,stroke-width:2px,color:#FFFFFF;

    B[EDA & Data Prep - Notebook]
    B --> C[Machine Learning - Notebook]
    C --> D[API Simulation - Notebook]
    D --> E[API Development]
    E -->|Dockerized| F[Docker Environment]
    E -->|Virtual Env/Conda| G[Virtual Environment/Conda]
    F --> H[Test and Use API]
    G --> H

    class A,B,C,D,E,F,G,H processNode;
Loading

🏗️ API Construction

After prototyping in the API - Energy_Forecasting Notebook, Here is how i designed the project structure:

Energy-Forecast-API/
│
├── docker/                                 # Docker configuration files
│   ├── Dockerfile                          # Instructions for building the Docker image
│   └── requirements.txt                    # API dependencies for the Docker image
│
├── src/                                    # Source code for the API
│   ├── __init__.py                         
│   ├── energy_forecasting.py               # Main script for energy forecasting logic
│   └── models/                             
│       ├── __init__.py                     
│       ├── base_energy_model.py            # Base class for energy models
│       ├── short_term_energy_model.py      # Short-term energy forecasting model
│       └── long_term_energy_model.py       # Long-term energy forecasting model
│
├── notebooks/                              
│   ├── EDA_Data_Preparation.ipynb          # Exploratory Data Analysis and Data Preparation
│   ├── ML_Experimentation.ipynb            # Machine Learning model experimentation
│   └── API_Design.ipynb                    # Development of class-based API in notebook form
│
├── data/                                   
│   ├── raw/                                # Unprocessed initial data
│   └── processed/                          # Processed data ready for analysis
│
├── results/                                # Outputs from analysis and model training notebooks
│
├── .env                                    # Environment variables for Docker and the application
├── Makefile                                # Make commands for easy project setup and management
├── requirements.txt                        # Project dependencies for development
├── .gitignore                              # Specifies intentionally untracked files to ignore
└── README.md                               # Comprehensive guide to setup and use the API

🚀 Run the Code

You're a few steps away from predicting energy usage! Choose one of the two setups based on your preference:

Option 1: Virtual Environment or Conda 🌿

If you like the Python environment approach, here's how to proceed:

  1. Environment Setup: Confirm Python is up and ready, and set up your virtual environment or Conda.

  2. Dependency Installation: In the project's main folder, run:

    pip install -r requirements.txt

    This installs all required libraries specified in requirements.txt.

  3. Environment Activation: With dependencies in place, activate your environment to proceed.

  4. Running the Application: Your environment is now primed for action.

    To get a short-term energy prediction, enter:

    make test-short-term

    You'll get a forecast like this:

    Short-term prediction for '2010-05-17': 1512.24
    

    For custom date predictions, use:

    make long-term DATE=2010-05-17

    And expect a response similar to:

    Long-term prediction for 2010-05-17: 1509.71
    

For more commands, type make help in the terminal.

Option 2: Docker 🐳

For Docker fans, make sure Docker is operational on your machine. You can containerize the app with ease 😉

  • Build Docker Image:

    make docker-build

    Construct your Docker image before running forecasts.

  • Run Short-Term Forecast in Docker:

    make test-docker-run

    See a prediction output like:

    Short-term prediction for 2010-05-17: 1513.30
    
  • Run Custom Forecast in Docker:

    make docker-run DATE=2010-05-17 MODEL=long

    Get a custom forecast, for example:

    Long-term prediction for 2010-05-17: 1514.91
    

⚠️ For further Docker commands, type make help in the terminal. If you wish to modify the command, including changing the docker volume path, you can do so in the Makefile. Just ensure you make these adjustments as needed.

🌐 Let's Connect!

You can connect with me on LinkedIn or check out my GitHub repositories:

About

This repository houses an Energy Forecasting API that uses Machine Learning to predict daily and monthly energy consumption from historical data. It's designed as a practical demonstration of a Machine Learning Engineering workflow, from initial analysis to a deployable API packaged with Docker.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published