Skip to content

Backfill historical OHLC feature in a Feature Store (Hopsworks) using an orchestration tool (Prefect).

Notifications You must be signed in to change notification settings

Paulescu/backfill-feature-store-with-prefect

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Feature backfilling

Backfill a feature group using Prefect and Hopsworks

Let's connect 🤗
TwitterLinkedInNewsletter

Table of contents

  1. What is this repo about?
  2. How to run this code
  3. Wannna build real-world ML products?

What is this repo about?

This repository shows how to backfill historical OHLC feature in a Feature Store (Hopsworks) using an orchestration tool (Prefect).

How to run this code

  • Create a Python virtual environment with the project dependencies with

    $ make init
    
  • Connect to your Prefect Cloud

    $ prefect cloud login 
    
  • Set environment variables necessary to talk to your Hopsworks feature store

    $ . ./set_hopsworks_credentials.sh
    
  • Backfill OHLC data for a range of dates (e.g. from 2023-01-01 to 2023-01-31)

    $ make from_day=2023-01-01 to_day=2023-01-31 backfill
    

Wannna build real-world ML products?

Check the Real-World ML Program, a hands-on, 3-hour course where you will learn how to design, build, deploy, and monitor complete ML products.

About

Backfill historical OHLC feature in a Feature Store (Hopsworks) using an orchestration tool (Prefect).

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published