Skip to content
@project-codeflare

project-codeflare

Simplifying and automating scaling of distributed AI/ML
README.md

Project CodeFlare

About

Project CodeFlare provides a simple, user-friendly abstraction for developing, resource-scaling, queuing, and management of distributed AI/ML and Python workloads on OpenShift Container Platform.

Project CodeFlare consists of the following components:

CodeFlare SDK and CLI to define, develop, and control remote distributed compute jobs and infrastructure from either a python-based environment or command-line interface

Multi-Cluster Application Dispatcher (MCAD) for queueing, resource quotas, and management of batch jobs

Instascale for on-demand resource scaling of an OpenShift cluster

CodeFlare Operator for automating deployment and configuration of the Project CodeFlare stack

Project CodeFlare Ecosystem

In addition to running standalone, Project CodeFlare is deployed as part of and integrated with the Open Data Hub.

Getting Started

Learning

Watch this video for an introduction to Project CodeFlare and what the stack can do. (Nov. 2022)

See this video as well for an updated demonstration of the basic stack functionality in-action. (Jun. 2023)

Quick Start

To get started using the Project CodeFlare stack, try this end-to-end example!

For more basic walk-throughs and in-depth tutorials, see our demo notebooks!

Development

See more details in any of the component repos linked above, and go to their issues page for open tasks/issues!

Architecture

We attempt to document all architectural decisions in our ADR documents. Start here to understand the architectural details of Project CodeFlare.

Getting Involved

Join our Slack community to get involved or ask questions.

License

Unless otherwise noted at a per-component level, this Project CodeFlare is licensed under the Apache-2.0 License.

Pinned

  1. codeflare codeflare Public

    Simplifying the definition and execution, scaling and deployment of pipelines on the cloud.

    Jupyter Notebook 217 36

  2. multi-cluster-app-dispatcher multi-cluster-app-dispatcher Public

    Holistic job manager on Kubernetes

    Go 106 62

  3. codeflare-cli codeflare-cli Public

    TypeScript 12 12

  4. codeflare-sdk codeflare-sdk Public

    An intuitive, easy-to-use python interface for batch resource requesting, access, job submission, and observation. Simplifying the developer's life while enabling access to high-performance compute…

    Python 22 37

  5. codeflare-operator codeflare-operator Public

    Operator for installation and lifecycle management of CodeFlare distributed workload stack

    Go 7 36

  6. instascale instascale Public

    On-demand Kubernetes/OpenShift cluster scaling and aggregated resource provisioning

    Go 10 19

Repositories

Showing 10 of 30 repositories
  • codeflare-sdk Public

    An intuitive, easy-to-use python interface for batch resource requesting, access, job submission, and observation. Simplifying the developer's life while enabling access to high-performance compute resources, either in the cloud or on-prem.

    project-codeflare/codeflare-sdk’s past year of commit activity
    Python 22 Apache-2.0 37 61 33 Updated Jun 21, 2024
  • codeflare-operator Public

    Operator for installation and lifecycle management of CodeFlare distributed workload stack

    project-codeflare/codeflare-operator’s past year of commit activity
    Go 7 Apache-2.0 36 28 10 Updated Jun 21, 2024
  • appwrapper Public

    AppWrapper controller for Kueue

    project-codeflare/appwrapper’s past year of commit activity
    Go 4 Apache-2.0 4 11 1 Updated Jun 20, 2024
  • codeflare-common Public

    Common packages for use with CodeFlare Distributed Workload stack.

    project-codeflare/codeflare-common’s past year of commit activity
    Go 0 Apache-2.0 9 2 0 Updated Jun 18, 2024
  • integration-tests Public

    Integration tests verifying integration of all CodeFlare components as part of ODH.

    project-codeflare/integration-tests’s past year of commit activity
    Go 0 Apache-2.0 5 0 0 Updated Jun 11, 2024
  • notebooks Public Forked from opendatahub-io/notebooks

    Notebook images for ODH

    project-codeflare/notebooks’s past year of commit activity
    Python 0 Apache-2.0 45 0 2 Updated Jun 6, 2024
  • community-operators-prod Public Forked from redhat-openshift-ecosystem/community-operators-prod

    community-operators metadata backing OpenShift OperatorHub

    project-codeflare/community-operators-prod’s past year of commit activity
    Dockerfile 0 Apache-2.0 495 0 1 Updated Jun 6, 2024
  • mcad Public

    MCAD v2

    project-codeflare/mcad’s past year of commit activity
    Go 10 Apache-2.0 10 9 1 Updated May 7, 2024
  • mcad-dashboard Public Forked from opendatahub-io/odh-dashboard

    Dashboard for MCAD

    project-codeflare/mcad-dashboard’s past year of commit activity
    TypeScript 3 Apache-2.0 140 2 0 Updated Mar 25, 2024
  • instaslice Public

    InstaSlice facilitates the use of Dynamic Resource Allocation (DRA) on Kubernetes clusters for GPU sharing

    project-codeflare/instaslice’s past year of commit activity
    Go 11 0 0 0 Updated Mar 19, 2024
View all repositories

Most used topics

Loading…