Build your end-to-end machine learning projects within one tool.

Getting Started • Documentation • Support • Contribution

Klops is an End-to-End machine learning development pipeline ops. It's built on top of Seldon Core, MLflow and DVC. The goal of this project is to make it easier for Data Scientists to develop, maintain, log their experiments and deploy their machine learning projects.

• Seldon Core Installed on your Kubernetes cluster. Guidance here
• MLflow Tracking Server deployed. Recommended to use the scenario 5 here
• Local or remote storage such as Amazon S3 or Google Cloud Storage (GCS) or similar.

Using pip is the best option if you want to get a stable version and an easier way to install.

$ pip install klops

This is the best way to get the latest published library, including the experimental version.

Clone this repo

$ git clone https://github.com/asrulsibaoel/klops.git

Change your directory to your klops folder:

$ cd klops

Install

$ python setup.py install

Klops consists of three modules. Versioning, Experiment and Deployment.

Klops Experiment is a class that wraps the MLflow Tracking. Below is a simple example to begin with.

from klops.experiment import Experiment
from sklearn.datasets import load_iris
from sklearn.naive_bayes import GaussianNB

experiment = Experiment(name="your-experiment-name", tracking_uri="https://<your-mlflow-host>:<port>")

X, y = load_iris(return_X_y=True)
HYPERPARAMETERS = {
    "n_estimators": 20,
    "max_depth": 10,
    "min_samples_split": 5
}

experiment.start(GaussianNB, x_train_data=X, y_train_data=y, tuner_args=HYPERPARAMETERS)

You would see your experiment result in your Mlflow Tracking UI like below:
For the complete tutorials could be seen through this documentation.

Klops Versioning is a kind of version control based on DVC. This module wrapped the command line and python APIs from DVC.

from klops.versioning import Versioning

versioning = Versioning()

# Track your file into dvc
versioning.add("myfile.csv")

# Add your DVC repository storage
versioning.add_remote("gs:https://your-bucket-name/your-path/")

# Push your changes to DVC
versioning.push()

Complete examples could be seen on this tutorial page.

Klops Deployment is a module to deploy the development machine learning projects into Seldon Core instance. Below is the example on how to done with.

from klops.deployment import Deployment
from klops.deployment.auth import GKEAuthentication

gke = GKEAuthentication(
    project_id="your-project-id",
    zone="your-project-region",
    cluster_id="your-cluster-id")
deployment = Deployment(gke, "seldon")

## Config file could be json or yaml/yml file.
config = deployment.load_deployment_configuration("<deployment-config>.json")
deployment.deploy(config)

Now you can access your API through this doc https://<ingress_url>/seldon/<namespace>/<model-name>/api/v1.0/doc/. Example result:

We are open for anyone who wants to contribute! Please read the CONTRIBUTING.md guide.