Add examples for (un)managed kernels and ML workers

docs/giskard_hub/installation_hub/mlworker/managed.ipynb

@@ -0,0 +1,202 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "4e5af785-ce9e-42cb-9e9a-e0874f85bd30",
+ "metadata": {},
+ "source": [
+ "# Managed ML worker\n",
+ "\n",
+ "In this notebook, you’ll learn how to create a project on Giskard hub and use a managed ML worker based on your current Python environment."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "148dbb23-6550-4bc1-b731-2daa6176e648",
+ "metadata": {},
+ "source": [
+ "## Install Giskard and run Giskard hub\n",
+ "\n",
+ "Make sure that you have installed giskard with hub feature"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "20e7ba3e-94eb-4282-a283-4ba9dd1f3839",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%pip install giskard[hub] --upgrade"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b599ef0c-47c7-4fdc-aefa-e7eaf936a02e",
+ "metadata": {},
+ "source": [
+ "If you have not yet run Giskard hub, run it on your local environment"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3be01d32-7864-4f18-b5be-a0641bf432cd",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!giskard hub start"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e8e6583b-c7b8-4d60-b755-5bc35450ea32",
+ "metadata": {},
+ "source": [
+ "## [Option 1] Create a project and a kernel for managed ML worker on Giskard Hub\n",
+ "\n",
+ "When you create a project and upload the models, datasets for it, you can let Giskard automatically create a Python kernel for managed ML worker. This kernel can describe your current Python environment, such as Python version, dependencies and their versions.\n",
+ "\n",
+ "This description can enable an execution environment, managed by your Giskard hub. Your model will be run there.\n",
+ "\n",
+ "To use this simplest option, you can create a project as usual:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f67dab32-9732-4329-9fa0-cbabca781089",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create a Giskard client after having install the Giskard server (see documentation)\n",
+ "api_key = \"<Giskard API key>\" #This can be found in the Settings tab of the Giskard hub\n",
+ "#hf_token = \"<Your Giskard Space token>\" #If the Giskard Hub is installed on HF Space, this can be found on the Settings tab of the Giskard Hub\n",
+ "\n",
+ "client = GiskardClient(\n",
+ " url=\"https://localhost:19000\", # Option 1: Use URL of your local Giskard instance.\n",
+ " # url=\"<URL of your Giskard hub Space>\", # Option 2: Use URL of your remote HuggingFace space.\n",
+ " key=api_key,\n",
+ " # hf_token=hf_token # Use this token to access a private HF space.\n",
+ ")\n",
+ "\n",
+ "project_key = \"my_project\"\n",
+ "my_project = client.create_project(project_key, \"PROJECT_NAME\", \"DESCRIPTION\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "320293df-6610-4b90-940f-4cb50a96fd3c",
+ "metadata": {},
+ "source": [
+ "The kernel will be created and the Python environment will be ready. Everything will be done automatically.\n",
+ "\n",
+ "You can check the Giskard hub to see more information. The created kernel should have a name with the project key of your project as a prefix, such as `my_project_kernel` or `my_project_kernel_<id>`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0d66deb8-8bdd-484d-a9b1-c18f437a580d",
+ "metadata": {},
+ "source": [
+ "## [Option 2] Create a project with an existing kernel on Giskard Hub\n",
+ "\n",
+ "You can also indicate the name of an existing kernel, either for a managed ML worker or an unmanaged ML worker.\n",
+ "\n",
+ "**Attention! In this case, you are in charge of the compatibility of the environments, including Python versions and dependencies, etc.**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "6a86c053-70a7-46ca-aae4-61a4f456c7f3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create a Giskard client after having install the Giskard server (see documentation)\n",
+ "api_key = \"<Giskard API key>\" #This can be found in the Settings tab of the Giskard hub\n",
+ "#hf_token = \"<Your Giskard Space token>\" #If the Giskard Hub is installed on HF Space, this can be found on the Settings tab of the Giskard Hub\n",
+ "\n",
+ "client = GiskardClient(\n",
+ " url=\"https://localhost:19000\", # Option 1: Use URL of your local Giskard instance.\n",
+ " # url=\"<URL of your Giskard hub Space>\", # Option 2: Use URL of your remote HuggingFace space.\n",
+ " key=api_key,\n",
+ " # hf_token=hf_token # Use this token to access a private HF space.\n",
+ ")\n",
+ "\n",
+ "project_key = \"my_project\"\n",
+ "kernel_name = \"<Python Kernel name>\"\n",
+ "my_project = client.create_project(project_key, \"PROJECT_NAME\", \"DESCRIPTION\", kernel_name)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9df6d970-78b2-4475-abfd-a69ae3f9a29b",
+ "metadata": {},
+ "source": [
+ "## Start a managed ML worker on Giskard hub\n",
+ "\n",
+ "You can start or stop a managed ML worker by accessing Giskard hub through a browser. Check our documentations to do so.\n",
+ "\n",
+ "You can start a managed ML worker from the cli as well, using the following command:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "9db52129-9052-4056-bc90-e6a32ac16ebe",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!giskard worker start -s -k YOUR_KEY -u https://<your IP address>:19000/ --name <your kernel name>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4e318a0b-e529-44c7-bd6c-4d6c75a5fcf8",
+ "metadata": {},
+ "source": [
+ "You can stop a managed ML worker from the cli using the following command:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "480597da-cbdc-4d4a-b6cf-751c03e385ee",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!giskard worker stop -s -k YOUR_KEY -u https://<your IP address>:19000/ --name <your kernel name>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cfde8f40-bd38-4dcb-a0ae-4da14ba0fadd",
+ "metadata": {},
+ "source": [
+ "For more advanced usages, please check our API references."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/giskard_hub/installation_hub/mlworker/unmanaged.ipynb

@@ -0,0 +1,152 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "4e5af785-ce9e-42cb-9e9a-e0874f85bd30",
+ "metadata": {},
+ "source": [
+ "# Unmanaged ML worker\n",
+ "\n",
+ "In this notebook, you’ll learn how to create a project on Giskard hub and use an unmanaged ML worker with your current Python environment."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "148dbb23-6550-4bc1-b731-2daa6176e648",
+ "metadata": {},
+ "source": [
+ "## Install Giskard and run Giskard hub\n",
+ "\n",
+ "Make sure that you have installed giskard with hub feature"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "20e7ba3e-94eb-4282-a283-4ba9dd1f3839",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%pip install giskard[hub] --upgrade"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b599ef0c-47c7-4fdc-aefa-e7eaf936a02e",
+ "metadata": {},
+ "source": [
+ "If you have not yet run Giskard hub, run it on your local environment"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3be01d32-7864-4f18-b5be-a0641bf432cd",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!giskard hub start"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e8e6583b-c7b8-4d60-b755-5bc35450ea32",
+ "metadata": {},
+ "source": [
+ "## Connect your ML worker to Giskard Hub\n",
+ "\n",
+ "When you want to create a project to upload the models, datasets for it, you will need a description of the Python environment needed by the models and datasets. Such description is called \"Python kernel\". This kernel can describe your current Python environment, such as Python version, dependencies and their versions.\n",
+ "\n",
+ "The easiest way to create an unmanaged kernel, is to connect an ML worker to Giskard hub, with an unique name. You can simply execute the following command to do so:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f67dab32-9732-4329-9fa0-cbabca781089",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!giskard worker start -k YOUR_KEY -u https://<your IP address>:19000/ --name <your kernel name>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "320293df-6610-4b90-940f-4cb50a96fd3c",
+ "metadata": {},
+ "source": [
+ "The kernel will be created automatically. You can check the Giskard hub to see more information."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0d66deb8-8bdd-484d-a9b1-c18f437a580d",
+ "metadata": {},
+ "source": [
+ "## Create a project with an existing kernel on Giskard Hub\n",
+ "\n",
+ "You can then create the project using the kernel name:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "6a86c053-70a7-46ca-aae4-61a4f456c7f3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create a Giskard client after having install the Giskard server (see documentation)\n",
+ "api_key = \"<Giskard API key>\" #This can be found in the Settings tab of the Giskard hub\n",
+ "#hf_token = \"<Your Giskard Space token>\" #If the Giskard Hub is installed on HF Space, this can be found on the Settings tab of the Giskard Hub\n",
+ "\n",
+ "client = GiskardClient(\n",
+ " url=\"https://localhost:19000\", # Option 1: Use URL of your local Giskard instance.\n",
+ " # url=\"<URL of your Giskard hub Space>\", # Option 2: Use URL of your remote HuggingFace space.\n",
+ " key=api_key,\n",
+ " # hf_token=hf_token # Use this token to access a private HF space.\n",
+ ")\n",
+ "\n",
+ "project_key = \"my_project\"\n",
+ "kernel_name = \"<Python Kernel name>\"\n",
+ "my_project = client.create_project(project_key, \"PROJECT_NAME\", \"DESCRIPTION\", kernel_name)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1bd46e67-9114-4e73-9b2b-9e60ebe75c2a",
+ "metadata": {},
+ "source": [
+ "Your worker will be associated with your project. Any executions of your models in this project will happen in your connected ML worker."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cfde8f40-bd38-4dcb-a0ae-4da14ba0fadd",
+ "metadata": {},
+ "source": [
+ "For more advanced usages, please check our API references."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}