This repo is an experiment to create a simple Kubernetes (K8S) Operator that can be used to create Jupyter Notebooks within a K8S cluster.
What this repo is trying to do is pull out only what is essential for deploying ONLY the Jupyter operator to the cluster.
NOTE: this work is preliminary and far from fully functional.
Minukube with these addons enabled:
- default-storageclass
- ingress
- ingress-dns
- storage-provisioner
Python 3.9.1 with kopf and kubernetes installed with Pip. I use a Conda environment.
$ kubectl create -f crd.yaml
For testing the operator is run manually from Python. This needs the INGRESS_DOMAIN environment variable to be set to the base domain name for the K8S cluster:
export INGRESS_DOMAIN=192.168.49.2.nip.io
$ kopf run handlers.py
[2021-01-20 16:31:09,827] kopf.reactor.activit [INFO ] Initial authentication has been initiated.
[2021-01-20 16:31:09,830] kopf.activities.auth [INFO ] Activity 'login_via_pykube' succeeded.
[2021-01-20 16:31:09,834] kopf.activities.auth [INFO ] Activity 'login_via_client' succeeded.
[2021-01-20 16:31:09,834] kopf.reactor.activit [INFO ] Initial authentication has finished.
[2021-01-20 16:31:09,844] kopf.engines.peering [WARNING ] Default peering object not found, falling back to the standalone mode.
$ kubectl create -f notebook-2.yaml
jupyternotebook.jupyter-on-kubernetes.test/notebook created
This deploys a very basic notebook environment. notebook-3 and notebook-4 illustrate other options.
In the kopf logs you see this:
[2021-01-20 16:12:42,393] kopf.objects [INFO ] [default/notebook] Handler 'jupyter' succeeded.
[2021-01-20 16:12:42,393] kopf.objects [INFO ] [default/notebook] Creation event is processed: 1 succeeded; 0 failed.
Check what has been created:
$ kubectl get jupyternotebooks.squonk.it
NAME URL PASSWORD
notebook http:https://notebook-default.192.168.49.2.nip.io/?token=CWM8PsfayLpd1qFj CWM8PsfayLpd1qFj
See what the Operator has created:
$ kubectl get all,ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME READY STATUS RESTARTS AGE
pod/notebook-675b679485-gnvqv 1/1 Running 0 3m4s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/notebook ClusterIP 10.102.82.29 <none> 8888/TCP 3m4s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/notebook 1/1 1 1 3m4s
NAME DESIRED CURRENT READY AGE
replicaset.apps/notebook-675b679485 1 1 1 3m4s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.extensions/notebook <none> notebook-default.192.168.49.2.nip.io 192.168.49.2 80 3m4s
Access the notebook at the URL listed (including the token). It works!
$ kubectl delete jupyternotebooks.squonk.it/notebook
jupyternotebook.squonk.it "notebook" deleted
$ kubectl delete crd/jupyternotebooks.squonk.it
customresourcedefinition.apiextensions.k8s.io "jupyternotebooks.squonk.it" deleted
The container image was created like this: Build the container image
docker build -t tdudgeon/jupyter-operator .
Push the container image
docker push tdudgeon/jupyter-operator .
Note that this deployment is very basic and assumes you are working in the default namespace.
The RBAC settings (the rbac.yaml file), are preliminary. They appear to work but may not cover all options and may be more permissive than are needed. To apply them run:
kubectl create -f rbac.yaml
Prior to getting these to work a service account with all privileges granted was created like this:
kubectl create sa jupyternotebooks-account
kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=default:jupyternotebooks-account
This should now not be necessary as using RBAC will be more secure.
Now create the deployment of the operator:
kubectl create -f deployment.yaml
Find the pod and look the logs for any errors. You should see something like this:
/usr/local/lib/python3.9/site-packages/kopf/reactor/running.py:168: FutureWarning: Absence of either namespaces or cluster-wide flag will become an error soon. For now, switching to the cluster-wide mode for backward compatibility.
warnings.warn("Absence of either namespaces or cluster-wide flag will become an error soon."
[2021-03-29 11:45:53,650] kopf.reactor.activit [INFO ] Initial authentication has been initiated.
[2021-03-29 11:45:53,651] kopf.activities.auth [DEBUG ] Activity 'login_via_client' is invoked.
[2021-03-29 11:45:53,652] kopf.activities.auth [DEBUG ] Client is configured in cluster with service account.
[2021-03-29 11:45:53,653] kopf.activities.auth [INFO ] Activity 'login_via_client' succeeded.
[2021-03-29 11:45:53,653] kopf.reactor.activit [INFO ] Initial authentication has finished.
[2021-03-29 11:45:54,057] kopf.clients.watchin [DEBUG ] Starting the watch-stream for customresourcedefinitions.v1.apiextensions.k8s.io cluster-wide.
[2021-03-29 11:45:54,082] kopf.clients.watchin [DEBUG ] Starting the watch-stream for jupyternotebooks.v1alpha1.squonk.it cluster-wide.
Now you can create a notebook like this:
kubectl create -f notebook-2.yaml
Lots of remaining work. In the immediate term:
- Review the CRD definintion which was hacked together from what the workshop deployment provided. (#2)
- Review the RBAC that is need for the ServiceAccount when running the operator container. (#4)