Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(crd): AmaltheaSession definition #560

Merged
merged 5 commits into from
Jun 25, 2024
Merged

Conversation

sgaist
Copy link
Collaborator

@sgaist sgaist commented Jun 17, 2024

Describe your changes

This PR implements the new AmaltheaSession CRD.

The goal of the this new session is to give user the option to use other development environments than just JupyterLab.

New Docker images as well as instructions will be created as well as part of the implementation.

Issue ticket number and link

Part of SwissDataScienceCenter/renku#3679

@sgaist sgaist requested review from ableuler, olevski and a team as code owners June 17, 2024 07:38
Copy link
Member

@olevski olevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for starting this Samuel. This is a great starting point.

A few things we will have to add in subsequent PRs:

  • port on which the session will run
  • information about ingress (i.e. tls secret and host name)

controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
Comment on lines +205 to +223
secret:
type: string
description: Name of the secret containing the credentials for the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
secret:
type: string
description: Name of the secret containing the credentials for the
secret:
type: string
description: Name of the Kubernetes secret located in the same namespace as the session, containing the TOML Rclone configuration for the remote that stores the data to be mounted. The `remote` field above should match the `remote` name in the Rclone configuration. Please note that the secret containing this configuration will be "adopted" as a child resource of the session and therefore will be deleted when the session is deleted.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this necessarily rclone-specific? If so we should clearly specify somewhere that dataSources are only meant to work with the rclone-csi driver

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how else we can generalize this. If we have no constraints on this secret at all it becomes unusable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The alternative is to ask for a list of pvc names and paths where they should be mounted in the session. Then we do not have to ask for a secret and it is the job of the user of amalthea to create the PVCs whichever way they see fit.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since a data source might be different things, what about having an rclone object a bit like the git object from the repositories above ?

That way, it makes it easier to add support for other data sources.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sgaist this makes sense. I will add this when I convert the CRD spec to the go framework we will use. You dont have to add it now just so we change it later.

controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
Copy link
Member

@rokroskar rokroskar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sgaist for kicking this off! Just a few minor suggestions from my side.

controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
controller/crds/amaltheasession.yaml Outdated Show resolved Hide resolved
Comment on lines +205 to +223
secret:
type: string
description: Name of the secret containing the credentials for the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this necessarily rclone-specific? If so we should clearly specify somewhere that dataSources are only meant to work with the rclone-csi driver

sgaist and others added 5 commits June 20, 2024 16:27
This is a first draft for the new AmaltheaSession CRD.

The goal is to have a definition that is complete enough yet
provides room for some customization.
Co-authored-by: Rok Roškar <[email protected]>
Co-authored-by: Tasko Olevski <[email protected]>
This will allow to user either a token or OIDC to authenticate a user.
This will also allow to request GPU in a standard fashion.
@sgaist
Copy link
Collaborator Author

sgaist commented Jun 20, 2024

Sorry guys, I just realized that I rebased on top of main out of habit and forgot that it nukes GitHub's conversations.

@Panaetius
Copy link
Member

do we need a targetNamespace parameter? Current sessions can be scheduled in a namespace other than where amalthea runs.
Also for the cross cluster stuff we're investigating a targetCluster but that can be added later.

@sgaist
Copy link
Collaborator Author

sgaist commented Jun 21, 2024

do we need a targetNamespace parameter? Current sessions can be scheduled in a namespace other than where amalthea runs. Also for the cross cluster stuff we're investigating a targetCluster but that can be added later.

I haven't followed the cross cluster design/implementation but I would say yes, let's keep that for later as it will be highly dependent on how the inter-cluster communication/sync is implemented.

@olevski olevski self-requested a review June 21, 2024 12:25
@olevski olevski requested a review from rokroskar June 25, 2024 11:46
@olevski olevski dismissed rokroskar’s stale review June 25, 2024 11:46

Changes have been addressed

@olevski olevski merged commit a89bad7 into main Jun 25, 2024
18 checks passed
@olevski olevski deleted the feature/amaltheasession branch June 25, 2024 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants