In the effort to set up a viable buildpacks workflow, this repo houses necessary manifests, code and values to deploy the actual build service itself.

There are components necessary to the healthy functioning of the service that will likely not be included in this repository, such as custom webhooks, etc.

See also https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_Buildpack_Implementation

You will need to install:

• docker -> as the container engine for everything.
• docker-compose -> for harbor (docker registry).
• minikube -> as dev kubernetes installation.

You can install it with the helper script:

• utils/get_harbor.sh After that you can use docker-compose to run the whole harbor system (see the output of the script for the exact command).

Once harbor is running, you will need to make sure that it's setup (user project created and such) so run:

• utils/setup_harbor.py

We will need to get a specific k8s version (1.20.11 is the current toolforge version when writing this, you might want to double check):

• minikube start --kubernetes-version=v1.20.11

If you want to do a full stack test, you'll need to deploy the buildpack admission controller too, for that follow the instructions here. NOTE: might be faster to build the buildpack admission controller image locally instead of pulling it.

If you want to check first what would be deployed, you can run:

• kubectl kustomize deploy/base-tekton | vim -
• kubectl kustomize deploy/devel | vim -

Deploying this system can be done with:

• kubectl apply -k deploy/base-tekton -> creates the CRDs and tekton related objects
• kubectl apply -k deploy/devel -> uses the CRDs defined above

• kubectl create examples/pipeline.yaml

At this point I recommend installing the tekton cli, that makes it easier to inspect (otherwise you have a bunch of json to parse).

Getting the taskruns:

• tkn -n image-build taskruns list

Showing the details for them:

• tkn -n image-build taskruns describe

Seeing the logs live of a specific run:

• tkn -n image-build taskruns logs -f minikube-user-buildpacks-pipelinerun-n8mbj-build-from-git-6r2hf

Of course, you can get all that info too with kubectl directly, though it's quite more terse (though might help debugging tricky issues):

• kubectl describe -n image-build taskruns.tekton.dev minikube-user-buildpacks-pipelinerun-n8mbj-build-from-git-6r2hf

If you want to remove everything you did to start from scratch, you can just:

• minikube delete

NOTE: this will delete the whole k8s cluster, if you are playing with other things in that same cluster, you might want want to delete each namespace/resource one by one instead.

This will not remove the volumes on harbor side, to do so you'll have to stop harbor:

• docker-compose -f .harbor/harbor/docker-compose.yml down -v --remove-orphans

And you'll need to delete the data directory (sudo is needed due to files created/modified inside the containers):

• sudo rm -rf .harbor/harbor/data

NOTE: For production, TBD