In this session we are going to Introduce Cluster API, a Kubernetes subproject that allows you to manage Kubernetes clusters lifecycle running anywhere using only Kubernetes YAML files. Let’s see how Azure Arc GitOps approach improves and simplify the day-2 operations of these clusters, where your Git repo is now the source of truth. Do you have problems managing identities and Network connection for your current CI/CD process? You don’t know how to manage multiple Kubernetes clusters in production? Then this talk/repo is for you!
Slide Deck: https://www.slideshare.net/JorgeArteiro/manage-your-kubernetes-cluster-with-cluster-api-azure-and-git-ops
Meetup reference: https://www.meetup.com/en-AU/Microsoft-Reactor-Sydney/events/279879195
Follow us at https://youtube.com/AzureTar , https://AzureTar.com and @AzureTar
(Dependencies) - All environment/installation scripts required.
(Management Cluster) - Cluster API management/control plane cluster creation, configuration and operations.
(Workload cluster) - Workload Clusters creation, configuration and operations using CAPIZ(Azure Provider https://capz.sigs.k8s.io/).
(General) - Assorted scripts and commands.
(Reference Links) - Useful links to go deeper on Kubernetes Cluster API
curl -L https://aka.ms/InstallAzureCli | bash
curl -L https://github.com/kubernetes-sigs/cluster-api/releases/download/v0.4.4/clusterctl-linux-amd64 -o clusterctl
clusterctl version
chmod +x ./clusterctl
sudo mv ./clusterctl /usr/local/bin/clusterctl
az aks install-cli
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
az extension list -o table
az upgrade (to upgrade all installed extensions)
az extension add -n connectedk8s or az extension update -n connectedk8s
az extension add -n k8s-configuration or az extension update -n k8s-configuration
az extension add -n aks-preview or az extension update -n aks-preview
Create Azure resource Group on eastus regions where GitOps preview is available
az group create -l eastus -n capi-controlplane
Create Azure Kubernetes Services (Edit Script with your IDs)
az aks create --resource-group capi-controlplane --name capi-controlplane \
--node-count 1 --node-vm-size Standard_DS2_v2 \
--network-plugin azure --network-policy calico \
--enable-addons monitoring,azure-policy \
--enable-managed-identity --generate-ssh-keys \
--vm-set-type VirtualMachineScaleSets --zones 1 2 3 --load-balancer-sku standard \
--enable-aad --aad-admin-group-object-ids "<AdminGroupObjectId>" \
--max-pods 110 \
--yes
(Management Cluster) Get AKS Management Cluster .kubeconfig Credential. Config will be merged on the ~/.kube/config file
az aks get-credentials --resource-group capi-controlplane --name capi-controlplane
kubectl get nodes (to test connection)
az feature register --namespace Microsoft.ContainerService --name AKS-GitOps
az provider register --namespace Microsoft.ContainerService
az provider register --namespace Microsoft.KubernetesConfiguration
az feature show --namespace Microsoft.ContainerService --name AKS-GitOps (make sure it's Registered)
az aks enable-addons -a gitops -n capi-controlplane -g capi-controlplane
(Management Cluster) Add GitOps Configuration to deploy workload cluster from YAML files, --git-path=clusters
az k8s-configuration create \
--name capi-controlplane --cluster-name capi-controlplane --resource-group capi-controlplane \
--operator-instance-name capi-controlplane --operator-namespace default \
--repository-url https://github.com/azuretar/clusterapi-gitops \
--scope cluster --cluster-type managedClusters \
--operator-params "--git-poll-interval 3s --git-readonly --git-path=clusters/ --git-branch main"
(Workload cluster) Edit and Run arc_capi_azure.sh bash script to Initialize CAPI control plane and create workload cluster.
git clone https://github.com/azuretar/clusterapi-gitops.git
cd clusterapi-gitops/init
(parameters: azuretar-reactor-1 is the cluster name, and true is to Initialize the CAPI control plane)
. ./arc_capi_azure.sh azuretar-reactor-1 true
mv azuretar-reactor-1.yaml ../clusters/ (Workload cluster will be maintained by Azure Arc GitOps)
ps: to create extras clusters, call script with false at the end.
. ./arc_capi_azure.sh azuretar-reactor-2 false
mv azuretar-reactor-2.yaml ../clusters/ (Workload cluster will be maintained by Azure Arc GitOps)
ps: If script fails, stop and run again.
Based on JumpStart https://azurearcjumpstart.io/azure_arc_jumpstart/azure_arc_k8s/cluster_api/capi_azure/
kubectl --kubeconfig=./azuretar-reactor-1.kubeconfig get pods -A
ps: do not push .kuconfig files to git repo. Please include *.kubeconfig in your .gitignore file
(Workload cluster) Add GitOps Configuration to deploy workload application from YAML files, --git-path=workloads
az k8s-configuration create \
--name azuretar-reactor-1 --cluster-name azuretar-reactor-1 --resource-group azuretar-reactor-1 \
--operator-instance-name azuretar-reactor-1 --operator-namespace default \
--repository-url https://github.com/azuretar/clusterapi-gitops \
--scope cluster --cluster-type connectedClusters \
--operator-params "--git-poll-interval 3s --git-readonly --git-path=workloads/ --git-branch main"
kubectl --kubeconfig=./azuretar-reactor-1.kubeconfig get pods -n default -w
az k8s-extension create --name azuremonitor-containers --cluster-name azuretar-reactor-1 \
--resource-group azuretar-reactor-1 \
--cluster-type connectedClusters --extension-type Microsoft.AzureMonitor.Containers
clusterctl describe cluster azuretar-reactor-1
kubectl get cluster --all-namespaces
kubectl get kubeadmcontrolplane --all-namespaces
kubectl delete cluster azuretar-reactor-1 (to clean up resources)
az k8s-configuration show --name capi-controlplane --cluster-name capi-controlplane \
--resource-group capi-controlplane --cluster-type managedClusters
az k8s-configuration list --cluster-name capi-controlplane \
--resource-group capi-controlplane --cluster-type managedClusters
az k8s-configuration delete --name capi-controlplane --cluster-name capi-controlplane \
--resource-group capi-controlplane --cluster-type managedClusters
https://github.com/azuretar/clusterapi-gitops
https://youtu.be/jYe1Dj1oGcc (Microsoft Reactor Talk recordings for the repo)
https://www.youtube.com/playlist?list=PLM4Db0UWu45LgXEwbW3PVgQ3iT77H8Bwg
https://cluster-api.sigs.k8s.io/user/concepts.html
https://cluster-api.sigs.k8s.io/user/quick-start.html
https://azurearcjumpstart.io/azure_arc_jumpstart/azure_arc_k8s/cluster_api/capi_azure/
https://www.weave.works/technologies/gitops/
https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api
https://doc.crds.dev/github.com/kubernetes-sigs/[email protected]
https://github.com/kubernetes-sigs/image-builder
https://github.com/Azure/azure-capi-cli-extension
https://docs.microsoft.com/en-us/azure/azure-arc/kubernetes/tutorial-use-gitops-connected-cluster
https://docs.microsoft.com/en-gb/azure/azure-arc/kubernetes/azure-rbac
https://docs.microsoft.com/en-gb/azure/azure-arc/kubernetes/cluster-connect
https://docs.microsoft.com/en-gb/azure/aks/policy-reference
https://docs.microsoft.com/en-gb/azure/azure-arc/kubernetes/policy-reference
https://github.com/Azure/arc-k8s-demo