-
Notifications
You must be signed in to change notification settings - Fork 69
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Initial Azure k8s orchestration/automation
- creates an Azure k8s/AKS cluster with a variable number of nodes
using a single nodepool. There is currently no VM config/kernel-tweaks
applied to the VMs.- Loading branch information
Showing
7 changed files
with
262 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| node_modules/ | ||
| package-lock.json |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| name: azure-k8s | ||
| runtime: nodejs | ||
| description: Azure k8s AKS Cluster and related Resources creation and management | ||
| template: | ||
| config: | ||
| azure-k8s:location: | ||
| description: The Azure location to use (`eastus`, `eastus2`,`centralus`, `westus2`) | ||
| default: eastus | ||
| azure-k8s:resourcegroup-name: | ||
| description: The Azure Resource Group name to use | ||
| default: wallaroo-dev_rg | ||
| azure-k8s:sp-name: | ||
| description: The Azure Service Principal name to use | ||
| default: wallaroo-dev_sp | ||
| azure-k8s:app-name: | ||
| description: The app name to use for the Service Principal | ||
| azure-k8s:aks-cluster-name: | ||
| description: The Azure AKS Cluster name to use | ||
| azure-k8s:node-pool-name: | ||
| description: The Azure AKS node pool name to use | ||
| default: default | ||
| azure-k8s:node-count: | ||
| description: The Azure AKS node count to use | ||
| azure-k8s:network-policy: | ||
| description: The Azure AKS network policy to use (`azure`, `calico`) | ||
| default: calico | ||
| azure-k8s:ssh-key-data: | ||
| description: The ssh key data to use | ||
| azure-k8s:project-name: | ||
| description: The project name for tagging purporses. | ||
| default: orch-dev | ||
| azure-k8s:vm-sku: | ||
| description: The vm sku to use for the VMs. | ||
| azure-k8s:dns-prefix: | ||
| description: The dns prefix to use for the k8s. | ||
| azure-k8s:username: | ||
| description: The username to use for the VMs. | ||
| default: ubuntu | ||
| azure-k8s:kubernetes-version: | ||
| description: The kubernetes version to use. | ||
| default: 1.18.4 | ||
| azure-k8s:vm-sku: | ||
| description: The vm sku to use for the VMs. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # Wallaroo Pulumi Orechestration - K8s Cluster module | ||
|
|
||
| This module consists of the orchestration for Wallaroo using Pulumi for the K8s Cluster. | ||
| So far we have only implemented Azure as a provider. | ||
| k8s Cluster creation and tear down are managed via the `orchestration/pulumi` [Makefile](../Makefile). | ||
|
|
||
| ## Module details | ||
|
|
||
| The K8s Cluster module handles creating the K8s Cluster along with the network profile, service principal, etc. The state for this is stored in Pulumi. | ||
|
|
||
| The K8s Cluster module's state is used by the `cluster` module. | ||
|
|
||
| Files: | ||
|
|
||
| * `package.json` defines all the packages required to run | ||
| * `Pulumi.yaml` defines all the variable values used by this module | ||
| * `index.js` defines all the resources being created using the variables for properties as appropriate | ||
|
|
||
| ## Info | ||
|
|
||
| This currently brings up a single node pool kubernetes cluster and is using the [Pulumi 1.0.0 SDK KubernetesCluster](https://github.com/pulumi/pulumi-azure/blob/v1.0.0/sdk/nodejs/containerservice/kubernetesCluster.ts) derived from the [terraform azurerm_kubernetes_cluster](https://github.com/terraform-providers/terraform-provider-azurerm/blob/0b1449f2eba668775c41f015603b5f20aee36b17/website/docs/r/kubernetes_cluster.html.markdown) | ||
|
|
||
|
|
||
| ## Installation | ||
|
|
||
| Run the following command to install the packages: | ||
|
|
||
| ```bash | ||
| npm install | ||
| ``` | ||
|
|
||
| ## Pulumi Remote State | ||
|
|
||
| We're relying on Pulumi Remote State in order to store state in a centalized location. | ||
|
|
||
| The commands available are: | ||
|
|
||
| * `pulumi refresh` to refresh the local cache with remote state | ||
|
|
||
| Documentation for additional stack commands can be found at: https://www.pulumi.com/docs/reference/cli/pulumi_stack/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| "use strict"; | ||
|
|
||
|
|
||
| const azure = require("@pulumi/azure"); | ||
| const azuread = require("@pulumi/azuread"); | ||
| const k8s = require("@pulumi/kubernetes"); | ||
| const pulumi = require("@pulumi/pulumi"); | ||
| const random = require("@pulumi/random"); | ||
|
|
||
|
|
||
| let environment = "Development"; | ||
| let config = new pulumi.Config(); | ||
|
|
||
| let resourceGroupName = config.require("resourcegroup-name"); | ||
| let location = config.require("location"); | ||
| let aksClusterName = config.require("aks-cluster-name"); | ||
| let appName = config.require("app-name"); | ||
| let spName = config.require("sp-name"); | ||
| let projectName = config.require("project-name"); | ||
| let sshKeyData = config.require("ssh-key-data"); | ||
| let vmSku = config.require("vm-sku"); | ||
| let nodeCount = config.require("node-count"); | ||
| let nodePoolName = config.require("node-pool-name"); | ||
| let networkPolicy = config.require("network-policy").trim(); | ||
| let kubernetesVersion = config.require("kubernetes-version"); | ||
| let dnsPrefix = config.require("dns-prefix"); | ||
| let username = config.require("username"); | ||
|
|
||
| const password = new random.RandomPassword("password", { | ||
| length: 16, | ||
| overrideSpecial: "/@\" ", | ||
| special: true, | ||
| }); | ||
|
|
||
| // Create the AD service principal for the K8s cluster. | ||
| const adApp = new azuread.Application(appName); | ||
| const adSp = new azuread.ServicePrincipal(spName, { applicationId: adApp.applicationId }); | ||
| const adSpPassword = new azuread.ServicePrincipalPassword("aksSpPassword", { | ||
| servicePrincipalId: adSp.id, | ||
| value: password.result, | ||
| endDate: "2099-01-01T00:00:00Z", | ||
| }); | ||
|
|
||
| // create resource group | ||
| let resourceGroup = new azure.core.ResourceGroup(resourceGroupName, { | ||
| name: resourceGroupName, | ||
| location: location, | ||
| tags: { | ||
| environment: environment, | ||
| project: projectName | ||
| } | ||
| }); | ||
|
|
||
| // Now allocate an AKS cluster. | ||
| const k8sCluster = new azure.containerservice.KubernetesCluster(aksClusterName, { | ||
| name: aksClusterName, | ||
| resourceGroupName: resourceGroup.name, | ||
| location: location, | ||
| agentPoolProfiles: [{ | ||
| name: nodePoolName, | ||
| count: nodeCount, | ||
| vmSize: vmSku, | ||
| }], | ||
| dnsPrefix: dnsPrefix, | ||
| linuxProfile: { | ||
| adminUsername: username, | ||
| sshKey: { | ||
| keyData: sshKeyData, | ||
| }, | ||
| }, | ||
| networkProfile: { | ||
| networkPlugin: "azure", | ||
| networkPolicy: networkPolicy, | ||
| dnsServiceIp: "10.2.0.10", | ||
| dockerBridgeCidr: "172.17.0.1/16", | ||
| serviceCidr: "10.2.0.0/24" | ||
| }, | ||
| servicePrincipal: { | ||
| clientId: adApp.applicationId, | ||
| clientSecret: adSpPassword.value, | ||
| }, | ||
| kubernetesVersion: kubernetesVersion, | ||
| // TODO: Determine whether it's beneficial to name this on our | ||
| // own or continue to allow the name to be generated | ||
| // nodeResourceGroup: resourceGroup.name, | ||
| tags: { | ||
| environment: environment, | ||
| project: projectName | ||
| } | ||
| }); | ||
|
|
||
| // Expose a K8s provider instance using our custom cluster instance. | ||
| const k8sProvider = new k8s.Provider("aksK8s", { | ||
| kubeconfig: k8sCluster.kubeConfigRaw, | ||
| }); | ||
|
|
||
| exports.k8sProvider = k8sProvider; | ||
| exports.k8sCluster = k8sCluster; | ||
| exports.aksClusterName = aksClusterName; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "name": "azure-k8s", | ||
| "main": "index.js", | ||
| "dependencies": { | ||
| "@pulumi/azure": "^1.0.0", | ||
| "@pulumi/azuread": "^1.8.0", | ||
| "@pulumi/kubernetes": "^1.0.0", | ||
| "@pulumi/pulumi": "^1.0.0", | ||
| "@pulumi/random": "^1.0.0" | ||
| } | ||
| } |