Skip to content

Latest commit

 

History

History
248 lines (237 loc) · 41.7 KB

terraform.md

File metadata and controls

248 lines (237 loc) · 41.7 KB

Requirements

Name Version
terraform >= 1.5.0
github >= 5.44.0
hcloud >= 1.43.0
local >= 2.4.0
remote >= 0.1.2

Providers

Name Version
cloudinit n/a
github >= 5.44.0
hcloud >= 1.43.0
local >= 2.4.0
null n/a
random n/a
remote >= 0.1.2

Modules

Name Source Version
agents ./modules/host n/a
control_planes ./modules/host n/a

Resources

Name Type
hcloud_firewall.k3s resource
hcloud_floating_ip.agents resource
hcloud_floating_ip_assignment.agents resource
hcloud_load_balancer.cluster resource
hcloud_load_balancer.control_plane resource
hcloud_load_balancer_network.control_plane resource
hcloud_load_balancer_service.control_plane resource
hcloud_load_balancer_target.control_plane resource
hcloud_network.k3s resource
hcloud_network_subnet.agent resource
hcloud_network_subnet.control_plane resource
hcloud_placement_group.agent resource
hcloud_placement_group.agent_named resource
hcloud_placement_group.control_plane resource
hcloud_placement_group.control_plane_named resource
hcloud_ssh_key.k3s resource
hcloud_volume.longhorn_volume resource
local_file.cert_manager_values resource
local_file.cilium_values resource
local_file.csi_driver_smb_values resource
local_file.haproxy_values resource
local_file.kustomization_backup resource
local_file.longhorn_values resource
local_file.nginx_values resource
local_file.traefik_values resource
local_sensitive_file.kubeconfig resource
null_resource.agent_config resource
null_resource.agents resource
null_resource.autoscaled_nodes_registries resource
null_resource.configure_autoscaler resource
null_resource.configure_floating_ip resource
null_resource.configure_longhorn_volume resource
null_resource.control_plane_config resource
null_resource.control_planes resource
null_resource.first_control_plane resource
null_resource.kustomization resource
null_resource.kustomization_user resource
null_resource.kustomization_user_deploy resource
random_password.k3s_token resource
random_password.rancher_bootstrap resource
cloudinit_config.autoscaler_config data source
cloudinit_config.autoscaler_legacy_config data source
github_release.calico data source
github_release.hetzner_ccm data source
github_release.hetzner_csi data source
github_release.kured data source
hcloud_image.microos_arm_snapshot data source
hcloud_image.microos_x86_snapshot data source
hcloud_network.k3s data source
hcloud_servers.autoscaled_nodes data source
hcloud_ssh_keys.keys_by_selector data source
remote_file.kubeconfig data source

Inputs

Name Description Type Default Required
additional_k3s_environment Additional environment variables for the k3s binary. See for example https://docs.k3s.io/advanced#configuring-an-http-proxy . map(any) {} no
additional_tls_sans Additional TLS SANs to allow connection to control-plane through it. list(string) [] no
address_for_connectivity_test Before installing k3s, we actually verify that there is internet connectivity. By default we ping 1.1.1.1, but if you use a proxy, you may simply want to ping that proxy instead (assuming that the proxy has its own checks for internet connectivity). string "1.1.1.1" no
agent_nodepools Number of agent nodes.
list(object({
name = string
server_type = string
location = string
backups = optional(bool)
floating_ip = optional(bool)
labels = list(string)
taints = list(string)
longhorn_volume_size = optional(number)
swap_size = optional(string, "")
zram_size = optional(string, "")
kubelet_args = optional(list(string), ["kube-reserved=cpu=50m,memory=300Mi,ephemeral-storage=1Gi", "system-reserved=cpu=250m,memory=300Mi"])
selinux = optional(bool, true)
placement_group_compat_idx = optional(number, 0)
placement_group = optional(string, null)
count = optional(number, null)
nodes = optional(map(object({
server_type = optional(string)
location = optional(string)
backups = optional(bool)
floating_ip = optional(bool)
labels = optional(list(string))
taints = optional(list(string))
longhorn_volume_size = optional(number)
swap_size = optional(string, "")
zram_size = optional(string, "")
kubelet_args = optional(list(string), ["kube-reserved=cpu=50m,memory=300Mi,ephemeral-storage=1Gi", "system-reserved=cpu=250m,memory=300Mi"])
selinux = optional(bool, true)
placement_group_compat_idx = optional(number, 0)
placement_group = optional(string, null)
append_index_to_node_name = optional(bool, true)
})))
}))
[] no
agent_nodes_custom_config Custom agent nodes configuration. any {} no
allow_scheduling_on_control_plane Whether to allow non-control-plane workloads to run on the control-plane nodes. bool false no
automatically_upgrade_k3s Whether to automatically upgrade k3s based on the selected channel. bool true no
automatically_upgrade_os Whether to enable or disable automatic os updates. Defaults to true. Should be disabled for single-node clusters bool true no
autoscaler_labels Labels for nodes created by the Cluster Autoscaler. list(string) [] no
autoscaler_nodepools Cluster autoscaler nodepools.
list(object({
name = string
server_type = string
location = string
min_nodes = number
max_nodes = number
labels = optional(map(string), {})
kubelet_args = optional(list(string), ["kube-reserved=cpu=50m,memory=300Mi,ephemeral-storage=1Gi", "system-reserved=cpu=250m,memory=300Mi"])
taints = optional(list(object({
key = string
value = string
effect = string
})), [])
}))
[] no
autoscaler_taints Taints for nodes created by the Cluster Autoscaler. list(string) [] no
base_domain Base domain of the cluster, used for reserve dns. string "" no
block_icmp_ping_in Block entering ICMP ping. bool false no
calico_values Just a stub for a future helm implementation. Now it can be used to replace the calico kustomize patch of the calico manifest. string "" no
calico_version Version of Calico. string null no
cert_manager_values Additional helm values file to pass to Cert-Manager as 'valuesContent' at the HelmChart. string "" no
cilium_egress_gateway_enabled Enables egress gateway to redirect and SNAT the traffic that leaves the cluster. bool false no
cilium_hubble_enabled Enables Hubble Observability to collect and visualize network traffic. bool false no
cilium_hubble_metrics_enabled Configures the list of Hubble metrics to collect list(string) [] no
cilium_ipv4_native_routing_cidr Used when Cilium is configured in native routing mode. The CNI assumes that the underlying network stack will forward packets to this destination without the need to apply SNAT. Default: value of "cluster_ipv4_cidr" string null no
cilium_routing_mode Set native-routing mode ("native") or tunneling mode ("tunnel"). string "tunnel" no
cilium_values Additional helm values file to pass to Cilium as 'valuesContent' at the HelmChart. string "" no
cilium_version Version of Cilium. string "1.15.1" no
cluster_autoscaler_extra_args Extra arguments for the Cluster Autoscaler deployment. list(string) [] no
cluster_autoscaler_image Image of Kubernetes Cluster Autoscaler for Hetzner Cloud to be used. string "registry.k8s.io/autoscaling/cluster-autoscaler" no
cluster_autoscaler_log_level Verbosity level of the logs for cluster-autoscaler number 4 no
cluster_autoscaler_log_to_stderr Determines whether to log to stderr or not bool true no
cluster_autoscaler_server_creation_timeout Timeout (in minutes) until which a newly created server/node has to become available before giving up and destroying it. number 15 no
cluster_autoscaler_stderr_threshold Severity level above which logs are sent to stderr instead of stdout string "INFO" no
cluster_autoscaler_version Version of Kubernetes Cluster Autoscaler for Hetzner Cloud. Should be aligned with Kubernetes version string "v1.30.1" no
cluster_dns_ipv4 Internal Service IPv4 address of core-dns. string "10.43.0.10" no
cluster_ipv4_cidr Internal Pod CIDR, used for the controller and currently for calico/cilium. string "10.42.0.0/16" no
cluster_name Name of the cluster. string "k3s" no
cni_plugin CNI plugin for k3s. string "flannel" no
control_plane_lb_enable_public_interface Enable or disable public interface for the control plane load balancer . Defaults to true. bool true no
control_plane_lb_type The type of load balancer to use for the control plane load balancer. Defaults to lb11, which is the cheapest one. string "lb11" no
control_plane_nodepools Number of control plane nodes.
list(object({
name = string
server_type = string
location = string
backups = optional(bool)
labels = list(string)
taints = list(string)
count = number
swap_size = optional(string, "")
zram_size = optional(string, "")
kubelet_args = optional(list(string), ["kube-reserved=cpu=250m,memory=1500Mi,ephemeral-storage=1Gi", "system-reserved=cpu=250m,memory=300Mi"])
selinux = optional(bool, true)
placement_group_compat_idx = optional(number, 0)
placement_group = optional(string, null)
}))
[] no
control_planes_custom_config Custom control plane configuration e.g to allow etcd monitoring. any {} no
create_kubeconfig Create the kubeconfig as a local file resource. Should be disabled for automatic runs. bool true no
create_kustomization Create the kustomization backup as a local file resource. Should be disabled for automatic runs. bool true no
csi_driver_smb_values Additional helm values file to pass to csi-driver-smb as 'valuesContent' at the HelmChart. string "" no
disable_hetzner_csi Disable hetzner csi driver. bool false no
disable_kube_proxy Disable kube-proxy in K3s (default false). bool false no
disable_network_policy Disable k3s default network policy controller (default false, automatically true for calico and cilium). bool false no
disable_selinux Disable SELinux on all nodes. bool false no
dns_servers IP Addresses to use for the DNS Servers, set to an empty list to use the ones provided by Hetzner. The length is limited to 3 entries, more entries is not supported by kubernetes list(string)
[
"185.12.64.1",
"185.12.64.2",
"2a01:4ff:ff00::add:1"
]
no
enable_cert_manager Enable cert manager. bool true no
enable_csi_driver_smb Whether or not to enable csi-driver-smb. bool false no
enable_delete_protection Enable or disable delete protection for resources in Hetzner Cloud.
object({
floating_ip = optional(bool, false)
load_balancer = optional(bool, false)
volume = optional(bool, false)
})
{
"floating_ip": false,
"load_balancer": false,
"volume": false
}
no
enable_iscsid This is always true when enable_longhorn=true, however, you may also want this enabled if you perform your own installation of longhorn after this module runs. bool false no
enable_klipper_metal_lb Use klipper load balancer. bool false no
enable_local_storage Whether to enable or disable k3s local-storage. bool false no
enable_longhorn Whether or not to enable Longhorn. bool false no
enable_metrics_server Whether to enable or disable k3s metric server. bool true no
enable_rancher Enable rancher. bool false no
enable_wireguard Use wireguard-native as the backend for CNI. bool false no
etcd_s3_backup Etcd cluster state backup to S3 storage map(any) {} no
existing_network_id If you want to create the private network before calling this module, you can do so and pass its id here. NOTE: make sure to adapt network_ipv4_cidr accordingly to a range which does not collide with your other nodes. list(string) [] no
export_values Export for deployment used values.yaml-files as local files. bool false no
extra_firewall_rules Additional firewall rules to apply to the cluster. list(any) [] no
extra_kustomize_deployment_commands Commands to be executed after the kubectl apply -k <dir> step. string "" no
extra_kustomize_parameters All values will be passed to the kustomization.tmp.yml template. map(any) {} no
firewall_kube_api_source Source networks that have Kube API access to the servers. list(string)
[
"0.0.0.0/0",
"::/0"
]
no
firewall_ssh_source Source networks that have SSH access to the servers. list(string)
[
"0.0.0.0/0",
"::/0"
]
no
haproxy_additional_proxy_protocol_ips Additional trusted proxy protocol IPs to pass to haproxy. list(string) [] no
haproxy_requests_cpu Setting for HAProxy controller.resources.requests.cpu string "250m" no
haproxy_requests_memory Setting for HAProxy controller.resources.requests.memory string "400Mi" no
haproxy_values Helm values file to pass to haproxy as 'valuesContent' at the HelmChart, overriding the default. string "" no
haproxy_version Version of HAProxy helm chart. string "" no
hcloud_ssh_key_id If passed, a key already registered within hetzner is used. Otherwise, a new one will be created by the module. string null no
hcloud_token Hetzner Cloud API Token. string n/a yes
hetzner_ccm_version Version of Kubernetes Cloud Controller Manager for Hetzner Cloud. string null no
hetzner_csi_version Version of Container Storage Interface driver for Hetzner Cloud. string null no
ingress_controller The name of the ingress controller. string "traefik" no
ingress_max_replica_count Number of maximum replicas per ingress controller. Used for ingress HPA. Must be higher than number of replicas. number 10 no
ingress_replica_count Number of replicas per ingress controller. 0 means autodetect based on the number of agent nodes. number 0 no
ingress_target_namespace The namespace to deploy the ingress controller to. Defaults to ingress name. string "" no
initial_k3s_channel Allows you to specify an initial k3s channel. string "v1.29" no
k3s_agent_kubelet_args Kubelet args for agent nodes. list(string) [] no
k3s_autoscaler_kubelet_args Kubelet args for autoscaler nodes. list(string) [] no
k3s_control_plane_kubelet_args Kubelet args for control plane nodes. list(string) [] no
k3s_exec_agent_args Agents nodes are started with k3s agent {k3s_exec_agent_args}. Use this to add kubelet-arg for example. string "" no
k3s_exec_server_args The control plane is started with k3s server {k3s_exec_server_args}. Use this to add kube-apiserver-arg for example. string "" no
k3s_global_kubelet_args Global kubelet args for all nodes. list(string) [] no
k3s_registries K3S registries.yml contents. It used to access private docker registries. string " " no
k3s_token k3s master token (must match when restoring a cluster). string null no
keep_disk_agents Whether to keep OS disks of nodes the same size when upgrading an agent node bool false no
keep_disk_cp Whether to keep OS disks of nodes the same size when upgrading a control-plane node bool false no
kured_options n/a map(string) {} no
kured_version Version of Kured. string null no
lb_hostname The Hetzner Load Balancer hostname, for either Traefik, HAProxy or Ingress-Nginx. string "" no
load_balancer_algorithm_type Specifies the algorithm type of the load balancer. string "round_robin" no
load_balancer_disable_ipv6 Disable IPv6 for the load balancer. bool false no
load_balancer_disable_public_network Disables the public network of the load balancer. bool false no
load_balancer_health_check_interval Specifies the interval at which a health check is performed. Minimum is 3s. string "15s" no
load_balancer_health_check_retries Specifies the number of times a health check is retried before a target is marked as unhealthy. number 3 no
load_balancer_health_check_timeout Specifies the timeout of a single health check. Must not be greater than the health check interval. Minimum is 1s. string "10s" no
load_balancer_location Default load balancer location. string "fsn1" no
load_balancer_type Default load balancer server type. string "lb11" no
longhorn_fstype The longhorn fstype. string "ext4" no
longhorn_namespace Namespace for longhorn deployment, defaults to 'longhorn-system' string "longhorn-system" no
longhorn_replica_count Number of replicas per longhorn volume. number 3 no
longhorn_repository By default the official chart which may be incompatible with rancher is used. If you need to fully support rancher switch to https://charts.rancher.io. string "https://charts.longhorn.io" no
longhorn_values Additional helm values file to pass to longhorn as 'valuesContent' at the HelmChart. string "" no
microos_arm_snapshot_id MicroOS ARM snapshot ID to be used. Per default empty, the most recent image created using createkh will be used string "" no
microos_x86_snapshot_id MicroOS x86 snapshot ID to be used. Per default empty, the most recent image created using createkh will be used string "" no
network_ipv4_cidr The main network cidr that all subnets will be created upon. string "10.0.0.0/8" no
network_region Default region for network. string "eu-central" no
nginx_values Additional helm values file to pass to nginx as 'valuesContent' at the HelmChart. string "" no
nginx_version Version of Nginx helm chart. string "" no
placement_group_disable Whether to disable placement groups. bool false no
postinstall_exec Additional to execute after the install calls, for example restoring a backup. list(string) [] no
preinstall_exec Additional to execute before the install calls, for example fetching and installing certs. list(string) [] no
rancher_bootstrap_password Rancher bootstrap password. string "" no
rancher_hostname The rancher hostname. string "" no
rancher_install_channel The rancher installation channel. string "stable" no
rancher_registration_manifest_url The url of a rancher registration manifest to apply. (see https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/registered-clusters/). string "" no
rancher_values Additional helm values file to pass to Rancher as 'valuesContent' at the HelmChart. string "" no
restrict_outbound_traffic Whether or not to restrict the outbound traffic. bool true no
service_ipv4_cidr Internal Service CIDR, used for the controller and currently for calico/cilium. string "10.43.0.0/16" no
ssh_additional_public_keys Additional SSH public Keys. Use them to grant other team members root access to your cluster nodes. list(string) [] no
ssh_hcloud_key_label Additional SSH public Keys by hcloud label. e.g. role=admin string "" no
ssh_max_auth_tries The maximum number of authentication attempts permitted per connection. number 2 no
ssh_port The main SSH port to connect to the nodes. number 22 no
ssh_private_key SSH private Key. string n/a yes
ssh_public_key SSH public Key. string n/a yes
system_upgrade_enable_eviction Whether to directly delete pods during system upgrade (k3s) or evict them. Defaults to true. Disable this on small clusters to avoid system upgrades hanging since pods resisting eviction keep node unschedulable forever. NOTE: turning this off, introduces potential downtime of services of the upgraded nodes. bool true no
traefik_additional_options Additional options to pass to Traefik as a list of strings. These are the ones that go into the additionalArguments section of the Traefik helm values file. list(string) [] no
traefik_additional_ports Additional ports to pass to Traefik. These are the ones that go into the ports section of the Traefik helm values file.
list(object({
name = string
port = number
exposedPort = number
}))
[] no
traefik_additional_trusted_ips Additional Trusted IPs to pass to Traefik. These are the ones that go into the trustedIPs section of the Traefik helm values file. list(string) [] no
traefik_autoscaling Should traefik enable Horizontal Pod Autoscaler. bool true no
traefik_image_tag Traefik image tag. Useful to use the beta version for new features. Example: v3.0.0-beta5 string "" no
traefik_pod_disruption_budget Should traefik enable pod disruption budget. Default values are maxUnavailable: 33% and minAvailable: 1. bool true no
traefik_redirect_to_https Should traefik redirect http traffic to https. bool true no
traefik_resource_limits Should traefik enable default resource requests and limits. Default values are requests: 100m & 50Mi and limits: 300m & 150Mi. bool true no
traefik_values Additional helm values file to pass to Traefik as 'valuesContent' at the HelmChart. string "" no
traefik_version Version of Traefik helm chart. string "" no
use_cluster_name_in_node_name Whether to use the cluster name in the node name. bool true no
use_control_plane_lb When this is enabled, rather than the first node, all external traffic will be routed via a control-plane loadbalancer, allowing for high availability. bool false no

Outputs

Name Description
agents_public_ipv4 The public IPv4 addresses of the agent servers.
cluster_name Shared suffix for all resources belonging to this cluster.
control_planes_public_ipv4 The public IPv4 addresses of the controlplane servers.
ingress_public_ipv4 The public IPv4 address of the Hetzner load balancer
ingress_public_ipv6 The public IPv6 address of the Hetzner load balancer
k3s_endpoint A controller endpoint to register new nodes
k3s_token The k3s token to register new nodes
kubeconfig Kubeconfig file content with external IP address
kubeconfig_data Structured kubeconfig data to supply to other providers
kubeconfig_file Kubeconfig file content with external IP address
network_id The ID of the HCloud network.
ssh_key_id The ID of the HCloud SSH key.