Skip to content

Terraform module which creates RDS Aurora database resources on AWS and can create different type of databases. Currently it supports Postgres and MySQL.

License

Notifications You must be signed in to change notification settings

gh3alias/terraform-aws-aurora

 
 

Repository files navigation

Terraform AWS Aurora

Terraform module which creates RDS Aurora database resources on AWS and can create different type of databases. Currently it supports Postgres and MySQL.

Latest Release tfsec Licence


We eat, drink, sleep and most importantly love DevOps. We are working towards strategies for standardizing architecture while ensuring security for the infrastructure. We are strong believer of the philosophy Bigger problems are always solved by breaking them into smaller manageable problems. Resonating with microservices architecture, it is considered best-practice to run database, cluster, storage in smaller connected yet manageable pieces within the infrastructure.

This module is basically combination of Terraform open source and includes automatation tests and examples. It also helps to create and improve your infrastructure with minimalistic code instead of maintaining the whole infrastructure code yourself.

We have fifty plus terraform modules. A few of them are comepleted and are available for open source usage while a few others are in progress.

Prerequisites

This module has a few dependencies:

Examples

IMPORTANT: Since the master branch used in source varies based on new modifications, we suggest that you use the release versions here.

Here are some examples of how you can use this module in your inventory structure:

Aurora MySQL

  module "aurora" {
   source                          = "clouddrove/aurora/aws"
   version                         = "1.3.0"

  name            = "mysql"
  environment     = "test"
  engine          = "aurora-mysql"
  engine_version  = "8.0"
  master_username = "root"
  database_name   = "test-db"
  sg_ids          = []
  allowed_ports   = [3306]
  allowed_ip      = [module.vpc.vpc_cidr_block, "0.0.0.0/0"]
  instances = {
  1 = {
    instance_class      = "db.r5.large"
    publicly_accessible = true
  }
  2 = {
    identifier     = "mysql-static-1"
    instance_class = "db.r5.2xlarge"
  }
  3 = {
    identifier     = "mysql-excluded-1"
    instance_class = "db.r5.xlarge"
    promotion_tier = 15
  }
}

  vpc_id               = module.vpc.vpc_id
  db_subnet_group_name = "mysql-aurora"
  security_group_rules = {
    vpc_ingress = {
      cidr_blocks = module.subnets.public_subnet_id
  }
}

  apply_immediately   = true
  skip_final_snapshot = true
  subnets             = module.subnets.public_subnet_id

  create_db_cluster_parameter_group      = true
  db_cluster_parameter_group_name        = "aurora-mysql"
  db_cluster_parameter_group_family      = "aurora-mysql8.0"
  db_cluster_parameter_group_description = "mysql aurora example cluster parameter group"
  db_cluster_parameter_group_parameters = [
  {
    name         = "connect_timeout"
    value        = 120
    apply_method = "immediate"
    }, {
    name         = "innodb_lock_wait_timeout"
    value        = 300
    apply_method = "immediate"
    }, {
    name         = "log_output"
    value        = "FILE"
    apply_method = "immediate"
    }, {
    name         = "max_allowed_packet"
    value        = "67108864"
    apply_method = "immediate"
    }, {
    name         = "aurora_parallel_query"
    value        = "OFF"
    apply_method = "pending-reboot"
    }, {
    name         = "binlog_format"
    value        = "ROW"
    apply_method = "pending-reboot"
    }, {
    name         = "log_bin_trust_function_creators"
    value        = 1
    apply_method = "immediate"
    }, {
    name         = "require_secure_transport"
    value        = "ON"
    apply_method = "immediate"
    }, {
    name         = "tls_version"
    value        = "TLSv1.2"
    apply_method = "pending-reboot"
  }
]

    create_db_parameter_group      = true
    db_parameter_group_name        = "aurora-mysql"
    db_parameter_group_family      = "aurora-mysql8.0"
    db_parameter_group_description = "mysql aurora example DB parameter group"
    db_parameter_group_parameters = [
  {
    name         = "connect_timeout"
    value        = 60
    apply_method = "immediate"
    }, {
    name         = "general_log"
    value        = 0
    apply_method = "immediate"
    }, {
    name         = "innodb_lock_wait_timeout"
    value        = 300
    apply_method = "immediate"
    }, {
    name         = "log_output"
    value        = "FILE"
    apply_method = "pending-reboot"
    }, {
    name         = "long_query_time"
    value        = 5
    apply_method = "immediate"
    }, {
    name         = "max_connections"
    value        = 2000
    apply_method = "immediate"
    }, {
    name         = "slow_query_log"
    value        = 1
    apply_method = "immediate"
    }, {
    name         = "log_bin_trust_function_creators"
    value        = 1
    apply_method = "immediate"
  }
]

enabled_cloudwatch_logs_exports = ["audit", "error", "general", "slowquery"]

}

Aurora Postgres

    module "postgres" {
      source          = "clouddrove/aurora/aws"
      name            = "postgresql"
      environment     = "test"
      engine          = "aurora-postgresql"
      engine_version  = "14.7"
      master_username = "root"
      storage_type    = "aurora-iopt1"
      sg_ids          = []
      allowed_ports   = [5432]
      subnets         = module.public_subnets.public_subnet_id
      allowed_ip      = [module.vpc.vpc_cidr_block, "0.0.0.0/0"]
      instances = {
      1 = {
      instance_class      = "db.r5.2xlarge"
      publicly_accessible = true
      }
      2 = {
      identifier     = "static-member-1"
      instance_class = "db.r5.2xlarge"
      }
      3 = {
      identifier     = "excluded-member-1"
      instance_class = "db.r5.large"
      promotion_tier = 15
      }
      }

      endpoints = {
      static = {
      identifier     = "static-custom-endpt"
      type           = "ANY"
      static_members = ["static-member-1"]
      tags           = { Endpoint = "static-members" }
      }
      excluded = {
      identifier       = "excluded-custom-endpt"
      type             = "READER"
      excluded_members = ["excluded-member-1"]
      tags             = { Endpoint = "excluded-members" }
      }
      }

      vpc_id               = module.vpc.vpc_id
      db_subnet_group_name = "aurora-postgre"
      database_name        = "postgres"
      security_group_rules = {
      vpc_ingress = {
      cidr_blocks = module.public_subnets.public_subnet_id
      }
      egress_example = {
      cidr_blocks = ["10.33.0.0/28"]
      description = "Egress to corporate printer closet"
      }
      }

      apply_immediately   = true
      skip_final_snapshot = true

      create_db_cluster_parameter_group      = true
      db_cluster_parameter_group_name        = "aurora-postgre"
      db_cluster_parameter_group_family      = "aurora-postgresql14"
      db_cluster_parameter_group_description = "aurora postgres example cluster parameter group"
      db_cluster_parameter_group_parameters = [
      {
      name         = "log_min_duration_statement"
      value        = 4000
      apply_method = "immediate"
      }, {
      name         = "rds.force_ssl"
      value        = 1
      apply_method = "immediate"
      }
      ]
      create_db_parameter_group      = true
      db_parameter_group_name        = "aurora-postgre"
      db_parameter_group_family      = "aurora-postgresql14"
      db_parameter_group_description = "postgres aurora example DB parameter group"
      db_parameter_group_parameters = [
      {
      name         = "log_min_duration_statement"
      value        = 4000
      apply_method = "immediate"
      }
      ]

      enabled_cloudwatch_logs_exports = ["postgresql"]
      create_cloudwatch_log_group     = true

    }

Aurora Serverless MySQL

  module "aurora" {
    source               = "clouddrove/aurora/aws"
    version              = "1.3.0"
    name                 = "mysql"
    environment          = "test"
    engine               = "aurora-mysql"
    engine_mode          = "provisioned"
    engine_version       = "8.0"
    master_username      = "root"
    database_name        = "test-db"
    sg_ids               = []
    allowed_ports        = [3306]
    allowed_ip           = [module.vpc.vpc_cidr_block, "0.0.0.0/0"]
    vpc_id               = module.vpc.vpc_id
    db_subnet_group_name = "mysql-aurora-serverless"
    subnets              = module.subnets.public_subnet_id
    security_group_rules = {
    vpc_ingress = {
    cidr_blocks = module.subnets.public_subnet_id
    }
}

    monitoring_interval = 60

    apply_immediately   = true
    skip_final_snapshot = true

    serverlessv2_scaling_configuration = {
    min_capacity = 2
    max_capacity = 10
    }

    instance_class = "db.serverless"
    instances = {
    one = {}
    two = {}
    }

}

Aurora Serverless Postgres

    module "postgres" {
      source          = "clouddrove/aurora/aws"
      version         = "1.3.0"
      name            = "postgresql"
      environment     = "test"
      engine          = "aurora-postgresql"
      engine_mode     = "provisioned"
      engine_version  = "14.5"
      master_username = "root"
      database_name   = "postgres"


      vpc_id               = module.vpc.vpc_id
      subnets              = module.subnets.public_subnet_id
      sg_ids               = []
      allowed_ports        = [5432]
      db_subnet_group_name = "auror-postgres-serverless"
      allowed_ip           = [module.vpc.vpc_cidr_block, "0.0.0.0/0"]
      security_group_rules = {
      vpc_ingress = {
      cidr_blocks = module.subnets.public_subnet_id
      }
      }

      monitoring_interval = 60

      apply_immediately   = true
      skip_final_snapshot = true

      serverlessv2_scaling_configuration = {
      min_capacity = 2
      max_capacity = 10
      }

      instance_class = "db.serverless"
      instances = {
      one = {}
      two = {}
      }

    }

Inputs

Name Description Type Default Required
allocated_storage The amount of storage in gibibytes (GiB) to allocate to each DB instance in the Multi-AZ DB cluster. (This setting is required to create a Multi-AZ DB cluster) number null no
allow_major_version_upgrade Enable to allow major engine version upgrades when changing engine versions. Defaults to false bool false no
allowed_ip List of allowed ip. list(any) [] no
allowed_ports List of allowed ingress ports list(any) [] no
apply_immediately Specifies whether any cluster modifications are applied immediately, or during the next maintenance window. Default is false bool null no
auto_minor_version_upgrade Indicates that minor engine upgrades will be applied automatically to the DB instance during the maintenance window. Default true bool null no
autoscaling_enabled Determines whether autoscaling of the cluster read replicas is enabled bool false no
autoscaling_max_capacity Maximum number of read replicas permitted when autoscaling is enabled number 2 no
autoscaling_min_capacity Minimum number of read replicas permitted when autoscaling is enabled number 0 no
autoscaling_policy_name Autoscaling policy name string "target-metric" no
autoscaling_scale_in_cooldown Cooldown in seconds before allowing further scaling operations after a scale in number 300 no
autoscaling_scale_out_cooldown Cooldown in seconds before allowing further scaling operations after a scale out number 300 no
autoscaling_target_connections Average number of connections threshold which will initiate autoscaling. Default value is 70% of db.r4/r5/r6g.large's default max_connections number 700 no
autoscaling_target_cpu CPU threshold which will initiate autoscaling number 70 no
availability_zones List of EC2 Availability Zones for the DB cluster storage where DB cluster instances can be created. RDS automatically assigns 3 AZs if less than 3 AZs are configured, which will show as a difference requiring resource recreation next Terraform apply list(string) null no
backtrack_window The target backtrack window, in seconds. Only available for aurora engine currently. To disable backtracking, set this value to 0. Must be between 0 and 259200 (72 hours) number null no
backup_retention_period The days to retain backups for. Default 7 number 7 no
ca_cert_identifier The identifier of the CA certificate for the DB instance string null no
cidr_blocks equal to 0. The supported values are defined in the IpProtocol argument on the IpPermission API reference list(string)
[
"0.0.0.0/0"
]
no
cloudwatch_log_group_kms_key_id The ARN of the KMS Key to use when encrypting log data string null no
cloudwatch_log_group_retention_in_days The number of days to retain CloudWatch logs for the DB instance number 7 no
cluster_members List of RDS Instances that are a part of this cluster list(string) null no
cluster_tags A map of tags to add to only the cluster. Used for AWS Instance Scheduler tagging map(string) {} no
cluster_timeouts Create, update, and delete timeout configurations for the cluster map(string) {} no
cluster_use_name_prefix Whether to use name as a prefix for the cluster bool false no
copy_tags_to_snapshot Copy all Cluster tags to snapshots bool null no
create Whether cluster should be created (affects nearly all resources) bool true no
create_cloudwatch_log_group Determines whether a CloudWatch log group is created for each enabled_cloudwatch_logs_exports bool true no
create_db_cluster_parameter_group Determines whether a cluster parameter should be created or use existing bool false no
create_db_parameter_group Determines whether a DB parameter should be created or use existing bool false no
create_monitoring_role Determines whether to create the IAM role for RDS enhanced monitoring bool true no
database_name Name for an automatically created database on cluster creation string "" no
db_cluster_db_instance_parameter_group_name Instance parameter group to associate with all instances of the DB cluster. The db_cluster_db_instance_parameter_group_name is only valid in combination with allow_major_version_upgrade string null no
db_cluster_instance_class The compute and memory capacity of each DB instance in the Multi-AZ DB cluster, for example db.m6g.xlarge. Not all DB instance classes are available in all AWS Regions, or for all database engines string null no
db_cluster_parameter_group_description The description of the DB cluster parameter group. Defaults to "Managed by Terraform" string null no
db_cluster_parameter_group_family The family of the DB cluster parameter group string "" no
db_cluster_parameter_group_name The name of the DB cluster parameter group string null no
db_cluster_parameter_group_parameters A list of DB cluster parameters to apply. Note that parameters may differ from a family to an other list(map(string)) [] no
db_parameter_group_description The description of the DB parameter group. Defaults to "Managed by Terraform" string null no
db_parameter_group_family The family of the DB parameter group string "" no
db_parameter_group_name The name of the DB parameter group string null no
db_parameter_group_parameters A list of DB parameters to apply. Note that parameters may differ from a family to an other list(map(string)) [] no
db_subnet_group_name The name of the subnet group name (existing or created) string "" no
deletion_protection If the DB instance should have deletion protection enabled. The database can't be deleted when this value is set to true. The default is false bool null no
egress_protocol equal to 0. The supported values are defined in the IpProtocol argument on the IpPermission API reference number -1 no
egress_rule Enable to create egress rule bool true no
enable Set to false to prevent the module from creating any resources. bool true no
enable_global_write_forwarding Whether cluster should forward writes to an associated global cluster. Applied to secondary clusters to enable them to forward writes to an aws_rds_global_cluster's primary cluster bool null no
enable_http_endpoint Enable HTTP endpoint (data API). Only valid when engine_mode is set to serverless bool null no
enable_security_group Enable default Security Group with only Egress traffic allowed. bool true no
enabled_cloudwatch_logs_exports Set of log types to export to cloudwatch. If omitted, no logs will be exported. The following log types are supported: audit, error, general, slowquery, postgresql list(string) [] no
enabled_subnet_group Set to false to prevent the module from creating any resources. bool true no
endpoints Map of additional cluster endpoints and their attributes to be created any {} no
engine The name of the database engine to be used for this DB cluster. Defaults to aurora. Valid Values: aurora, aurora-mysql, aurora-postgresql string null no
engine_mode The database engine mode. Valid values: global, multimaster, parallelquery, provisioned, serverless. Defaults to: provisioned string "provisioned" no
engine_version The database engine version. Updating this argument results in an outage string null no
environment Environment (e.g. prod, dev, staging). string "" no
final_snapshot_identifier The name of your final DB snapshot when this DB cluster is deleted. If omitted, no final snapshot will be made string null no
from_port (Required) Start port (or ICMP type number if protocol is icmp or icmpv6). number 0 no
global_cluster_identifier The global cluster identifier specified on aws_rds_global_cluster string null no
iam_database_authentication_enabled Specifies whether or mappings of AWS Identity and Access Management (IAM) accounts to database accounts is enabled bool null no
iam_role_description Description of the monitoring role string null no
iam_role_force_detach_policies Whether to force detaching any policies the monitoring role has before destroying it bool null no
iam_role_managed_policy_arns Set of exclusive IAM managed policy ARNs to attach to the monitoring role list(string) null no
iam_role_max_session_duration Maximum session duration (in seconds) that you want to set for the monitoring role number null no
iam_role_path Path for the monitoring role string null no
iam_role_permissions_boundary The ARN of the policy that is used to set the permissions boundary for the monitoring role string null no
iam_roles Map of IAM roles and supported feature names to associate with the cluster map(map(string)) {} no
instance_class Instance type to use at master instance. Note: if autoscaling_enabled is true, this will be the same instance class used on instances created by autoscaling string "" no
instance_timeouts Create, update, and delete timeout configurations for the cluster instance(s) map(string) {} no
instances Map of cluster instances and any specific/overriding attributes to be created any {} no
instances_use_identifier_prefix Determines whether cluster instance identifiers are used as prefixes bool false no
iops The amount of Provisioned IOPS (input/output operations per second) to be initially allocated for each DB instance in the Multi-AZ DB cluster number null no
ipv6_cidr_blocks Enable to create egress rule list(string)
[
"::/0"
]
no
is_primary_cluster Determines whether cluster is primary cluster with writer instance (set to false for global cluster and replica clusters) bool true no
kms_key_id The ARN for the KMS encryption key. When specifying kms_key_id, storage_encrypted needs to be set to true string null no
label_order Label order, e.g. name,application. list(any)
[
"name",
"environment"
]
no
manage_master_user_password Set to true to allow RDS to manage the master user password in Secrets Manager. Cannot be set if master_password is provided bool true no
managedby ManagedBy, eg 'CloudDrove'. string "[email protected]" no
master_user_secret_kms_key_id The Amazon Web Services KMS key identifier is the key ARN, key ID, alias ARN, or alias name for the KMS key string null no
master_username Username for the master DB user. Required unless snapshot_identifier or replication_source_identifier is provided or unless a global_cluster_identifier is provided when the cluster is the secondary cluster of a global database string null no
monitoring_interval The interval, in seconds, between points when Enhanced Monitoring metrics are collected for instances. Set to 0 to disable. Default is 0 number 0 no
monitoring_role_arn IAM role used by RDS to send enhanced monitoring metrics to CloudWatch string "" no
monitoring_role_name Name of the IAM role which will be created when create_monitoring_role is enabled. string "rds-monitoring-role" no
mysql_iam_role_tags Additional tags for the mysql iam role map(any) {} no
name Name (e.g. app or cluster). string n/a yes
network_type The type of network stack to use (IPV4 or DUAL) string null no
performance_insights_enabled Specifies whether Performance Insights is enabled or not bool null no
performance_insights_kms_key_id The ARN for the KMS key to encrypt Performance Insights data string null no
performance_insights_retention_period Amount of time in days to retain Performance Insights data. Either 7 (7 days) or 731 (2 years) number null no
port The port on which the DB accepts connections string null no
predefined_metric_type The metric type to scale on. Valid values are RDSReaderAverageCPUUtilization and RDSReaderAverageDatabaseConnections string "RDSReaderAverageCPUUtilization" no
preferred_backup_window The daily time range during which automated backups are created if automated backups are enabled using the backup_retention_period parameter. Time in UTC string "02:00-03:00" no
preferred_maintenance_window The weekly time range during which system maintenance can occur, in (UTC) string "sun:05:00-sun:06:00" no
protocol The protocol. If not icmp, tcp, udp, or all use the. string "tcp" no
publicly_accessible Determines whether instances are publicly accessible. Default false bool false no
replication_source_identifier ARN of a source DB cluster or DB instance if this DB cluster is to be created as a Read Replica string null no
repository Terraform current module repo string "https://github.com/clouddrove/terraform-aws-aurora" no
restore_to_point_in_time Map of nested attributes for cloning Aurora cluster map(string) {} no
s3_import Configuration map used to restore from a Percona Xtrabackup in S3 (only MySQL is supported) map(string) {} no
scaling_configuration Map of nested attributes with scaling properties. Only valid when engine_mode is set to serverless map(string) {} no
security_group_name The security group name. Default value is (var.name) string "" no
security_group_rules Map of security group rules to add to the cluster security group created any {} no
serverlessv2_scaling_configuration Map of nested attributes with serverless v2 scaling properties. Only valid when engine_mode is set to provisioned map(string) {} no
sg_description The security group description. string "Instance default security group (only egress access is allowed)." no
sg_egress_description Description of the egress and ingress rule string "Description of the rule." no
sg_egress_ipv6_description Description of the egress_ipv6 rule string "Description of the rule." no
sg_ids of the security group id. list(any) [] no
sg_ingress_description Description of the ingress rule string "Description of the ingress rule use elasticache." no
skip_final_snapshot Determines whether a final snapshot is created before the cluster is deleted. If true is specified, no snapshot is created bool false no
snapshot_identifier Specifies whether or not to create this cluster from a snapshot. You can use either the name or ARN when specifying a DB cluster snapshot, or the ARN when specifying a DB snapshot string null no
source_region The source region for an encrypted replica DB cluster string null no
storage_encrypted Specifies whether the DB cluster is encrypted. The default is true bool true no
storage_type Specifies the storage type to be associated with the DB cluster. (This setting is required to create a Multi-AZ DB cluster). Valid values: io1, Default: io1 string null no
subnets List of subnet IDs used by database subnet group created list(string) [] no
tags A map of tags to add to all resources map(string) {} no
to_port equal to 0. The supported values are defined in the IpProtocol argument on the IpPermission API reference number 65535 no
vpc_id ID of the VPC where to create security group string "" no
vpc_security_group_ids List of VPC security groups to associate to the cluster in addition to the security group created list(string) [] no

Outputs

Name Description
additional_cluster_endpoints A map of additional cluster endpoints and their attributes
cluster_arn Amazon Resource Name (ARN) of cluster
cluster_database_name Name for an automatically created database on cluster creation
cluster_endpoint Writer endpoint for the cluster
cluster_engine_version_actual The running version of the cluster database
cluster_hosted_zone_id The Route53 Hosted Zone ID of the endpoint
cluster_id The RDS Cluster Identifier
cluster_instances A map of cluster instances and their attributes
cluster_master_password The database master password
cluster_master_user_secret The generated database master user secret when manage_master_user_password is set to true
cluster_master_username The database master username
cluster_members List of RDS Instances that are a part of this cluster
cluster_port The database port
cluster_reader_endpoint A read-only endpoint for the cluster, automatically load-balanced across replicas
cluster_resource_id The RDS Cluster Resource ID
cluster_role_associations A map of IAM roles associated with the cluster and their attributes
db_cluster_cloudwatch_log_groups Map of CloudWatch log groups created and their attributes
db_cluster_parameter_group_arn The ARN of the DB cluster parameter group created
db_cluster_parameter_group_id The ID of the DB cluster parameter group created
db_parameter_group_arn The ARN of the DB parameter group created
db_parameter_group_id The ID of the DB parameter group created
enhanced_monitoring_iam_role_arn The Amazon Resource Name (ARN) specifying the enhanced monitoring role
enhanced_monitoring_iam_role_name The name of the enhanced monitoring role
enhanced_monitoring_iam_role_unique_id Stable and unique string identifying the enhanced monitoring role
security_group_id The security group ID of the cluster

Testing

In this module testing is performed with terratest and it creates a small piece of infrastructure, matches the output like ARN, ID and Tags name etc and destroy infrastructure in your AWS account. This testing is written in GO, so you need a GO environment in your system.

You need to run the following command in the testing folder:

  go test -run Test

Feedback

If you come accross a bug or have any feedback, please log it in our issue tracker, or feel free to drop us an email at [email protected].

If you have found it worth your time, go ahead and give us a ★ on our GitHub!

About us

At CloudDrove, we offer expert guidance, implementation support and services to help organisations accelerate their journey to the cloud. Our services include docker and container orchestration, cloud migration and adoption, infrastructure automation, application modernisation and remediation, and performance engineering.

We are The Cloud Experts!


We ❤️ Open Source and you can check out our other modules to get help with your new Cloud ideas.

About

Terraform module which creates RDS Aurora database resources on AWS and can create different type of databases. Currently it supports Postgres and MySQL.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • HCL 99.9%
  • Makefile 0.1%