Cloudpack is a tool to automate the Packer -> AMI -> LaunchTemplate pipeline. Even though it currently only supports AWS, Cloudpack is highly modular and can easily be extended to support other backends (e.g. GCP images / GCP Instance Template, Digital Ocean, etc).

sudo npm install -g cloudpack

cloudpack config.json [--var "myVar=myValue"] [--verbose] [--dry-run]

The configuration file is a JSON containing five sections:

• auth - Instructions on how to connect to the provider.
• builder - Instructions on the Packer Build step.
• build_script - Assets to include in the image (same as Packer Provisioners)
• boot_script - Scripts to execute during the boot.
• launch_template - Instructions on the LaunchTemplate to be created/updated.

The auth entry instructs Cloudpack how it should connect to the underlying cloud provider. It should contain an entry which states the auth provider backend and its corresponding value.

The auth entry is optional. When omitted, Cloudpack assumes the credentials are stored on the standard environment variables or shared configuration/credential files.

Currently supported backends are: aws.

The AWS auth backend contains three keys: access_key, secret_key and region. When set, Cloudpack will use the given values to authenticate with AWS (including during the Packer Build step). If any of these keys are empty or omitted, Cloudpack assumes the credentials are available on the corresponding environment variables or credentials file.

NOTE: By default, Node.js's AWS SDK does not check the shared configuration file at ~/.aws/config. You can force Cloudpack to use this file by setting the AWS_SDK_LOAD_CONFIG environment variable to a truthy value.

Example

{
  "auth": {
    "aws": {
      "access_key": "myAccessKey",
      "secret_key": "mySecretKey",
      "region": "us-west-2"
    }
  }
}

The builder entry is required and contains the information that should be relayed to Packer during the Packer Build step. Similar to auth, it contains a key that represents the builder, and the corresponding values it needs to properly build the image.

Currently supported builders are: amazon-chroot.

The amazon-chroot backend implements the amazon-chroot builder. The values in this entry, alongside other parts of the Cloudpack configuration file, will be used to generate a valid Packer template, which will then be built.

There is no need to relay the access_key and secret_key entries; those values are inferred from the auth section.

Resources generated on success:

• amiId - ID of the AMI resource generated by Packer.
• snapshotId - ID of the Snapshot resource generated by Packer.

NOTE: amazon-chroot builder requires root privileges.

Example:

{
  "builder": {
    "amazon-chroot": {
      "ami_name": "MyProductionApp-{{ timestamp }}",
      "source_ami": "ami-0d1a13e419ec9285d",
      "nvme_device_path": "/dev/nvme1n1p",
      "device_path": "/dev/sdf",
      "ena_support": true
    }
  }
}

The build_script section contains a list of commands that should be applied to the running Packer VM before it is turned into a static image. They use the same syntax of Packer provisioners.

The build script execution is ordered; that is, it follows the same order declared in the configuration file.

The build_script entry is optional and may be omitted.

Example:

{
  "build_script": [
    {
      "type": "file",
      "source": "~/myapp.tar",
      "destination": "/deploy/myapp.tar"
    },
    {
      "type": "shell",
      "inline": [ "tar -xvf /deploy/myapp.tar" ]
    }
  ]
}

The boot_script section contains a list of commands that should be executed during the boot process.

The configuration must include a type entry, which states how the boot script will be executed. It may be executed either as rc.local, which creates an executable /etc/rc.local entry, or as user-data, which relies on AWS UserData.

Currently only rc.local execution method is implemented.

The user must, then, set the cmds entry, which should contain a list of string commands to be executed. This list will be used to generate a shell script, which will then be used by rc.local or user-data.

The user may also specify the keys flags and shebang, which applies custom flags and shebang to the generated shell script. When omitted, Cloudpack defaults to not setting any flags, and to using the #!/usr/bin/env bash shebang.

The boot_script entry is optional. When omitted, Cloudpack assumes nothing should be executed during boot.

NOTE: The rc.local method works on systemd due to its rc-local.service, which is included by default. This service will execute /etc/rc.local if it is an executable file.

NOTE: The user-data method is currently unsupported.

Example:

{
  "boot_script": {
    "type": "rc.local",
    "flags": "eu",
    "shebang": "#!/bin/sh",
    "cmds": [
      "os=$(uname)",
      "echo \"I love $os\" > /tmp/foo"
    ]
  }
}

Generated shell script:

#!/bin/sh
set -eu


os=$(uname)
echo "I love $os" > /tmp/foo

The launch_template section defines what action Cloudpack should take when the image is generated successfully by Packer.

Similar to auth and builder, launch_template requires the user to specify a backend.

launch_template is optional. When omitted, Cloudpack assumes it should do nothing with the generated image.

Currently supported backends are: aws.

The AWS launch template backend allows the user to create a new LaunchTemplate from scratch, or to update the values of a previously created LaunchTemplate, thus setting a new version for that LaunchTemplate.

When creating a new launch template from scratch, the user must specify the template_name, and template_data keys. The template_data must contain a map of values that should be set on the new template. There must exist at least one entry in this map.

When creating a new version of a previously created launch template, the user must specify the source_template and the modifications key, containing the set of changes that should be applied to that LaunchTemplate.

Moreover, when creating a new version, it is usually desirable to inherit the values of a previous version. Cloudpack supports this by enabling the source_version key, which is a string pointing to the version that should be the basis of the new template.

Resources generated on success:

• launchTemplateId - ID of the LaunchTemplate resource created/updated.

Note: The values within template_data and modifications use the same nomenclature the official AWS API does. Any value in there is relayed to the API. Here's a list of valid parameters.

Example (creating new template):

{
  "launch_template": {
    "aws": {
      "template_name": "MyCoolTemplate",
      "template_data": {
        "ImageId": "{{ runtime.amiId }}"
      }
    }
  }
}

Example (creating new version off of existing one):

{
  "launch_template": {
    "aws": {
      "source_template": "lt-0d058a349ebcab476",
      "source_version": "1",
      "modifications": {
        "ImageId": "{{ runtime.amiId }}"
      }
    }
  }
}

The Cloudpack configuration file supports two types of variables: user variables and runtime variables.

User variables are prefixed with var and must be defined at the CLI, using the --var flag. All user variables defined at the configuration file must be set.

Runtime variables are prefixed with runtime and are set internally by Cloudpack, as soon as the requested runtime variable is resolved. Valid values for runtime variables are:

• runtime.amiId - Contains the ID of the AMI generated by Packer.
• runtime.snapshotId - Contains the ID of the snapshot generated by Packer.
• runtime.launchTemplateId - Contains the ID of the LaunchTemplate created/modified by the AWS LaunchTemplate backend.

NOTE: Runtime variables are only accessible after they are created. For example, runtime.amiId is generated during the Packer Build step, and as such cannot be used prior to that.

Packer template variables, like {{ timestamp }} are also supported, but they only work within the Packer build step. In other words, if you use Packer variables on builder or build_script, they will be expanded properly. Otherwise, they will keep their literal value.

Functional CLI + configuration file example

CLI command:

sudo cloudpack config.json --var "version=1.0"

config.json:

{
  "auth": {
    "aws": {
      "region": "us-west-2"
    }
  },
  "builder": {
    "amazon-chroot": {
      "ami_name": "MyProductionApp-{{ var.version}}-{{ timestamp }}",
      "source_ami": "ami-0d1a13e419ec9285d",
      "nvme_device_path": "/dev/nvme1n1p",
      "device_path": "/dev/sdf",
      "ena_support": true
    }
  },
  "build_script": [
    {
      "type": "shell",
      "environment_vars": [
        "FOO=bar"
      ],
      "inline": [
        "echo \"$FOO\" > /foo"
      ]
    }
  ],
  "boot_script": {
    "type": "rc.local",
    "flags": "eu",
    "cmds": [
      "echo 'Hello World' > /tmp/foo",
      "val=$(cat /tmp/foo)",
      "echo \"value is $val\" > /tmp/foo2"
    ]
  },
  "launch_template": {
    "aws": {
      "source_template": "lt-0d058a349ebcab476",
      "source_version": "1",
      "modifications": {
        "ImageId": "{{ runtime.amiId }}"
      }
    }
  }
}

Output:

{
  "amiId": "ami-032c4edf187ab5ff6",
  "snapshotId": "snap-0cbbd8b0af7805603",
  "launchTemplateId": "lt-0d058a349ebcab476"
}

• Packer supports multiple builders in a single template file. Cloudpack supports only one per configuration file.

• Some values accepted by the Packer build or the AWS API are not currently relayed to the corresponding APIs.