Skip to content

Commit

Permalink
Refactor iso_* settings and adjacent bug-fixes (vmware-samples#93)
Browse files Browse the repository at this point in the history 
- Uses vendor / distributor `.iso` file name and checksum.
- Adds/updates build variables to include `iso_path`, `iso_file`, `iso_checksum_type`, and `iso_checksum_value`.
- Updates code style for the variables passed to a templatefile.
- Replaces the term "redhat-variant" with "redhat-derivative".
- Adds Ansible to Ubuntu 20.04 and 18.06 vmware-samples#54
  • Loading branch information
@tenthirtyam
tenthirtyam committed Oct 3, 2021
1 parent 6d1c38b commit ab4ab97
Show file tree
Hide file tree
Showing 43 changed files with 588 additions and 379 deletions.
126 changes: 67 additions & 59 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -128,49 +128,50 @@ The files are distributed in the following directories.
* **`certificates`** - contains the Trusted Root Authority certificates.
* **`manifests`** - manifests created after the completion of each build.

### Step 2 - Prepare the Guest Operating Systems ISOs
### Step 2 - Download the Guest Operating Systems ISOs

1. Download the x64 guest operating system [.iso][iso] images.

**Linux Distributions**
* VMware Photon OS 4 Server
* [Download][download-linux-photon-server-4] the latest release of the **FULL** `.iso` image.
* [Download][download-linux-photon-server-4] the latest release of the **FULL** `.iso` image. (_e.g._ `photon-4.0-ca7c9e933.iso`)
* Ubuntu Server 20.04 LTS
* [Download][download-linux-ubuntu-server-20-04-lts] the latest **LIVE** release `.iso` image.
* [Download][download-linux-ubuntu-server-20-04-lts] the latest **LIVE** release `.iso` image. (_e.g._ `ubuntu-20.04.2-live-server-amd64.iso`)
* Ubuntu Server 18.04 LTS
* [Download][download-linux-ubuntu-server-18-04-lts] the latest legacy **NON-LIVE** release` .iso` image.
* [Download][download-linux-ubuntu-server-18-04-lts] the latest legacy **NON-LIVE** release `.iso` image. (_e.g._ `ubuntu-18.04.6-server-amd64.iso`)
* Red Hat Enterprise Linux 8 Server
* [Download][download-linux-redhat-server-8] the latest release of the **FULL** (e.g. `RHEL-8-x86_64-dvd1.iso`) `.iso` image.
* [Download][download-linux-redhat-server-8] the latest release of the **FULL** `.iso` image. (_e.g._ `RHEL-8-x86_64-dvd1.iso`)
* AlmaLinux 8 Server
* [Download][download-linux-almalinux-server-8] the latest release of the **FULL** (e.g. `AlmaLinux-8-x86_64-dvd1.iso`) `.iso` image.
* [Download][download-linux-almalinux-server-8] the latest release of the **FULL** `.iso` image. (_e.g._ `AlmaLinux-8-x86_64-dvd1.iso`)
* Rocky Linux 8 Server
* [Download][download-linux-rocky-server-8] the latest release of the **FULL** (e.g. `Rocky-8-x86_64-dvd1.iso`) `.iso` image.
* [Download][download-linux-rocky-server-8] the latest release of the **FULL** `.iso` image. (_e.g._ `Rocky-8-x86_64-dvd1.iso`)
* CentOS Stream 8 Server
* [Download][download-linux-centos-stream-8] the latest release of the **FULL** (e.g. `CentOS-Stream-8-x86_64-dvd1.iso`) `.iso` image.
* [Download][download-linux-centos-stream-8] the latest release of the **FULL** `.iso` image. (_e.g._ `CentOS-Stream-8-x86_64-dvd1.iso`)
* CentOS Linux 8 Server
* [Download][download-linux-centos-server-8] the latest release of the **FULL** (e.g. `CentOS-8-x86_64-dvd1.iso`) `.iso` image.
* [Download][download-linux-centos-server-8] the latest release of the **FULL** `.iso` image. (_e.g._ `CentOS-8-x86_64-dvd1.iso`)

**Microsoft Windows**
* Microsoft Windows Server 2022
* Microsoft Windows Server 2019
* Microsoft Windows Server 2016
* Microsoft Windows 10
* Microsoft Windows 10 Professional

2. Rename your guest operating system `.iso` images. The examples in this repository _generally_ use the format of `iso-family-vendor-type-version.iso`.
3. Obtain the checksum type (_e.g._ `sha256`, `md5`, etc.) and checksum value for each guest operating system `.iso` image. This will be use in the build input variables.

Example: `iso-linux-ubuntu-server-20-04-lts.iso`
4. [Upload][vsphere-upload] your guest operating system `.iso` images to the ISO datastore and paths that will be used in your variables.

3. Obtain the SHA-512 checksum for each guest operating system `.iso` image. This will be use in the build input variables.

Example:

* macOS terminal: `shasum -a 512 [filename.iso]`
* Linux shell: `sha512sum [filename.iso]`
* Windows command: `certutil -hashfile [filename.iso] sha512`

4. [Upload][vsphere-upload] your guest operating system `.iso` images to the datastore and path defined in your common variables.
Example: `builds/<type>/<build>/*.auto.pkvars.hcl`
```
common_iso_datastore = "sfo-w01-cl01-ds-nfs01"
```

Example: `[sfo-w01-ds-nfs01] /iso`.
Example: `config/common.pkvars.hcl`
```
iso_path = "iso/linux/photon"
iso_file = "photon-4.0-ca7c9e933.iso"
iso_checksum_type = "md5"
iso_checksum_value = "d8c4bc561e68afaf7815518f78a5b4ab"
```

### Step 3 - Configure the Variables

Expand Down Expand Up @@ -204,18 +205,25 @@ For example, this is useful for the purposes of running machine image builds for

#### **Build Variables**

Edit the `/config/build.pkvars.hcl` file to configure the following:
Edit the `config/build.pkvars.hcl` file to configure the following:

* Credentials for the default account on machine images.

Example: `/config/build.pkvars.hcl`
Example: `config/build.pkvars.hcl`

```
build_username = "rainpole"
build_password = "<plaintext_password>"
build_password_encrypted = "<sha512_encrypted_password>"
build_key = "<public_key>"
```
You can also override the `build_key` value with contents of a file, if required.

For example:

```
build_key = file("${path.root}/config/ssh/build_id_ecdsa.pub")
```

Generate a SHA-512 encrypted password for the _`build_password_encrypted`_ using various other tools like OpenSSL, mkpasswd, etc.

Expand Down Expand Up @@ -250,36 +258,43 @@ Your identification has been saved in /Users/rainpole/.ssh/id_ecdsa.
Your public key has been saved in /Users/rainpole/.ssh/id_ecdsa.pub.
```

The content of the public key, `build_key`, is added the key to the `.ssh/authorized_keys` file of the `build_username` on the guest operating system.

The content of the public key, `build_key`, is added the key to the `.ssh/authorized_keys` file of the `build_username` on the guest operating system.

>**WARNING**: Replace the default public keys and passwords.
>By default, both Public Key Authentication and Password Authentication are enabled for Linux distributions. If you wish to disable Password Authentication and only use Public Key Authentication, comment or remove the portion of the associated script in the `/scripts` directory.
>By default, both Public Key Authentication and Password Authentication are enabled for Linux distributions. If you wish to disable Password Authentication and only use Public Key Authentication, comment or remove the portion of the associated script in the `scripts` directory.
#### **Ansible Variables**

Edit the `/config/ansible.pkvars.hcl` file to configure the following:
Edit the `config/ansible.pkvars.hcl` file to configure the following:

* Credentials for the Ansible account on Linux machine images.

Example: `/config/ansible.pkvars.hcl`
Example: `config/ansible.pkvars.hcl`

```
ansible_username = "ansible"
ansible_key = "<public_key>"
```
>**NOTE**: A random password is generated for the Ansible user.
You can also override the `ansible_key` value with contents of a file, if required.

For example:

```
build_key = file("${path.root}/config/ssh/ansible_id_ecdsa.pub")
```

#### **Common Variables**

Edit the `/config/common.pkvars.hcl` file to configure the following common variables:
Edit the `config/common.pkvars.hcl` file to configure the following common variables:

* Virtual Machine Settings
* Template and Content Library Settings
* Removable Media Settings
* Boot and Provisioning Settings

Example: `/config/common.pkvars.hcl`
Example: `config/common.pkvars.hcl`

```
// Virtual Machine Settings
Expand All @@ -295,8 +310,6 @@ common_content_library_destroy = true
// Removable Media Settings
common_iso_datastore = "sfo-w01-cl01-ds-nfs01"
common_iso_path = "iso"
common_iso_hash = "sha512"
// Boot and Provisioning Settings
common_data_source = "http"
Expand All @@ -323,12 +336,12 @@ common_http_ip = "172.16.11.254"

#### **Proxy Variables**

Edit the `/config/proxy.pkvars.hcl` file to configure the following:
Edit the `config/proxy.pkvars.hcl` file to configure the following:

* SOCKS proxy settings used for connecting to Linux machine images.
* Credentials for the proxy server (Optional).

Example: `/config/proxy.pkvars.hcl`
Example: `config/proxy.pkvars.hcl`

```
communicator_proxy_host = "proxy.rainpole.io"
Expand All @@ -338,29 +351,27 @@ communicator_proxy_password = "<plaintext_password>"
```
#### **Red Hat Subscription Manager Variables**

Edit the `/config/redhat.pkvars.hcl` file to configure the following:
Edit the `config/redhat.pkvars.hcl` file to configure the following:

* Credentials for your Red Hat Subscription Manager account.

Example: `/config/redhat.pkvars.hcl`
Example: `config/redhat.pkvars.hcl`

```
rhsm_username = "rainpole"
rhsm_password = "<plaintext_password>"
```

These variables are **only** used if you are performing a Red Hat Enterprise Linux Server build to register the image with Red Hat Subscription Manager and run a `sudo yum update -y` within the shell provisioner. Before the build completes, the machine image is unregistered from Red Hat Subscription Manager.


These variables are **only** used if you are performing a Red Hat Enterprise Linux Server build to register the image with Red Hat Subscription Manager and run a `sudo dnf update -y` within the shell provisioner. Before the build completes, the machine image is unregistered from Red Hat Subscription Manager.

#### **vSphere Variables**

Edit the `/buils/vsphere.pkvars.hcl` file to configure the following:
Edit the `builds/vsphere.pkvars.hcl` file to configure the following:

* vSphere Endpoint and Credentials
* vSphere Settings

Example: `/config/vsphere.pkvars.hcl`
Example: `config/vsphere.pkvars.hcl`

```
vsphere_endpoint = "sfo-w01-vc01.sfo.rainpole.io"
Expand All @@ -382,14 +393,16 @@ Edit the `*.auto.pkvars.hcl` file in each `builds/<type>/<build>` folder to conf
* CPU Cores `(init)`
* Memory in MB `(init)`
* Primary Disk in MB `(init)`
* .iso Image File `(string)`
* .iso Image SHA-512 Checksum `(string)`
* .iso Path `(string)`
* .iso File `(string)`
* .iso Checksum Type `(string)`
* .iso Checksum Value `(string)`

>**Note**: All `variables.auto.pkvars.hcl` default to using the the recommended firmware for the guest operating system, the [VMware Paravirtual SCSI controller][vmware-pvscsi] and the [VMXNET 3][vmware-vmxnet3] network card device types.
>**Note**: All `variables.auto.pkvars.hcl` default to using the [VMware Paravirtual SCSI controller][vmware-pvscsi] and the [VMXNET 3][vmware-vmxnet3] network card device types.
#### **Using Environmental Variables**

Some of the variables may include sensitive information and environmental data that you would prefer not to save to clear text files. You can add there to environmental variables using the example below:
Some of the variables may include sensitive information and environmental data that you would prefer not to save to clear text files. You can add these to environmental variables using the example below:

```
export PKR_VAR_ansible_username="<ansible_password>"
Expand All @@ -412,8 +425,8 @@ export PKR_VAR_vsphere_cluster="<vsphere_cluster>"
export PKR_VAR_vsphere_datastore="<vsphere_datastore>>"
export PKR_VAR_vsphere_network="<vsphere_network>"
export PKR_VAR_vsphere_folder="<vsphere_folder>"
```

## Step 4 - Modify the Configurations and Scripts

If required, modify the configuration and scripts files, for the Linux distributions and Microsoft Windows.
Expand All @@ -422,8 +435,6 @@ If required, modify the configuration and scripts files, for the Linux distribut

Username and password variables are passed into the kickstart or cloud-init files for each Linux distribution as Packer template files (`.pkrtpl.hcl`) to generate these on-demand.

A SHA-512 encrypted password for the `root` account and the _`build_username`_ (e.g. `rainpole`). It also adds the _`build_username`_ to the sudoers.

### Microsoft Windows Unattended amd Scripts

Variables are passed into the [Microsoft Windows][microsoft-windows-unattend] unattend files (`autounattend.xml`) as Packer template files (`autounattend.pkrtpl.hcl`) to generate these on-demand.
Expand Down Expand Up @@ -537,23 +548,20 @@ Happy building!!!

[//]: Links

[chocolatey]: https://chocolatey.org/why-chocolatey
[cloud-init]: https://cloudinit.readthedocs.io/en/latest/
[credits-maher-alasfar-twitter]: https://twitter.com/vmwarelab
[credits-maher-alasfar-github]: https://github.com/vmwarelab/cloud-init-scripts
[credits-owen-reynolds-twitter]: https://twitter.com/OVDamn
[credits-owen-reynolds-github]: https://github.com/getvpro/Build-Packer/blob/master/Scripts/Install-VMTools.ps1
[download-git]: https://git-scm.com/downloads
[download-linux-almalinux-server-8]: https://mirrors.almalinux.org/isos.html
[download-linux-centos-server-8]: http:https://isoredirect.centos.org/centos/8/isos/x86_64/
[download-linux-centos-stream-8]: http:https://isoredirect.centos.org/centos/8-stream/isos/x86_64/
[download-linux-photon-server-4]: https://packages.vmware.com/photon/4.0/
[download-linux-ubuntu-server-20-04-lts]: https://releases.ubuntu.com/20.04.1/
[download-linux-ubuntu-server-18-04-lts]: http:https://cdimage.ubuntu.com/ubuntu/releases/18.04.5/release/
[download-linux-redhat-server-8]: https://access.redhat.com/downloads/content/479/
[download-linux-redhat-server-7]: https://access.redhat.com/downloads/content/69/
[download-linux-almalinux-server-8]: https://mirrors.almalinux.org/isos.html
[download-linux-rocky-server-8]: https://download.rockylinux.org/pub/rocky/8/isos/x86_64/
[download-linux-centos-stream-8]: http:https://isoredirect.centos.org/centos/8-stream/isos/x86_64/
[download-linux-centos-server-8]: http:https://isoredirect.centos.org/centos/8/isos/x86_64/
[download-linux-centos-server-7]: http:https://isoredirect.centos.org/centos/7/isos/x86_64/
[download-linux-ubuntu-server-18-04-lts]: http:https://cdimage.ubuntu.com/ubuntu/releases/18.04.5/release/
[download-linux-ubuntu-server-20-04-lts]: https://releases.ubuntu.com/20.04.1/
[hashicorp]: https://www.hashicorp.com/
[iso]: https://en.wikipedia.org/wiki/ISO_image
[microsoft-kms]: https://docs.microsoft.com/en-us/windows-server/get-started/kmsclientkeys
Expand All @@ -565,8 +573,8 @@ Happy building!!!
[packer-install]: https://www.packer.io/intro/getting-started/install.html
[packer-plugin-vsphere]: https://www.packer.io/docs/builders/vsphere/vsphere-iso
[packer-plugin-windows-update]: https://github.com/rgl/packer-plugin-windows-update
[packer-variables]: https://www.packer.io/docs/from-1.5/variables#variable-definitions-pkrvars-hcl-files
[photon-kickstart]: https://vmware.github.io/photon/assets/files/html/3.0/photon_user/kickstart.html
[packer-variables]: https://www.packer.io/docs/templates/hcl_templates/variables
[photon-kickstart]: https://vmware.github.io/photon/docs/user-guide/kickstart-through-http/packer-template/
[redhat-kickstart]: https://access.redhat.com/labs/kickstartconfig/
[ssh-keygen]: https://www.ssh.com/ssh/keygen/
[vmware-pvscsi]: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.hostclient.doc/GUID-7A595885-3EA5-4F18-A6E7-5952BFC341CC.html
Expand Down
2 changes: 1 addition & 1 deletion build.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,7 @@ info() {
echo "License: Apache License Version 2.0."
echo ""
echo "Versions Used:"
echo "HashiCorp Packer >= 1.7.6."
echo "HashiCorp Packer >= 1.7.6"
echo "HashiCorp Packer Plugin for VMware vSphere >= 1.0.1"
echo "HashiCorp Packer Plugin for Windows Update >= 0.14.0"
echo ""
Expand Down
4 changes: 1 addition & 3 deletions builds/common.pkrvars.hcl.example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,11 @@ common_content_library_destroy = true

// Removable Media Settings
common_iso_datastore = "sfo-w01-cl01-ds-nfs01"
common_iso_path = "iso"
common_iso_hash = "sha512"

// Boot and Provisioning Settings
common_data_source = "http"
common_http_ip = null
common_http_port_min = 8000
common_http_port_max = 8099
common_ip_wait_timeout = "20m"
common_shutdown_timeout = "15m"
common_shutdown_timeout = "15m"
8 changes: 5 additions & 3 deletions builds/linux/almalinux-8/linux-almalinux.auto.pkrvars.hcl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,10 @@ vm_disk_thin_provisioned = true
vm_network_card = "vmxnet3"

// Removable Media Settings
iso_file = "iso-linux-almalinux-8.iso"
iso_checksum = "44f56b309fa0cccf41f4d8a5ca3c9b91bb7da510ff8d8faf93fadef9262df872cb824aa7499c5ada5a7ae743be6b38261ed458c465d84e2504e38ca6f05107d7"
iso_path = "iso/linux/rhel-derivative"
iso_file = "AlmaLinux-8.4-x86_64-dvd.iso"
iso_checksum_type = "sha256"
iso_checksum_value = "44f56b309fa0cccf41f4d8a5ca3c9b91bb7da510ff8d8faf93fadef9262df872cb824aa7499c5ada5a7ae743be6b38261ed458c465d84e2504e38ca6f05107d7"

// Boot Settings
vm_boot_order = "disk,cdrom"
Expand All @@ -41,5 +43,5 @@ communicator_port = 22
communicator_timeout = "30m"

// Provisioner Settings
scripts = ["scripts/linux/redhat-variant.sh"]
scripts = ["scripts/linux/redhat-derivative.sh"]
inline = []
32 changes: 21 additions & 11 deletions builds/linux/almalinux-8/linux-almalinux.pkr.hcl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,13 @@ locals {
buildtime = formatdate("YYYY-MM-DD hh:mm ZZZ", timestamp())
path_manifest = "${path.cwd}/manifests/"
data_source_content = {
"/ks.cfg" = templatefile("${abspath(path.root)}/data/ks.pkrtpl.hcl", { build_username = var.build_username, build_password_encrypted = var.build_password_encrypted, vm_guest_os_language = var.vm_guest_os_language, vm_guest_os_keyboard = var.vm_guest_os_keyboard, vm_guest_os_timezone = var.vm_guest_os_timezone })
"/ks.cfg" = templatefile("${abspath(path.root)}/data/ks.pkrtpl.hcl", {
build_username = var.build_username
build_password_encrypted = var.build_password_encrypted
vm_guest_os_language = var.vm_guest_os_language
vm_guest_os_keyboard = var.vm_guest_os_keyboard
vm_guest_os_timezone = var.vm_guest_os_timezone
})
}
data_source_command = var.common_data_source == "http" ? "inst.ks=http:https://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg" : "inst.ks=cdrom:/ks.cfg"
}
Expand Down Expand Up @@ -70,20 +76,24 @@ source "vsphere-iso" "linux-almalinux" {
notes = "Built by HashiCorp Packer on ${local.buildtime}."

// Removable Media Settings
iso_paths = ["[${var.common_iso_datastore}] ${var.common_iso_path}/${var.iso_file}"]
iso_checksum = "${var.common_iso_hash}:${var.iso_checksum}"
iso_paths = ["[${var.common_iso_datastore}] ${var.iso_path}/${var.iso_file}"]
iso_checksum = "${var.iso_checksum_type}:${var.iso_checksum_value}"
http_content = var.common_data_source == "http" ? local.data_source_content : null
cd_content = var.common_data_source == "disk" ? local.data_source_content : null

// Boot and Provisioning Settings
http_ip = var.common_data_source == "http" ? var.common_http_ip : null
http_port_min = var.common_data_source == "http" ? var.common_http_port_min : null
http_port_max = var.common_data_source == "http" ? var.common_http_port_max : null
http_content = var.common_data_source == "http" ? local.data_source_content : null

cd_content = var.common_data_source == "disk" ? local.data_source_content : null

boot_order = var.vm_boot_order
boot_wait = var.vm_boot_wait
boot_command = ["up", "e", "<down><down><end><wait>", "text ${local.data_source_command}", "<enter><wait><leftCtrlOn>x<leftCtrlOff>"]
boot_order = var.vm_boot_order
boot_wait = var.vm_boot_wait
boot_command = [
"up",
"e",
"<down><down><end><wait>",
"text ${local.data_source_command}",
"<enter><wait><leftCtrlOn>x<leftCtrlOff>"
]
ip_wait_timeout = var.common_ip_wait_timeout
shutdown_command = "echo '${var.build_password}' | sudo -S -E shutdown -P now"
shutdown_timeout = var.common_shutdown_timeout
Expand Down Expand Up @@ -134,7 +144,7 @@ build {
}

post-processor "manifest" {
output = "${local.path_manifest}${local.buildtime}-${var.vm_guest_os_family}-${var.vm_guest_os_vendor}.json"
output = "${local.path_manifest}${local.buildtime} ${var.vm_guest_os_family}-${var.vm_guest_os_vendor}.json"
strip_path = false
}
}
Loading

0 comments on commit ab4ab97

Please sign in to comment.