Internally, we use this tooling to support the testing of our software on macOS, and the Nix installer itself.
This repository makes many assumptions about your workflow and how you want to use this code. These assumptions are a byproduct of the repository only being used internally, and are likely not difficult to remove. If you use this code and documentation for yourself, consider sending contributions upstream that make it easier for people to use.
Set up macOS machines to automatically erase and provision themselves on a Tailscale network with Buildkite. An erase/reinstall cycle can complete in less than 10 minutes, making it suitable for regular automation.
This README and tooling is public documentation for Determinate Systems, Inc.'s internal use. The goal of making it public is to share the information, and foster the use of ephemeral macOS machines running Nix.
- We assume you are using recent Macs with either a T2 chip or Apple Silicon.
- You're using Mosyle MDM. Other MDMs might work, but we're focused on Mosyle. Feel free to send pull requests supporting other MDMs.
- Your Macs are already part of your Apple Business Manager account. Once you have an Apple Business Manager account, they can provide documentation on adding existing Macs.
- A USB-A Logitech Unifying Receiver to act as a mouse and keyboard. The Logitech receiver doesn't need to be configured or paired. Note that macOS is very paricular about the mouse and keyboard hardware directly after erasing. The TinyPilot KVM was not recognized as a mouse. Using a USB hub between the mouse and the computer didn't work either. A cheap Targus mouse's dongle worked as well.
- A USB-C thumb disk formatted and named "CONFIG". SSH keys and other persistent state is stored here.
- A "Dummy" HMDI plug to convince macOS to stay alive. A TinyPilot KVM works as well. Any display should be fine.
- A TRRS 3.5mm male audio jack to disable built-in speakers and microphones. Optional.
- 2x Thunderbolt 4 / USB-C
- 1x 1Gbase-T Ethernet (10Gbase-T optional)
- 1x HDMI
- 2x USB-A
- 1x 3.5mm headphone jack
- 4x Thunderbolt 3 / USB-C
- 1x 1Gbase-T Ethernet (10Gbase-T optional)
- 1x HDMI
- 2x USB-A
- 1x 3.5mm headphone jack
Front:
- 2x Thunderbolt 4 / USB-C
- 1x SDXC
Back:
- 4x Thunderbolt 4 / USB-C
- 1x 10Gbase-T Ethernet
- 1x HDMI
- 2x USB-A
- 1x 3.5mm headphone jack
Select the device in Management
,
then Devices Overview
,
then select the More
menu.
Click Erase device
.
Change Obliteration Behavior
to Do not Obliterate
.
This requires a T2 or Apple Silicon chip.
See "ObliterationBehavior" on https://developer.apple.com/documentation/devicemanagement/erasedevicecommand/command/.
After erasing, the machine should fully boot and configure itself without any human interaction. The main tasks here are to configure the region, language, and the initial user account.
On the Organization
tab,
select Apple Basic Setup
,
select Enrollment
,
click Automated Device Enrollment
to get to the Device Enrollment (DEP)
page.
Click your default profile.
- Tick
If enabled, macOS will automatically advance through all Setup Assistant screens. Available for macOS 11+ when connected to Ethernet.
- Select your language and region
- Untick
Prompt user to create an account
- Move on to
Create additional local admin during Setup Assistant
- Enter a full name and use
ephemeraladmin
for the username. Note that other pieces of this system depends on the user being namedephemeraladmin
. - Change the
Password
dropdown to automatically generate a password for each device - Tick
Set this account to be managed.
- Set
Rename devices after enrollment
tomac-ephemeral-%SerialNumber%
Click Save
.
The described automation is applied to specific machines through Device Groups
.
On the Management
tab,
on the left side under Devices
,
select Device Groups
,
click Add Device Group
.
- Name it
Ephemeral CI
- Add your macs to the group
Click Save
.
In general, software updates should be applied quickly and without any user interaction. I want to be able to forget this machine exists after setup, so we have fully automated the update process.
On the Management
tab,
on the left side under Management Profiles
,
select Software Update
,
click Add new profile
.
If the profile type isn't there,
click Activate New Profile Type
,
search for it by name,
click Activate
,
then click Add new profile
.
- Name the profile
Automatic Updates
- All of the defaults are fine as is
Under Profile Assignment
,
click + Add Assignment
,
select Devices from specific Devices Group
,
tick Ephemeral CI
.
Click Save
.
If the machine sleeps it is generally not easy to wake it back up.
On my Mac Studio, waking it back up requires physically pressing the Power
button on the back.
I tried using a wireless mouse and a KVM, but neither were able to replace it.
This profile disables sleeping.
On the Management
tab,
on the left side under Management Profiles
,
select Energy Saver
,
click Add new profile
.
If the profile type isn't there,
click Activate New Profile Type
,
search for it by name,
click Activate
,
then click Add new profile
.
- Name the profile
Don't sleep
- Select the
Desktop
profile tab - Set
Put the display(s) to sleep after:
to2 minutes
- Set
Put the computer to sleep after:
toNever
- Set
Put the hard disk(s) to sleep after
toDo not configure this option
- Under
Wake options
, tickWake for Ethernet network administrator access
- Under
Other options
, tickStart up automatically after a power failure
Under Profile Assignment
,
click + Add Assignment
,
select Devices from specific Devices Group
,
tick Ephemeral CI
.
Click Save
.
Our provisioning script uses SSH keys stored on an external volume to survive wipes. Apple widely prohibits programs from reading removable storage. This means Mosyle MDM agent cannot access removable media out of the box.
This profile allows Mosyle to access removable storage.
Note that we don't actually enable anything in this profile except a single checkbox for the Self-Service app. That is intentional: that tickbox is all we need.
On the Management
tab,
on the left side under Management Profiles
,
select Security & Privacy
,
near the top of the screen select the Privacy
tab
click Add new profile
.
If the profile type isn't there,
click Activate New Profile Type
,
search for it by name,
click Activate
,
then click Add new profile
.
- Name the profile
Allow Mosyle access to Removable Volumes
- Tick
Install the Privacy Preferences Policy Control settings for the Mosyle Self-Service app to allow access to all necessary files and application data.
Under Profile Assignment
,
click + Add Assignment
,
select Devices from specific Devices Group
,
tick Ephemeral CI
.
Click Save
.
Autologin is necessary to allow fast erases and reprovisions.
Modern macOS software and hardware has two erase modes: "Erase All Content and Settings" (EACS) and "Obliterate". EACS takes approximately 5 minutes and involves a brief reboot after clearing the existing content and settings. Obliterate completely erases the disk and then rewrites the operating system, annd can take up to several hours. Obliterate is the only option on older hardware.
EACS is the preferred method of implementing an ephemeral macOS machine because of the fast cycle time. In order for EACS to work, the machine must have a "Bootstrap Token" escrowed with our MDM server. The only way to escrow a bootstrap token is to have an administrative user log in.
This profile creates an administrative user with a random, unknown password, and causes it to automatically log in. After creating the user, the machine is rebooted to cause the login to happen.
On the Management
tab,
on the left side under Management Profiles
,
select Custom Commands
,
click Add new profile
.
If the profile type isn't there,
click Activate New Profile Type
,
search for it by name,
click Activate
,
then click Add new profile
.
- Name the profile
Autologin as CI
- Select the
Code
profile tab - Click the code text box
- Paste the contents of
auto-login.sh
into the box - Click the checkmark in the top right of the Code Edit window
- Select the
Execution Settings
profile tab - For
Execute Command
selectOnly based on schedule or events
- For
Event
tickUpon Enrollment Only
Under Profile Assignment
,
click + Add Assignment
,
select Devices from specific Devices Group
,
tick Ephemeral CI
.
Click Save
.
Configure SSH keys and start the SSH daemon for the DEP-managed administrative user, ephemeraladmin
.
This script runs very frequently to ensure SSH is both running, and your users' keys are on the machine.
On the Management
tab,
on the left side under Management Profiles
,
select Custom Commands
,
click Add new profile
.
If the profile type isn't there,
click Activate New Profile Type
,
search for it by name,
click Activate
,
then click Add new profile
.
- Name the profile
Setup SSH
- Select the
Code
profile tab - Click the code text box
- Paste the contents of
setup-ssh.sh
into the box - Edit the list of GitHub user names near line 9 to match your own users
- Click the checkmark in the top right of the Code Edit window
- Select the
Execution Settings
profile tab - For
Execute Command
selectOnly based on schedule or events
- For
Event
untickUpon Enrollment Only
- For
Event
tickEvery start up of the Mac
,Every user sign-in
, andEvery "Device Info Update"
.
Under Profile Assignment
,
click + Add Assignment
,
select Devices from specific Devices Group
,
tick Ephemeral CI
.
Click Save
.
Installs Nix and nix-darwin, which is configured to run a Buildkite agent and join our Tailscale network.]
Note that right now this code assumes you're installing everything for DetSys purposes. It is an explicit goal for this repository to support configuring things for your purposes without necessarily having to fork the repo. Please open issues discussing or send PRs improving this.
First configure a tag to assign to ephemeral macs, by adding this to your Tailscale ACL:
"tagOwners": {
"tag:ephemeral-mac-ci": ["[email protected]"],
}
The actual acquisition of pre-auth tokens is done through Vault on our systems (see setup-vault.sh
).
Save the buildkite agent token into /Volumes/CONFIG/buildkite.token
.
On the Management
tab,
on the left side under Management Profiles
,
select Custom Commands
,
click Add new profile
.
If the profile type isn't there,
click Activate New Profile Type
,
search for it by name,
click Activate
,
then click Add new profile
.
- Name the profile
Install Nix
- Select the
Code
profile tab - Click the code text box
- Paste the contents of
install-nix-fetcher.sh
into the box - Edit the last lines (
repo
,branch
,cfgpath
) to point to your repository and configuration. Note you can use Mosyle's tags and variables to do dynamic configuration dispatch. See the end for an example. - Click the checkmark in the top right of the Code Edit window
- Select the
Execution Settings
profile tab - For
Execute Command
selectOnly based on schedule or events
- For
Event
untickUpon Enrollment Only
- For
Event
tickEvery user sign-in
Under Profile Assignment
,
click + Add Assignment
,
select Devices from specific Devices Group
,
tick Ephemeral CI
.
Click Save
.
Shows the public key of the private key generated on the box.
On the Management
tab,
on the left side under Management Profiles
,
select Custom Commands
,
click Add new profile
.
If the profile type isn't there,
click Activate New Profile Type
,
search for it by name,
click Activate
,
then click Add new profile
.
- Name the profile
Show Public SSH Key
- Select the
Code
profile tab - Click the code text box
- Paste
cat /Volumes/CONFIG/buildkite-agent/sshkey.pub
into the box - Click the checkmark in the top right of the Code Edit window
- Select the
Execution Settings
profile tab - For
Execute Command
selectOnly based on schedule or events
- For
Event
untickUpon Enrollment Only
- For
Event
tickEvery start up of the Mac
- For
Event
tickEvery user sign-in
- For
Event
tickEvery "Device info" update"
Under Profile Assignment
,
click + Add Assignment
,
select Devices from specific Devices Group
,
tick Ephemeral CI
.
Click Save
.
repo="https://github.com/DeterminateSystems/macos-ephemeral.git"
branch="HEAD"
cfgpath="config.nix"
if (echo "%Tags%" | grep -q "beta"); then
branch="beta"
cfgpath="configuration.nix"
fi