Deployment · Amazon EC2 Mac

Two macOS guests on every Dedicated Host you already pay for

An EC2 Mac Dedicated Host is one Mac, billed for a minimum of 24 hours whether you use it or not. Apple's licence permits two macOS virtual machines on that host. This is how you get them, and how to make the networking behave behind a Nitro ENI.

The host needs macOS 27 or later on an Apple silicon family. Check MacOSLatestSupportedVersions before you allocate.

Why bother

EC2 Mac bills per Dedicated Host, not per instance, and there is a one-to-one mapping between a host and the instance on it, you cannot slice a Mac host the way you slice a Linux one. Apple's software licence agreement also imposes a 24-hour minimum allocation, and that clock starts when the host is allocated, not when you launch an instance.

So the unit of cost is a whole Mac for at least a day. Running one build agent on it leaves the licence's second and third macOS guests unused. Spooktacular turns that host into a small pool: the host itself, plus two macOS VMs that reset to a clean base in seconds.

The limit is Apple's, and it is enforced macOS permits two virtual machines per host. Spooktacular refuses to start a third rather than letting you discover the limit in the middle of a build. Linux guests are unlimited and are not covered by that clause, so a mixed fleet of two macOS VMs plus Linux runners on one host is legitimate.

Try it locally first

Do not let a 24-hour host allocation be the place you learn the tool. Everything except the instance metadata works identically on a Mac on your desk, and the first run is the slow one either way, so rehearse locally:

$git clone https://github.com/Spooky-Labs/spooktacular.git && cd spooktacular
$./build-app.sh release && sudo spook create dev

That builds the shared base image once, 10 to 20 minutes. And then every later create takes seconds. Once you have seen spook create, spook start, spook ip and spook list behave on your own machine, the EC2 path below is the same commands with the install moved into user data. Full local walkthrough: Get started.

Choosing an instance

Spooktacular needs Apple silicon, so the mac1 family is out, those are Intel Mac minis. Use an Apple silicon family, and pick on memory, because memory is what caps how many guests you can run comfortably:

FamilySilicon CPUMemory Suits
mac2M18 cores16 GiBTight for two macOS guests; fine for one plus Linux
mac2-m2M28 cores24 GiBTwo modest guests
mac2-m2proM2 Pro12 cores32 GiBA comfortable two-guest runner host
mac-m4M410 cores24 GiBFaster builds at mac2-m2 memory
mac-m4proM4 Pro14 cores48 GiBBest fit, and a 2 TB instance store per host
mac2-m1ultraM1 Ultra20 cores128 GiBMemory-heavy guests

Check the host's macOS support before you allocate. macOS guests need macOS 27 on the host, because the base-and-overlay disk format uses DiskImageKit. AWS exposes what each host can run:

$aws ec2 describe-mac-hosts --query 'MacHosts[].{Host:HostId,macOS:MacOSLatestSupportedVersions}'
mac-m4pro has a card worth playing The M4 Pro family ships a 2 TB instance store volume per Dedicated Host. That is the right home for the base-image cache and for VM overlays: it is local storage rather than network-attached EBS, so the first base build and every subsequent boot get lower latency. It is ephemeral, which suits a cache exactly.

Allocating the host

An EC2 Mac instance only runs on a Dedicated Host, so allocate one first:

$aws ec2 allocate-hosts --instance-type mac-m4pro.metal --availability-zone us-east-1a --auto-placement on --quantity 1

Note the returned host ID. The 24-hour clock starts now. Launch onto it, into the subnet you allocated the host in:

$aws ec2 run-instances --instance-type mac-m4pro.metal --placement HostId=h-0123456789abcdef0 --image-id ami-<macos-27-arm64> --subnet-id subnet-abc123 --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=500,VolumeType=gp3,Iops=5000,Throughput=200}' --user-data file://install-spooktacular.sh
Do not skimp on the EBS volume APFS is the default macOS filesystem and is tuned for SSDs, so AWS recommends an SSD-backed volume, gp3 or io2, and suggests starting at 5000 IOPS and 200 MB/s throughput. Spooktacular is disk-heavy in exactly the way that matters here: a base image is roughly 40 GB, every overlay grows from near zero, and a warm-pool reset is a file operation. An undersized volume shows up as slow boots, not as an error.

Sizing the volume

This is the step people miss. Asking EC2 for a 500 GB root volume does not give macOS 500 GB: the AMI's APFS container stays its original size, and the extra space sits unallocated until you grow the container. A 40 GB base image plus overlays will fill the default and fail in confusing ways.

Grow it on first boot, before anything else runs:

expand the APFS container
PDISK=$(diskutil list physical external | head -n1 | cut -d" " -f1)
APFSCONT=$(diskutil list physical external | grep "Apple_APFS" | tr -s " " | cut -d" " -f8)
yes | diskutil repairDisk "$PDISK"
diskutil apfs resizeContainer "$APFSCONT" 0

0 means "use all available space". repairDisk is what makes the newly visible space usable; skipping it is why the resize sometimes appears to do nothing.

Installing from user data

User data runs as root on first boot, which is exactly the privilege the one privileged step needs. Writing the provisioner into the shared base image. Put the whole install there and the host comes up ready, with no interactive step and no sudo prompt to answer.

install-spooktacular.sh, EC2 user data
#!/bin/zsh
set -euo pipefail

# 1. Give macOS the whole EBS volume.
PDISK=$(diskutil list physical external | head -n1 | cut -d" " -f1)
APFSCONT=$(diskutil list physical external | grep "Apple_APFS" | tr -s " " | cut -d" " -f8)
yes | diskutil repairDisk "$PDISK"
diskutil apfs resizeContainer "$APFSCONT" 0

# 2. Build Spooktacular. Homebrew on Apple silicon lives in /opt.
su ec2-user -c '/opt/homebrew/bin/brew update'
su ec2-user -c 'git clone https://github.com/Spooky-Labs/spooktacular.git ~/spooktacular'
su ec2-user -c 'cd ~/spooktacular && ./build-app.sh release'
ln -sf /Users/ec2-user/spooktacular/Spooktacular.app/Contents/MacOS/spook /usr/local/bin/spook

# 3. Runner credentials go in the SYSTEM keychain: this service is root,
#    and root cannot read a user's login keychain.
security add-generic-password -s com.spooktacular.github -a my-org \
  -w "$(aws secretsmanager get-secret-value --secret-id github-pat --query SecretString --output text)" \
  -U /Library/Keychains/System.keychain

# 4. Build the shared base image once. This is the slow step, and the only
#    privileged one — it happens here, at launch, not on someone's first job.
spook create warm-01 --no-start

✓ host ready: base image built, one VM staged
Why the System keychain, specifically A personal access token is the only accepted source for runner registration: never a flag, an environment variable, or a file on disk. On a workstation it lives in your login keychain. On EC2 Mac, where Spooktacular runs as a root service, root cannot read a user's login keychain, so it must be the System keychain. Getting this wrong produces a runner that never registers and an error that looks like a missing token.

Pull the token from Secrets Manager or SSM Parameter Store as above rather than baking it into user data. User data is readable from the instance metadata service by anything running on the host.

Networking that works behind a Nitro ENI

This is where EC2 Mac differs most from a Mac on a desk, and where a virtualization tool most easily goes wrong.

An EC2 instance reaches the VPC through a Nitro elastic network interface. That interface is bound to the addresses and MAC that EC2 assigned it. A guest VM in bridged mode puts its own MAC address on the wire, which the VPC does not know about, so bridged guests are the wrong model on EC2 even when they appear to come up. Do not reach for it.

Spooktacular's default is the model that does work: every VM gets its own private subnet with a DHCP-reserved address behind host NAT, and anything the guest should expose is published onto a host port:

$spook create web --publish 8080:80 --publish 2222:22

Outbound traffic from a guest leaves through the host's ENI, so it inherits the instance's route table, NAT gateway and VPC endpoints with nothing extra to configure. Inbound reaches a guest only through a published port, which means the thing you already use to control access, the security group on the host, controls access to your guests too. There is no second, VM-shaped firewall to keep in sync.

Consequences worth designing around
  • Two guests cannot publish the same host port. That fails immediately and loudly rather than silently picking a winner.
  • Guests are not directly addressable from elsewhere in the VPC. If a peer service must reach a guest, it reaches the host's private address on the published port.
  • spook ip returns the guest's address on its private subnet. It is a metadata read, the address is reserved when the VM is created, so orchestration never has to poll for it.

Wiring up runners

With the base image built at launch, a runner is one command. A short-lived registration token is minted from the Keychain-stored PAT seconds before the guest boots, so no long-lived credential ever reaches the VM:

$spook create runner-01 --github-runner --github-repo my-org/my-repo --github-token-keychain my-org --ephemeral

--ephemeral means the runner accepts one job and exits. Pair it with a reset and each job starts from a base image that has never run anyone else's code. Which is the property that makes a shared Mac host defensible for CI in the first place.

Resetting between jobs

A reset discards the VM's overlay and puts a fresh one back on the untouched base. The base image is sealed read-only and is never booted, so its layer identity cannot drift, which means a pool can prove its base is pristine rather than assume it.

$spook delete runner-01 --force && spook create runner-01 --github-runner --github-repo my-org/my-repo --github-token-keychain my-org --ephemeral

Because the create is an overlay layer plus two APFS clones, this costs seconds, not another macOS install. Cloning a 5 GB VM measures at 30 ms and consumes no additional disk.

EC2-specific gotchas

Apple silicon hosts allow one bootable volume

AWS documents that Apple silicon Mac instances must have only one bootable volume, and that each attached volume supports only one additional admin user. Plan on a single root volume and keep extra capacity as data volumes.

SIP is per-volume, and slow to change

On Apple silicon Mac instances, System Integrity Protection settings apply at the volume level, not the instance level, so a newly attached root volume does not inherit them. Changing SIP can take up to 90 minutes, during which the instance is unreachable, and the setting does not carry into AMIs or snapshots you create afterwards. Spooktacular does not require SIP changes: this matters only if something else in your image does.

The 24-hour clock is on the host, not the instance

Stopping or terminating the instance does not release the host, and you cannot release the host until 24 hours after allocation. Automate release explicitly, or you will pay for idle Macs.

Homebrew is not where you expect

On Apple silicon it lives at /opt/homebrew/bin/brew, not /usr/local/bin/brew. Scripts copied from Intel-era EC2 Mac guides fail on this line first.

Screen Sharing needs a password set

The ec2-user account has no password initially, so set one before enabling Screen Sharing on the host. Guests are separate: a --remote-desktop VM provisions its own admin account and enables Screen Sharing itself on first boot, reachable through a published port.