Hive Hive
Sign in

feat(runner-image, xcresult-processor): own macOS+Xcode base image

GitHub issue · Closed

Metadata
Source
tuist/tuist #10834
Updated
Jun 24, 2026
Domains
Compute
Details

Replaces the dependency on Cirrus Labs’ macos-tahoe-xcode:N Tart catalog with our own in-house base image, broken into per-Xcode profiles selectable downstream. Cirrus’s catalog lags Apple’s Xcode releases by weeks; with the CAPI-managed vm-image-builder fleet rotating Mac minis under us, the previous host-bound xcodes signin runbook was about to become unreliable anyway.

Part 1 — own the macOS+Xcode base image (Layer 1)

New Layer 1: ghcr.io/tuist/macos-tahoe-xcode:<xcode-version-dashes>

  • infra/macos-xcode-image/macos-xcode.pkr.hcl — Packer template that layers Xcode + dev tools (xcbeautify, swiftformat, swiftlint, swiftgen, mint, carthage, fastlane, cocoapods, libimobiledevice, ideviceinstaller, ios-deploy) + Apple WWDR + DeveloperID certs on top of ghcr.io/cirruslabs/macos-tahoe-base:latest. Tuist CLI intentionally not preinstalled — customer workflows pin their own version.
  • .github/workflows/macos-xcode-image.ymlworkflow_dispatch triggered with xcode_version as input. Pulls the .xip from our in-house mirror (see Part 2), Packer-builds the VM, pushes it via tart push to GHCR. Outer 3x retry on the push since blob uploads are content-addressed.
  • infra/macos-xcode-image/AGENTS.md — Architecture diagram + Xcode-promotion runbook + RSS-subscription runbook.

Tag scheme (per Namespace-style profile model — one Xcode per image, profile-selectable downstream):

xcode_version Push tag Bundle path Alias
26.5 :26-5 /Applications/Xcode_26.5.app (none)
26.4.1 :26-4-1 /Applications/Xcode_26.4.1.app Xcode_26.4.appXcode_26.4.1.app
26.0.1 :26-0-1 /Applications/Xcode_26.0.1.app Xcode_26.0.appXcode_26.0.1.app

Slimmed Layer 2: runner-image + xcresult-processor-image

Both now inherit from Layer 1 — they drop the Xcode install, dev tools, and WWDR cert provisioning that previously lived in each Packer file. Layer 2 rebuilds on every commit now cost ~2 min instead of ~30 min.

  • Runner image push tags shift to per-Xcode profile: :macos-<xcode-version-dashes>-<semver> (immutable, for rollback) + :macos-<xcode-version-dashes> (rolling, what the chart digest pin tracks).
  • xcresult-processor keeps :<semver> + :latest tagging (internal image tied to server release).
  • Both release jobs derive the Layer 1 base from per-image XCODE_VERSION pin files at infra/runner-image/XCODE_VERSION / infra/xcresult-processor-image/XCODE_VERSION — each path is under its image’s release-include-path so check-releases triggers on Xcode bumps.

Part 2 — the xcode-xips mirror, populated by a local maintainer task

The Layer 1 workflow does not talk to Apple. It pulls .xips from ghcr.io/tuist/xcode-xips:<version> — an in-house mirror populated by a mise run xcode-mirror:upload <version> task that runs on a maintainer’s Mac. Eliminates the keychain-on-host runbook that broke once vm-image-builder Mac minis moved into the CAPI-managed fleet.

Why this and not an in-cluster worker that mirrors automatically: Apple migrated /appleauth/auth/signin to SRP (the client never sends plaintext password; computes an SRP proof against a server-issued salt). Implementing SRP in bash isn’t viable, app-specific passwords don’t authenticate against developer.apple.com, and xcodes keeps its post-2FA cookies in HTTPCookieStorage.shared per-process — there’s nothing to persist into a 1Password Secret that the cluster could replay. So we run the auth dance locally, where 2FA-on-trusted-device works naturally, and lean on xcodereleases.com’s RSS feed for the “new Xcode just shipped” notification:

/feed subscribe https://xcodereleases.com/api/all.rss

in any Slack channel. Operator runs mise run xcode-mirror:upload <version> in response, ~6x/year.

The upload task:

  • mise run xcode-mirror:upload <version>xcodes downloads the .xip (handles SRP + 2FA; session is cached in the local keychain ~30 days), oras push uploads to ghcr.io/tuist/xcode-xips:<version>. Tools come from the repo-root mise.toml (oras, xcodes, jq, gh).

Builder host operations (vm-image-builder-NN)

The macos-xcode-image.yml workflow assumes a host with the following manual setup. Until the fleet is owned by cluster-api-provider-scaleway-applesilicon, this is the bootstrap checklist for each new Scaleway Apple Silicon Mac mini that joins the vm-image-builder label set:

  1. Install Homebrew, then brew install tart aria2 gh xcodes oras hashicorp/tap/packer (plus brew tap cirruslabs/cli for tart).
  2. Enable auto-login for the runner user so the box comes up with a real GUI session (System Settings → Users & Groups → Automatically log in as → user, then reboot). Without this, Apple’s Virtualization Framework refuses to create VM host keys (“Failed to create new HostKey”) because it can’t access the Secure Enclave from a session-less LaunchDaemon context. FileVault must be off.
  3. Register the GitHub Actions runner as a user-level LaunchAgent (the default svc.sh install path). Auto-login from step 2 makes the agent load on boot.
  4. Trigger one tart VM run interactively. macOS prompts once for “Local Network” permission (Sequoia/Tahoe’s privacy gate on the vmnet framework — without this grant, bridge100 never comes up and Packer’s SSH-to-VM step times out). Grant via System Settings → Privacy & Security → Local Network. The permission persists across reboots.

A first-class config-management owner for this checklist (k8s + tart-cri + a CR per host, or an Ansible playbook, or a Packer-built host image) is intentionally out of scope here. The current vm-image-builder-01 is set up by hand and the steps above are the runbook for the next one until that lands.

Bootstrap order

  1. One-time: Subscribe xcodereleases.com/api/all.rss in the infra-ops Slack channel.
  2. First populate: maintainer runs mise run xcode-mirror:upload <version> for each Xcode we want available (26.5, 26.4.1, …, the rolling Tahoe set).
  3. Trigger macos-xcode-image.yml to build Layer 1 images for each profile.
  4. First commit on main after merge triggers release-runner-image / release-xcresult-processor-image which build against the Layer 1 base.

Test plan

  • packer validate -syntax-only clean on all three .pkr.hcl files (macos-xcode-image, runner-image, xcresult-processor-image).
  • actionlint clean on all four workflows (only pre-existing self-hosted runner-label warnings remain).
  • mix compile --warnings-as-errors clean.
  • helm template tuist infra/helm/tuist -f values-managed-common.yaml -f values-managed-production.yaml renders without error; no stale TUIST_XCODE_MIRROR_* env vars or xcode-mirror-external-secrets ExternalSecret left behind.
  • mise run xcode-mirror:upload 26.5 validated end-to-end from a maintainer Mac during PR development — xcodes SRP signin + 2FA prompt works, .xip lands in ghcr.io/tuist/xcode-xips:26.5.
  • macos-xcode-image.yml dispatched against a fresh vm-image-builder-01 host with xcode_version=26.5: Packer build (17:40) + tart push to GHCR (10:31) = ~28 min total, succeeded on first attempt, ghcr.io/tuist/macos-tahoe-xcode:26-5 published.

What this PR ultimately ships

  • The Layer 1 / Layer 2 split — every macOS image rebuild gets ~30 min faster, owns its own Xcode catalog independent of Cirrus.
  • A single mise task for refreshing the .xip mirror, with the toolchain pinned in the root mise.toml.
  • A documented host-bootstrap checklist for vm-image-builder Mac minis, valid until the CAPI provider owns these declaratively.
  • Documentation pointing at the RSS feed as the “new Xcode shipped” notification surface.

No new in-cluster service, no Apple credentials in 1Password / External Secrets, no quarterly cookie-refresh runbook. The earlier draft of this PR introduced a Tuist.XcodeMirror Oban worker for in-cluster automation; the SRP migration and xcodes’ lack of cookie persistence made that path effectively impossible, so it’s been removed. The remaining work is small enough that the previous Phase 1/2/3 framing no longer applies.

🤖 Generated with Claude Code

Comments
TA
tuist-atlas[bot] Jun 5, 2026

The changes from this PR are now available in release xcresult-processor-image@0.11.0. The xcresult-processor-image has been slimmed as Layer 2, now inheriting from the new Layer 1 base image, reducing rebuild times from ~30 min to ~2 min per commit.

TA
tuist-atlas[bot] Jun 5, 2026

The macOS+Xcode base image feature is now available in runners-controller@0.8.0. Update to this version to use the new Layer 1/Layer 2 image split and in-house Xcode catalog.