Hive Hive
Sign in

feat(runner-image, xcresult-processor): own macOS+Xcode base image

GitHub issue · Closed

Metadata
Source
tuist/tuist #10834
Updated
Jun 24, 2026
Domains
Compute
Details

Replaces the dependency on Cirrus Labs’ macos-tahoe-xcode:N Tart catalog with our own in-house base image, broken into per-Xcode profiles selectable downstream. Cirrus’s catalog lags Apple’s Xcode releases by weeks; with the CAPI-managed vm-image-builder fleet rotating Mac minis under us, the previous host-bound xcodes signin runbook was about to become unreliable anyway.

Part 1 — own the macOS+Xcode base image (Layer 1)

New Layer 1: ghcr.io/tuist/macos-tahoe-xcode:<xcode-version-dashes>

  • infra/macos-xcode-image/macos-xcode.pkr.hcl — Packer template that layers Xcode + dev tools (xcbeautify, swiftformat, swiftlint, swiftgen, mint, carthage, fastlane, cocoapods, libimobiledevice, ideviceinstaller, ios-deploy) + Apple WWDR + DeveloperID certs on top of ghcr.io/cirruslabs/macos-tahoe-base:latest. Tuist CLI intentionally not preinstalled — customer workflows pin their own version.
  • .github/workflows/macos-xcode-image.ymlworkflow_dispatch triggered with xcode_version as input. Pulls the .xip from our in-house mirror (see Part 2), Packer-builds the VM, pushes it via tart push to GHCR. Outer 3x retry on the push since blob uploads are content-addressed.
  • infra/macos-xcode-image/AGENTS.md — Architecture diagram + Xcode-promotion runbook + RSS-subscription runbook.

Tag scheme (per Namespace-style profile model — one Xcode per image, profile-selectable downstream):

xcode_version Push tag Bundle path Alias
26.5 :26-5 /Applications/Xcode_26.5.app (none)
26.4.1 :26-4-1 /Applications/Xcode_26.4.1.app Xcode_26.4.appXcode_26.4.1.app
26.0.1 :26-0-1 /Applications/Xcode_26.0.1.app Xcode_26.0.appXcode_26.0.1.app

Slimmed Layer 2: runner-image + xcresult-processor-image

Both now inherit from Layer 1 — they drop the Xcode install, dev tools, and WWDR cert provisioning that previously lived in each Packer file. Layer 2 rebuilds on every commit now cost ~2 min instead of ~30 min.

  • Runner image push tags shift to per-Xcode profile: :macos-<xcode-version-dashes>-<semver> (immutable, for rollback) + :macos-<xcode-version-dashes> (rolling, what the chart digest pin tracks).
  • xcresult-processor keeps :<semver> + :latest tagging (internal image tied to server release).
  • Both release jobs derive the Layer 1 base from per-image XCODE_VERSION pin files at infra/runner-image/XCODE_VERSION / infra/xcresult-processor-image/XCODE_VERSION — each path is under its image’s release-include-path so check-releases triggers on Xcode bumps.

Part 2 — the xcode-xips mirror, populated by a local maintainer task

The Layer 1 workflow does not talk to Apple. It pulls .xips from ghcr.io/tuist/xcode-xips:<version> — an in-house mirror populated by a mise run xcode-mirror:upload <version> task that runs on a maintainer’s Mac. Eliminates the keychain-on-host runbook that broke once vm-image-builder Mac minis moved into the CAPI-managed fleet.

Why this and not an in-cluster worker that mirrors automatically: Apple migrated /appleauth/auth/signin to SRP (the client never sends plaintext password; computes an SRP proof against a server-issued salt). Implementing SRP in bash isn’t viable, app-specific passwords don’t authenticate against developer.apple.com, and xcodes keeps its post-2FA cookies in HTTPCookieStorage.shared per-process — there’s nothing to persist into a 1Password Secret that the cluster could replay. So we run the auth dance locally, where 2FA-on-trusted-device works naturally, and lean on xcodereleases.com’s RSS feed for the “new Xcode just shipped” notification:

/feed subscribe https://xcodereleases.com/api/all.rss

in any Slack channel. Operator runs mise run xcode-mirror:upload <version> in response, ~6x/year.

The upload task:

  • mise run xcode-mirror:upload <version>xcodes downloads the .xip (handles SRP + 2FA; session is cached in the local keychain ~30 days), oras push uploads to ghcr.io/tuist/xcode-xips:<version>. Tools come from the repo-root mise.toml (oras, xcodes, jq, gh).

Builder host operations (vm-image-builder-NN)

The macos-xcode-image.yml workflow assumes a host with the following manual setup. Until the fleet is owned by cluster-api-provider-scaleway-applesilicon, this is the bootstrap checklist for each new Scaleway Apple Silicon Mac mini that joins the vm-image-builder label set:

  1. Install Homebrew, then brew install tart aria2 gh xcodes oras hashicorp/tap/packer (plus brew tap cirruslabs/cli for tart).
  2. Enable auto-login for the runner user so the box comes up with a real GUI session (System Settings → Users & Groups → Automatically log in as → user, then reboot). Without this, Apple’s Virtualization Framework refuses to create VM host keys (“Failed to create new HostKey”) because it can’t access the Secure Enclave from a session-less LaunchDaemon context. FileVault must be off.
  3. Register the GitHub Actions runner as a user-level LaunchAgent (the default svc.sh install path). Auto-login from step 2 makes the agent load on boot.
  4. Trigger one tart VM run interactively. macOS prompts once for “Local Network” permission (Sequoia/Tahoe’s privacy gate on the vmnet framework — without this grant, bridge100 never comes up and Packer’s SSH-to-VM step times out). Grant via System Settings → Privacy & Security → Local Network. The permission persists across reboots.

A first-class config-management owner for this checklist (k8s + tart-cri + a CR per host, or an Ansible playbook, or a Packer-built host image) is intentionally out of scope here. The current vm-image-builder-01 is set up by hand and the steps above are the runbook for the next one until that lands.

Bootstrap order

  1. One-time: Subscribe xcodereleases.com/api/all.rss in the infra-ops Slack channel.
  2. First populate: maintainer runs mise run xcode-mirror:upload <version> for each Xcode we want available (26.5, 26.4.1, …, the rolling Tahoe set).
  3. Trigger macos-xcode-image.yml to build Layer 1 images for each profile.
  4. First commit on main after merge triggers release-runner-image / release-xcresult-processor-image which build against the Layer 1 base.

Test plan

  • packer validate -syntax-only clean on all three .pkr.hcl files (macos-xcode-image, runner-image, xcresult-processor-image).
  • actionlint clean on all four workflows (only pre-existing self-hosted runner-label warnings remain).
  • mix compile --warnings-as-errors clean.
  • helm template tuist infra/helm/tuist -f values-managed-common.yaml -f values-managed-production.yaml renders without error; no stale TUIST_XCODE_MIRROR_* env vars or xcode-mirror-external-secrets ExternalSecret left behind.
  • mise run xcode-mirror:upload 26.5 validated end-to-end from a maintainer Mac during PR development — xcodes SRP signin + 2FA prompt works, .xip lands in ghcr.io/tuist/xcode-xips:26.5.
  • macos-xcode-image.yml dispatched against a fresh vm-image-builder-01 host with xcode_version=26.5: Packer build (17:40) + tart push to GHCR (10:31) = ~28 min total, succeeded on first attempt, ghcr.io/tuist/macos-tahoe-xcode:26-5 published.

What this PR ultimately ships

  • The Layer 1 / Layer 2 split — every macOS image rebuild gets ~30 min faster, owns its own Xcode catalog independent of Cirrus.
  • A single mise task for refreshing the .xip mirror, with the toolchain pinned in the root mise.toml.
  • A documented host-bootstrap checklist for vm-image-builder Mac minis, valid until the CAPI provider owns these declaratively.
  • Documentation pointing at the RSS feed as the “new Xcode shipped” notification surface.

No new in-cluster service, no Apple credentials in 1Password / External Secrets, no quarterly cookie-refresh runbook. The earlier draft of this PR introduced a Tuist.XcodeMirror Oban worker for in-cluster automation; the SRP migration and xcodes’ lack of cookie persistence made that path effectively impossible, so it’s been removed. The remaining work is small enough that the previous Phase 1/2/3 framing no longer applies.

🤖 Generated with Claude Code

Flights

Investigate, reproduce, or fix this item in an isolated repository. Each Flight preserves its outcome and agent session.

New Flights are paused Configure model inference, GitHub, and a sandbox provider to start another Flight. Existing results remain available below.
No Flights yet

Start a Flight and preserve its objective, outcome, and session here.

Comments
TA
tuist-atlas[bot] Jun 5, 2026

The changes from this PR are now available in release xcresult-processor-image@0.11.0. The xcresult-processor-image has been slimmed as Layer 2, now inheriting from the new Layer 1 base image, reducing rebuild times from ~30 min to ~2 min per commit.

TA
tuist-atlas[bot] Jun 5, 2026

The macOS+Xcode base image feature is now available in runners-controller@0.8.0. Update to this version to use the new Layer 1/Layer 2 image split and in-house Xcode catalog.