Hive
feat(server): gate production on an oldest-supported-CLI acceptance suite
GitHub issue · Closed
What changed
Adds a backward-compatibility acceptance suite to the production deployment cascade that runs the oldest CLI version we still support (3-month window, currently pinned to 4.155.0) against the freshly-deployed canary, and blocks production promotion on it.
e2e/module_cache_backward_compat.bats— using the pinned old binary: logs into canary, creates a throwaway project under thetuistorg, warms the module cache withtuist cache, wipes the local cache (isolated viaXDG_CACHE_HOME/XDG_STATE_HOME/XDG_CONFIG_HOME), then forces a remote pull with a focusedtuist generate. The validation is behavioral: a cached xcframework can only be linked in the generated project if the signed cache-artifact download succeeded, so exit code + xcframework-linked proves an old CLI pulled from cache without a signature rejection (no brittle log-string matching).e2e/fixtures/module_cache_app/— minimal iOS app + one static framework, built only from long-stable ProjectDescription APIs so the old CLI can compile the manifest..github/workflows/server-production-deployment.yml— newbackward-compatibility-acceptancejob (needs canary,tuist-macos,server-k8s-canaryenv), added toproduction.needs. Pinned viaMIN_SUPPORTED_TUIST_VERSION.server/priv/docs/en/cli/compatibility.md(+ CLI docs sidebar) — a customer-facing Compatibility page under the CLI docs stating the 3-month support window. This is a public commitment, so it lives in the product docs (modeled on how tools like Stripe publish a versioning and support policy), not the internal handbook.
Why
The existing acceptance-tests job always uses the CLI built from HEAD, so it can never catch a server change that breaks an older CLI. That is exactly the cache-signature incident: the server stopped attaching the cryptographic signature to cache-artifact downloads (downloadCacheArtifact); newer CLIs had already stopped requiring it, but older CLIs still ran SignatureVerifierMiddleware and rejected every cache response, dropping cache hit rate to 0% and wiping out CI for customers pinned to an older release. This suite pins that floor and exercises the exact path that broke, blocking promotion if an older-but-supported CLI can’t pull from cache.
Version selection
The server’s minimum supported CLI version is @minimum_supported_cli_version in server/lib/tuist_web/plugs/warnings_header_plug.ex (currently 4.118.1; below it the server emits a deprecation warning and degrades server-side features). We pin the gate to 4.155.0 (released 2026-03-04, ~3 months back), comfortably above that floor and within the 3-month window. Both the auth login server flag (--path on older, --url on newer) and the project create --build-system requirement changed across the window, so the suite is written for the current (>= 4.155) interface; move the pin forward only.
How this was validated
The real gate cannot be dispatched without running the full deploy cascade, so the job was validated in isolation against the live canary via a temporary standalone workflow (added during review, now removed). Dispatched manually against 4.155.0, it ran green end-to-end:
# Using tuist binary: 4.155.0
# Logging in to https://canary.tuist.dev...
# Creating throwaway project tuist/bc-...
# tuist cache output: ... All cacheable targets have been cached successfully as xcframeworks
# tuist generate output: ...
ok 1 oldest supported CLI pulls the module cache from canary without a signature error
That ok 1 confirms the full chain on the pinned version: install 4.155.0 -> login -> project create -> tuist cache warm (built and uploaded the xcframework) -> wipe local cache -> tuist generate focused pull from canary -> exit 0 and the cached xcframework linked in the generated project.
Issues found and fixed while validating against canary:
auth loginserver flag flips across versions (--pathvs--url); handled with a fallback.project createrequires--build-systemnon-interactively on >= 4.155 (older versions reject it), so the pin and the flag must agree.test_helperload placement andXDG_CONFIG_HOMEisolation (credentials live there, not under the cache dir).- Dropped the version-pinned Select Xcode step to match main’s macos-xcode-image change (it failed on fleet nodes missing the pinned Xcode).
Merge note
The gate is blocking (production.needs). While validating, canary was returning ~80% 500s on writes due to Postgres connection-pool exhaustion (DBConnection.ConnectionError, ~999 in 6h on canary, 0 on production), a separate canary incident tracked under its own task. Hold merge until canary is healthy so a flaky canary cannot block production promotion through this gate.
🤖 Generated with Claude Code