What changed
Suite-granularity test sharding wrote each shard’s per-suite selection into the test products bundle’s .xctestrun as OnlyTestIdentifiers, then ran xcodebuild test-without-building -testProductsPath <bundle>. This now derives -only-testing identifiers (Target/Suite for suite granularity, bare Target for module granularity) and passes them to xcodebuild instead. ShardService no longer rewrites the bundle’s .xctestrun at all; both shard consumers (TestService.runShard and XcodeBuildTestCommandService) forward the identifiers.
ShardService: Shard now carries testIdentifiers + testProductsAreTemporary; filterXCTestRun and the xctestrun mutation are removed. Downloaded/extracted products are left untouched, and only Tuist-owned temp products are cleaned up.TestService.runShard / XcodeBuildTestCommandService: emit -only-testing for shard.testIdentifiers; drop the -xctestrun switching.
Net -479 / +237 — mostly deleting the dead filtering path and its unit tests.
Why
OnlyTestIdentifiers in an .xctestrun filters XCTest only; it does not filter Swift Testing tests. The acceptance suites are entirely Swift Testing (import Testing + struct … { @Test … }), so when a shard’s products bundle downloaded successfully, test-without-building launched the test host, matched zero Swift Testing tests, and exited 0 — reporting ✔ The project tests ran successfully while running nothing.
This was systematic: every successful-download shard ran 0 tests; the only variation across CI runs was which shard’s Tigris bundle.zip download timed out (a separate, untouched bug). The result is that no sharded acceptance run had actually been executing the suite.
-only-testing is the documented mechanism that filters both XCTest and Swift Testing at the suite (Target/Suite, 2-component) level; the known Swift Testing -only-testing parenthesis gotcha is function-level only (3rd component), which sharding does not use.
Why this approach over the obvious alternative
Keeping the .xctestrun rewrite and trying to make OnlyTestIdentifiers select Swift Testing suites isn’t viable: OnlyTestIdentifiers can only ever exclude Swift Testing (any value you set excludes it, because no Swift Testing test matches an XCTest-style identifier) — it can never select one. Swift Testing selection isn’t a static xctestrun artifact at all; it’s a runtime test-session concept that -only-testing drives.
This was verified empirically against a minimal Swift Testing bundle:
- Six
OnlyTestIdentifiers formats (Suite, Suite/, Suite/test, Suite/test(), Suite/test()(), Target/Suite) all selected 0 Swift Testing tests; the same key with an XCTest class name runs it fine.
-only-testing writes no modified xctestrun — so it isn’t “OnlyTestIdentifiers with a better format.” Inspecting the xctest host under -only-testing shows an empty XCTestConfigurationFilePath and no --filter/SWT_*/selected-IDs arg or env var; the chosen tests are negotiated over the runtime test session (XPC). No xctestrun key (OnlyTestIdentifiers, CommandLineArguments, EnvironmentVariables, …) can carry it.
So -only-testing arguments are the only supported selection path that works for both frameworks. They compose with -testProductsPath, and dropping the rewrite also removes the per-shard .xctestrun files (multiple shards sharing one local products dir now just read it).
Tests
- Unit (
ShardServiceTests): rewritten to spec the new derivation — suite granularity → ["AppTests/LoginTests", …], module granularity → ["AppTests", …] — and to assert the bundle’s .xctestrun is left unmodified.
- Wiring (
TestServiceTests): asserts -only-testing AppTests reaches xcodebuild.
- E2E (
TestAcceptanceTests): new suite-granularity shard test over a fixture that now mixes one XCTest class with two Swift Testing suites (MacFrameworkSwiftTestingTests). It runs both shards, parses each result bundle via XCResultService, and asserts the Swift Testing tests actually executed across the shards. This verifies the fixed behavior end-to-end (green path). Caveat: because it is itself a Swift Testing test, it cannot reliably go red on a regression of the outer suite-granularity sharding — such a regression would silently skip it along with every other Swift Testing test. Robustly catching that regression class is the zero-executed guard’s job (see follow-ups).
Validation
The exact mechanism the fix turns on was reproduced directly with xcodebuild test-without-building against a minimal Swift Testing bundle (one XCTest class + two Swift Testing suites, mirroring the fixture):
Selecting Swift Testing suite GreetingTests via… |
Result |
| Control (no filter) |
all 3 suites run |
OLD — OnlyTestIdentifiers=["GreetingTests"] in the xctestrun (what filterXCTestRun wrote) |
Executed 0 tests … TEST EXECUTE SUCCEEDED — the false-green bug |
NEW — -only-testing LibTests/GreetingTests (what this PR emits) |
greeting_isStable() runs and passes |
XCTest contrast — OnlyTestIdentifiers=["LegacyTests"] (an XCTest class) |
testHello runs — proving the failure is Swift-Testing-specific |
That is the before/after in isolation: OnlyTestIdentifiers on a Swift Testing suite selects nothing (0 tests, silent success); -only-testing on the same suite runs it; and OnlyTestIdentifiers still works for XCTest, so the breakage is Swift-Testing-specific.
CI (green): the full compile chain (Build, Build Acceptance Tests, SwiftPM Build, Linux Build/Unit Tests, Lint), Unit Tests, and both Acceptance Tests (0/1) shards pass — i.e. the new unit + e2e tests and the whole suite-granularity acceptance run are green with the fix.
Local Tuist build/run is blocked in some sandboxes (the package graph hits the FileSystem product-name conflict under swift build, and tuist generate can fail on a BasicContainers dependency error), so the table above and CI are the validators.
Out of scope / follow-ups
- Zero-executed guard (recommended). A production check that fails when a shard executes 0 tests against a non-empty selection would have caught this at the source, and is the robust guard for this regression class — since the e2e test, being Swift Testing, is itself skipped by the bug rather than failing. Not included here.
- The Tigris
bundle.zip download timeouts that intermittently hard-fail a shard are a separate, pre-existing issue.
How to test locally
On a machine where the Tuist project builds:
tuist test MacFrameworkTests --build-only --shard-total 2 --shard-granularity suite --shard-skip-upload -- -testProductsPath /tmp/p.xctestproducts -destination platform=macOS (in examples/xcode/generated_ios_app_with_tests).- For each shard:
tuist test MacFrameworkTests --without-building --shard-index N --result-bundle-path /tmp/shard-N.xcresult -- -testProductsPath /tmp/p.xctestproducts -destination platform=macOS.
- Confirm the
MacFrameworkGreetingTests / MacFrameworkValueTests Swift Testing suites actually run (result bundle shows greeting_isStable / greeting_isNotEmpty), rather than the run completing instantly with zero tests.
🤖 Generated with Claude Code
T
