feat(server): model xctest runner errors as run/target-level Errors, not test cases

Models xctest (<pid>) encountered an error runner errors the way Xcode does — as a first-class Errors concept per target — instead of as test cases. This is the proper fix for the test_case.created webhook flood + test-count pollution that the normalize commit pulled from #11604 only band-aided.

Background

When xctest itself errors (a target whose .xctest bundle can’t be loaded, or an app that can’t launch), it emits a synthetic test case named xctest (<pid>) encountered an error. The pid varies per run, so the parser ingested each as a distinct, brand-new test case: unbounded unique test_cases rows, a test_case.created per occurrence (one project fanned out ~7,460 failing webhook deliveries in 3h), and a failed count that was ~95% junk (356 of 376 “failures” in a real run).

Opening that run’s .xcresult in Xcode showed the right model: a separate “Errors” section, one per target, with the real message (Failed to create a bundle instance representing '.../<Target>.xctest'. Check that the bundle exists on disk). These aren’t test failures — they’re targets that never ran.

What changed

• Parser (XCResultParser): classifies runner-error nodes out of the test cases into TestSummary.errors ([{target, message}], deduped per target since the pid varies), and marks the run failed when errors exist. Real test cases untouched. New unit test.
• Model + ingestion: new ClickHouse test_run_errors table (id, test_run_id, project_id, module_name, message, inserted_at) + Ecto schema; create_test persists them via create_run_errors. Because they’re no longer test cases, no test_case_runs and no test_case.created webhooks — the flood and pollution die at the source.
• Dashboard: an “Errors” card on the run detail, styled as a sibling of the test-failure cards — each error a card with a destructive alert icon, the target as a bold title (or a “Run error” fallback for target-less ones), and the message in the same monospace block failures use. The card sits directly above the failures card, below Test Details. Test counts stay clean; run status reflects the errors.
• “Failures” -> “Test Failures”: renamed the failures section on the run page (tab, overview card, failures tab) to align with Xcode’s terminology and make the distinction from run/target-level Errors obvious. The single test-case-run page keeps “Failures” (no Errors there to disambiguate).
• Docs: server/data-export.md documents the new table; a marketing changelog entry (2026.07.02-test-run-errors) announces the Errors section + rename with a screenshot.

Design decisions (the agreed defaults)

• Errors modelled at the target/module level (module_name); run-level for target-less ones (module_name empty).
• Not double-counted: Errors are separate from the failed-test count (unlike Xcode, which shows both); run status is failed if either. That’s the point — stop inflating the failure count.
• Chose the errors model over the earlier name-normalization stopgap (which kept them as test cases). That commit was pulled from #11604.

Deferred (follow-ups, noted)

• CLI local inspect test carry: the remote/processor path (the flood source) ingests the parse directly and is fully covered. Local-mode runs would carry errors via a new errors field on the /tests API payload + CLI client regen — deferred to keep this PR off the OpenAPI-regen path.
• Top-level run errors (e.g. Cannot launch simulated executable, not tied to a target) come from the action log rather than the test tree; v1 captures the per-target errors (the bulk). The run-level shape (module_name empty) is already supported for when we add them.
• Backfill: existing per-pid junk test_cases rows aren’t cleaned (forward-only).

Validation

• mix compile clean (no new warnings on changed files); mix format + swiftformat + prettier clean; mix gettext.extract run (only dashboard_tests.pot committed — additive: adds Run error + Test Failures, nothing removed).
• UI verified against a locally seeded run with both errors and failures: Errors card renders as failure-card siblings, sits above Test Failures, and the renamed labels render (tab + cards). Changelog entry verified rendering at /changelog/2026.07.02-test-run-errors with the image loading.
• Swift: added a parser unit test asserting errors are lifted out, deduped per target, real tests survive, and the run is marked failed. Local swift test on the nested xcresult_nif package is blocked by an unrelated SwiftPM registry/SCM dedup resolution error; CI builds the NIF and runs it.
• Server-side ingestion/UI have no unit tests here (they mirror the existing test_run_destinations path); verified by compile + review.

🤖 Generated with Claude Code

No GitHub comments yet.