Hive Hive
Sign in

Rolling flaky/reliability alerts no longer double-count re-inserted runs

Metadata
Source
GitHub
Version
xcresult-processor-image@0.39.2
Domains
Published
Jul 03, 2026 · 14:26 UTC
Repository
tuist/tuist
Update

Tuist’s server-side rolling flaky and reliability alert evaluation now counts each test run exactly once over the configured window. Previously, runs that were re-inserted to mark them as flaky were counted multiple times (2-4x) in the recent-runs aggregate, which inflated flakiness_rate and flaky_run_count, and deflated reliability_rate. In practice this caused flaky-test automations to incorrectly auto-skip genuinely reliable test cases and to mute genuinely healthy ones. Rolling flakiness_rate, flaky_run_count, and reliability_rate alerts (across all supported window sizes) now deduplicate by run timestamp before computing the rate, so threshold-driven skips and mutes only fire on tests that are actually meeting the criteria.