Replacing static snapshots with temporal context in testing and automation
This week we replaced static metric snapshots with time-aware histories across test analytics and automation settings. Knowing the current state is only half the battle; understanding the trajectory is what lets you act.
A single number rarely tells the whole story. When we look at a test suite or an automation pipeline, a static snapshot hides the trajectory that led to the current state. This week, we focused on replacing those flat aggregates with temporal context, giving you the history behind the metric.
Consider test durations. Previously, the Test Cases table showed a single mean duration for the last 50 runs. A single stalled run or a paused debugger could skew that average, pushing a normally fast test to the top of your slowest list. We replaced this with p50, p90, and p99 percentiles alongside the average, scoped to a 14-day window and your chosen environment. Reading these together reveals the actual behavior of a test. A test whose p50 and p99 match is consistently fast, while one with a towering p99 is usually quick but occasionally stalls. You can now sort by median or worst tail, separating the truly slow from the occasionally problematic.
But knowing the distribution of a test duration is only part of the picture. You also need to know how its reliability evolved. A test sitting at 93 percent reliability is a very different problem if it has been stable all month versus if it regressed on Tuesday. We replaced the static four-number summary on the test case page with interactive charts. Every metric is now a trend. The runs chart stacks daily outcomes, turning a vague reliability drift into a regression with a specific date. Flakiness and reliability are drawn against a fixed axis, so steady performance reads as a flat line rather than a cliff.
This shift from snapshot to history extends beyond testing into project automation. When an automation behaves unexpectedly, the current configuration rarely explains why. We added a configuration history to project automation settings. Each entry records who made the change, when it happened, and the exact fields that shifted from their previous values. The latest five changes appear first, keeping the immediate context visible while preserving the full audit trail.
We are building a product that respects the timeline of your codebase. Static numbers tell you where you are. Histories tell you how you got there, which is the only way to figure out where you are going.
Narrated from 3 public drops.