I think the durable fix is to make run completion a persisted-state problem first, and treat the push notification as a best-effort accelerator.
I would avoid clustering the macOS processor nodes with the web nodes for this. It would couple a specialized parsing fleet to the live web cluster for the sake of a dashboard refresh, and it would bring extra operational surface: Erlang distribution credentials, node discovery, network reachability, and remote process messaging across two very different runtime roles. That feels like the wrong boundary.
My preferred shape would be:
- Keep processor nodes isolated.
- Make
TestRunLivepoll while the run is in any transient state, bothprocessingandin_progress. - Stop polling only after a terminal state is visible, ideally with one delayed follow-up refresh so ClickHouse buffered rows have time to land.
- Keep Phoenix Publish-Subscribe (https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html) as an optimization when the event originates on the web cluster.
- If we still want push from the processor side, use an explicit web-side notifier or outbox instead of a one-off broadcast relay hidden inside
Tests.create_test/1.
There are also two concrete issues in this implementation:
- The relayed payload is a
%Tuist.Tests.Test{}struct, but several aggregate dashboards match{:test_created, %{name: "test"}}, so they still ignore the event. - Sharded runs can wait in
in_progress, but the detail page only subscribes and polls forprocessing, so those pages can still stay stale.
So I would remove the Oban relay from this path and make the LiveViews resilient to missed notifications. That keeps the processor boundary clean and makes the dashboard correct even when messages or jobs are delayed.