feat(server,kura): track Kura node usage and add Usage dashboard

Metadata

Source

tuist/tuist #10979

Updated

Jun 24, 2026

Domains

Kura

Details

Adds the Kura node usage tracking pipeline (node rollups → ClickHouse) and an account-scoped Usage dashboard so customers and Tuist can see what each managed Kura mesh moves. The plumbing pieces (node-side rollups, internal ingestion endpoint, control-plane auth) and the dashboard are split into two commits for review.

Summary

Ingestion (feat(kura,server): record per-tenant Kura node usage rollups) — Kura nodes now aggregate per-tenant traffic into hourly rollups (bytes, request count, direction, operation, region) and POST them to an internal Tuist endpoint. Server side adds the kura_usage_events ClickHouse table + a daily aggregating materialized view, a shared-secret-authenticated internal controller, Tuist.Kura.Usage.create_events/1, and Helm/secret wiring for the new shared secret. data-export.md updated.
Dashboard (feat(server): add Kura usage dashboard) — New /<account>/usage LiveView under the account layout with three clickable widgets (Egress / Ingress / Requests) that switch the chart series, a per-region time-series chart, a per-node breakdown table, and project + date-range filters. Gated by a new FeatureFlags.kura_enabled?/1 so the sidebar entry and the page only show when Kura is enabled for the account. Read queries live in Tuist.Kura.Usage and run against ClickHouseRepo.
Local dev — priv/repo/seeds.exs now emits 30 days of synthetic hourly Kura usage events across four nodes in three regions for two projects, with realistic egress/ingress asymmetry, so the dashboard renders non-empty after seeding.

How to test locally

# Bootstrap the dev stack

brew services start postgresql@16
mise run clickhouse:start
mise install
mise run db:reset     # picks up the new kura_usage_events migration


# Seed Kura usage events alongside the rest of the dev data

cd server
mix run priv/repo/seeds.exs

# Start the dev server and log in as tuistrocks@tuist.dev / tuistrocks

mise run dev

Then visit http://localhost:8080/tuist/usage and:

Confirm the Usage sidebar entry only appears for accounts with Kura enabled (dev mode auto-enables it).
Click each widget (Egress / Ingress / Requests) and watch the chart series + y-axis formatter swap.
Switch the project filter between Any, tuist, and android; switch the date range between 24 hours / 7 days / 30 days and confirm the bucket flips between hourly and daily resolution.
Confirm the per-node table shows Egress and Ingress bytes columns and that totals are sorted by combined bytes desc.

Ingestion side:

# Ingestion controller tests

cd server
mix test test/tuist_web/controllers/internal/kura_usage_controller_test.exs

# Full Kura module tests

mix test test/tuist/kura_test.exs

Comments

F

fortmarek May 28, 2026

Finding: clarify whether Usage should reuse existing cache events or become a billing-grade metering source

Before making kura_usage_events a separate Usage source of truth, could we explain why aggregating the cache events we already track is not enough? Kura already emits project-scoped cache analytics for successful Xcode/Gradle uploads/downloads, and the existing ClickHouse tables include action, size, project, endpoint, and timestamp, which could power much of this dashboard.

If this Usage pipeline is intended for billing, then a separate durable metering path may be the better design because the existing analytics queue can drop events or circuit-open. In that case, though, this path needs to be treated explicitly as billing-grade: durable delivery, reliable dedupe by event_id, export documentation, and retry/idempotency tests. Without that distinction, we end up with two cache-usage sources that can disagree.

F

fortmarek May 28, 2026

@pepicrft I’d add to the above finding that it might be useful to split the usage also based on the type (Xcode Cache, Gradle Cache, etc.)

P

pepicrft May 28, 2026

Good point. The reason for a separate pipeline is that not all cache traffic will be bound to a project. For example, I want to meter cacheable scripts too, and those don’t fit the project-scoped cache event model. Kura sees every byte going in and out of the node regardless of the artifact’s owner, so metering at that layer keeps the schema uniform as more cache use cases land.

The billing-grade pieces you called out are in this PR already: durable RocksDB outbox on the Kura side, deterministic event_id, at-least-once delivery, and idempotent server-side aggregation via argMax(col, inserted_at), with a regression test that double-inserts the same event_id and asserts totals don’t move.

P

pepicrft May 28, 2026

Good call. The data is there already, kura_usage_events records an artifact_kind per event (xcode, gradle, nx, metro, module, reapi), so the per-type split is purely a UI follow-up. I’ll layer in a Type filter dropdown and a per-type breakdown card in a separate PR to keep this one scoped.

F

fortmarek May 28, 2026

[P2] server/priv/ingest_repo/migrations/20260527120000_create_kura_usage_events.exs introduces a new customer-data store for usage metering (kura_usage_events includes account/project identifiers plus byte and request counts), but server/data-export.md is not updated in this PR. The repo guidance says new tables/customer data storage need to be reflected in the data export inventory so legal/customer exports remain complete. If we intentionally want this as a follow-up, can we link that tracked follow-up here; otherwise I think this should land with the schema change.

TA

tuist-atlas[bot] May 30, 2026

The Kura node usage tracking and Usage dashboard feature is now available in xcresult-processor-image@0.10.0. Update to this version to use it.

TA

tuist-atlas[bot] May 30, 2026

The Kura node usage tracking and Usage dashboard feature is now available. Update to server@1.203.0 to use it.

TA

tuist-atlas[bot] Jun 3, 2026

The Kura node usage tracking and Usage dashboard feature is now available in kura@0.6.0. Update to get these improvements.