fix(cli): stop per-op CAS analytics fsync from starving the daemon

What

Two related changes to the CAS daemon’s analytics write path:

1. Fix (the impactful one): open the analytics database with PRAGMA synchronous = OFF instead of NORMAL, so the per-op metadata insert no longer fsyncs.
2. Instrumentation: record transfer_duration (network) and codec_duration ((de)compression) alongside the existing total duration per op.

Why — root cause of the hosted-runner CAS slowdown

Hosted macOS runners (tuist-macos) were ~3× slower than a competitor on the warm Xcode-CAS (compilation cache) benchmark: each materialize measured p50 138 ms vs 49 ms, same op count, same payload, same hit rate. A layered investigation ruled the cost out of every obvious place:

• Not the network / PN. The PN cache node serves each artifact <1 ms; the full round-trip measured from inside a runner VM is ~2 ms (reused conn). A faithful replica of the daemon’s exact URLSession transport does ~2 ms/op even with all cores saturated.
• Not bandwidth or decompression. Per-op duration is independent of payload size (corr ≈ 0).
• Not async/middleware/auth hops. 8–16 hops through a shared actor stay ~free even under CPU saturation.

The one thing left was what storeMetadata does after every op: a fire-and-forget Task { storeCASOutput(...) } — a blocking SQLite insert. The database used synchronous = NORMAL + wal_autocheckpoint = 1, i.e. a checkpoint+fsync on nearly every insert, running on the shared cooperative thread pool. During a build, swiftc compilers saturate the same disk writing object files, so each per-op fsync stalls, ties up cooperative threads, and starves the daemon’s load/decompress tasks. The op latency balloons — and the slower the disk’s fsync under contention (a Tart VM’s sparse-image disk vs bare NVMe), the worse it gets, which is the bulk of the gap vs the competitor (whose VM disk fsyncs faster, so it pays a much milder version of the same tax).

Measured in isolation (real SQLite, the daemon’s exact PRAGMAs, fire-and-forget per op), under CPU + disk-fsync contention mimicking a real build:

So the fsync, not the insert, is the amplifier, and it’s size-independent and concurrency-scaling — matching the observed signature exactly.

How

• synchronous = OFF: the analytics DB is disposable per-build telemetry with no durability requirement, so skipping fsync is safe. The insert becomes a page-cache append and no longer blocks the pool. wal_autocheckpoint = 1 is kept so the main .db stays current for the build-report upload (which copies the .db, not the -wal sidecar).
• transfer_duration / codec_duration: CASService.load/save time the network call and the (de)compression call separately and persist both. The remainder (duration − transfer − codec) is the daemon’s per-op scheduling/overhead — which this investigation showed is exactly the starvation the fix removes. Columns live in the shared cas_outputs schema (CLI writer + server NIF reader; the reader selects columns explicitly, so it’s unaffected) and are back-filled on existing DBs via ALTER TABLE.

Impact

Affects every CAS build, not just the benchmark — every customer using the compilation cache was paying this per-op fsync tax. The fix is disk-config-independent: it removes the dependency on fsync speed entirely, so it helps hosted runners and self-hosted setups alike. (An orthogonal infra lever — Tart --root-disk-opts sync=none — exists but carries a host-wide durability tradeoff; this targeted fix is preferred.)

Validation

• Mechanism measured on a real runner host with faithful Swift replicas (transport, async hops, and the SQLite write under contention) — the table above isolates the fsync as the cause.
• synchronous=OFF eliminates fsync by SQLite semantics, so the per-op write collapses to the “no write” case (~3.6 ms under the same contention).
• Local swift build is blocked in this environment by an unrelated SwiftPM resolution issue (Duplicate keys ContentType / multiple similar targets 'Path'); CI is the build gate. The CASAnalyticsDatabase round-trip test and the CASServiceTests mock stubs are updated for the new signature.
• Once in a canary, the runners-benchmark report strategy harvests cas_analytics.db per runner, so the transfer_duration/codec_duration split and the post-fix per-op latency can be read directly.