Hive Hive
Sign in

Durable logs for abnormal self-hosted runner deaths

Metadata
Source
GitHub
Version
runners-controller@0.14.0
Domains
Published
Jun 23, 2026 · 12:21 UTC
Repository
tuist/tuist
Update

When a Tuist self-hosted runner pod dies mid-job, the controller now captures the runner container’s log tail and re-emits it on the controller’s own stdout before the pod is reaped. This preserves a diagnostic trail (“runner death log captured”, tagged with pod, pool, and endedAt) that previously vanished the instant the reap deleted the pod. Capture is bounded (200 lines / 64 KB), deduplicated, and only fires on abnormal exits (non-zero, SIGKILLed, or a reap with no clean exit), so a workflow that simply fails its own tests is not logged. After deploy, abnormal runner deaths are queryable in Loki with {pod=~".*runners-controller.*"} |= "runner death log captured". No runtime behavior change for jobs.