Hive
feat(test): add agent-native test capability
GitHub issue · Closed
Summary
This adds an agent-native test capability to Once so coding agents can discover test targets, choose the tests affected by a set of changed files, run them, and read normalized results through the same CLI and MCP surfaces they already use for graph operations. The motivation is to make tests part of the structured build graph instead of forcing agents to infer project-specific runner commands from prose or ad hoc scripts.
The implementation keeps runner and toolchain knowledge in Starlark rules. Rust only consumes generic graph data, declared sources, dependencies, affected inputs, actions, and once.test_results.v1 output. That keeps the core extensible while still giving concrete ecosystems useful entry points: ShellSpec tests are exposed through a dedicated test rule, and Apple test bundles can opt into Swift Testing execution.
Approach And Tradeoffs
The main design choice was to model tests as a generic capability rather than teaching Rust about ShellSpec, Swift Testing, or any other runner. A broader script_test abstraction would have been simpler to name, but it would hide too much runner-specific contract in a vague escape hatch. Concrete Starlark rules give each ecosystem a discoverable schema, runnable examples, affected-test metadata, and normalized result output while preserving a generic Rust execution path.
Affected-test selection is graph based: tests are selected from declared sources, dependencies, and explicit affected inputs. That makes the MCP tools useful to agents without baking file naming conventions or toolchain semantics into the CLI. The same contract is available from once query subcommands so humans can reproduce what an agent sees from the terminal.
Verification
mise exec -- cargo fmt --all -- --checkmise exec -- cargo test -p once-cli -- --nocapturemise exec -- cargo test -p once-frontend --test examples -- --nocapturemise exec -- cargo build --releasemise exec -- shellspec spec/apple_spec.shmise exec -- shellspec spec/test_capability_spec.sh
No GitHub comments yet.