What changed
Hive’s model gateway now accepts embedding requests through POST /inference/v1/embeddings.
The implementation reuses the existing profile and token model:
- Validates the requested stable model name against the token-bound profile.
- Rewrites that model to the configured upstream embedding model before proxying.
- Forwards the upstream response without exposing provider credentials to clients.
- Records each request with an explicit
embedding operation on inference usage rows.
- Estimates embedding cost from input tokens, with output tokens recorded as zero.
- Documents the new endpoint and embedding pricing behavior for self-hosted operators.
Why
Atlas currently has to point document embedding traffic directly at the upstream provider because Hive only relayed chat completions. That bypass makes model changes and cost attribution harder to manage centrally.
Routing embeddings through Hive gives operators the same stable profile name, provider credential hiding, token attribution, and cost tracking that chat completion clients already get.
Approach
I kept embeddings as another gateway operation rather than introducing a separate embedding subsystem. The new controller action follows the same authentication, model authorization, provider lookup, and response forwarding path as chat completions, but uses the upstream /embeddings path and non-streaming response handling.
Usage rows now store an operation value so chat completion and embedding traffic can be distinguished later without changing the profile and token model.
Impact
Existing chat completion clients continue to use the same endpoint and profile configuration. Existing usage rows receive chat_completion as the default operation through the migration.
Operators can create an embedding profile, point OpenAI-compatible embedding clients at Hive’s /inference/v1 base address, and use the Hive profile token as the bearer credential. Embedding profiles should set the input token price for the upstream model so Hive can estimate spend from provider-reported usage.
Validation
mix format
mix test test/hive/inference_test.exs test/hive_web/controllers/inference_controller_test.exs
mix test test/hive_web/live/ops_live/inference_test.exs
mix compile --warnings-as-errors
mix test returned 801 passing tests.
- Ran a one-off local proxy check that sent a Hive embeddings request through the controller to a local OpenAI-compatible upstream over the real
Req client, then verified model rewriting, upstream authorization, response forwarding, and costed usage recording.