Hive Hive
Sign in

Traffic-driven Kura placement

#87 · Tuist · Public · Created directly

Kura
Proposed
Proposal

Important

Nobody is choosing regions today. On 2026-08-20, nine of the ten live customer Kura instances belonged to accounts whose storage-region setting was “all regions”, 47 of 58 demand-active paid accounts had no service region assigned at all (resolution fell through to the US East default with no error and no capacity event), and 30 demand-active Air accounts restricted to Europe were refused service with no visible signal. The one mechanism that does place accounts by latency is an operator doing it by hand: US West is reachable only through an explicit per-account assignment. This spec makes that manual practice the system.

Summary

Stop asking customers to choose cache regions. The dashboard currently offers a region picker with create and destroy controls, and the question it asks (“where should your cache servers run?”) is one most customers cannot answer, because the answer depends on where their CI runs, which they often do not know and which changes under them. This spec replaces the picker with Tuist-owned placement driven by observed traffic:

  • Air keeps exactly one instance, placed in the region that receives the majority of the account’s cache-driven traffic, following that majority when it durably moves.
  • Pro and Enterprise run as many instances as sustained demand justifies: when qualifying traffic from a location crosses an absolute threshold, we start an instance in the matching region; when a secondary instance’s demand stays below a much lower threshold for a long window, we retire it. The mechanism is shared across both plans; the thresholds are per-plan configuration, and Pro’s conditions for creating a new instance can be more lenient than, or simply different from, Enterprise’s.
  • The create and destroy controls disappear. What stays customer-facing is the existing residency constraint (where data may live) and a read-only serving status that tells the account its cache follows its traffic. Where servers run, how many there are, and how much disk they claim becomes an implementation detail, the same way the private runner-cache regions already place themselves by following the runner fleet with no customer control and nobody missing one.

The placer’s only novel decision is which account-region instances should exist, at what claim size. Everything below that line is expressed as standard Kubernetes scheduling inputs on machinery the clusters already run, so the runtime behavior is mostly configuration, not new orchestration (see “The Kubernetes levers”).

This is also the separate, privacy-reviewed measurement design that spec #78 required before any placement decision could use client geography. Origin attribution happens only at the control plane, maps the client address to a coarse origin inside the request path, discards the address immediately, and persists only per-account aggregates. Kura nodes stay free of geographic lookup, preserving spec #78’s decision to remove it.

Motivation

Region selection is a question customers cannot answer, and the evidence is that they do not answer it. The callout numbers above decompose into three distinct failure modes, all live today:

  • The setting goes unset. Nine of ten live customer instances sit on all-regions accounts; spec #78 observed the same shape a month earlier (65 of 78 paid accounts allowed all regions on 2026-07-21). The picker exists; the population walks past it.
  • Unset falls back silently. 47 of 58 demand-active paid accounts resolve to the US East default with nothing recorded and nobody deciding. A customer in Europe whose CI is in Europe gets a US cache and no indication that a better placement exists.
  • Set can be unservable. 30 demand-active Air accounts restricted to Europe are refused outright because Air is served from US East only. The refusal is invisible: those accounts simply have no warm cache and no way to learn why.

Latency is the product: the value of a warm cache is the round trip it saves. Delegating the main latency lever to the party with the least information about traffic origins is the wrong allocation of the decision, and the manual escape hatch (an operator assigning US West per account, by hand, for latency) is this spec’s job description already being done without the data to do it well.

Current state

  • Two dashboard surfaces offer the picker: the cache page (server/lib/tuist_web/live/cache_live.ex) and the Kura servers card in server/lib/tuist_web/live/account_settings_live.ex. Both derive their choices from Regions.selectable/0 minus already-occupied regions, enforce one server per account and region, and expose create and destroy buttons. The current copy (“Each region can have at most one server”) leaks the infrastructure model into the product surface.
  • The managed catalog is US East (Vint Hill), US West (Hillsboro), EU Central (Paris), and a Canada East entry that exists in the catalog but is not enabled in production.
  • Underneath the picker, Tuist.Kura.AccountPolicies.resolve/1 already derives one service region per account, in order: the account’s storage-region setting (a compliance control with values all, Europe, USA), then a versioned operator assignment (console-only; the assignment functions exist but have no UI or task caller anywhere in the repo), then stickiness to whichever live instance already exists, then a US East default. Air resolves only to US East.
  • Disk claims are statically derived the same way regions are: Regions.storage_profile/1 maps the plan to a constant (Enterprise 50Gi, Pro 30Gi, Air 8Gi), and Tuist.Kura.StorageClaims holds hand-set per-account operator overrides. That module’s own doc concedes the problem: the plan is “a weak predictor of any individual account”, and the measured spread inside a single plan reaches two orders of magnitude in both directions. The local storage class rejects in-place expansion, so a claim is pinned at instance build and changes only when an operator override re-pins it and the instance is rebuilt.
  • Tuist.Kura.Demand records that an account wants cache, at cache-endpoint resolution, and resolves where only at flush time. Nothing anywhere records where traffic comes from: the request middleware resolves the true client address for rate limiting and discards it, the CLI measures per-endpoint latency in selectBestEndpoint and discards the measurement, and run events record which endpoint the CLI chose, which reveals preference only among instances that already exist.
  • The demand-driven lifecycle from spec #78 already owns whether and when an instance exists: provision on first qualifying demand, archive after 90 complete inactive days (60 for Air under capacity pressure), Enterprise never archived by inactivity.
  • Kura is terminal storage. There is no object-storage tier behind it: a miss is a 404 and the client rebuilds and re-uploads. A new or cold-returned instance starts empty and warms from real traffic, or from a cross-region peer when the account has one.
  • A known hazard: changing an account’s resolved service region today cold-provisions an instance in the new region and strands the old one forever, because Enterprise instances never archive and nothing couples resolution changes to moves.

Proposal

Tuist owns placement (decided)

Remove the customer-facing create and destroy controls from the cache page and the account settings card. Placement authority consolidates in the control plane: the lifecycle that already decides whether an instance exists (the placer, from here on) also decides where it exists and, for paid plans, how many exist. Provisioning where demand exists becomes provisioning where demand from that place exists.

Two customer-facing things survive, and the distinction between them is the heart of the change:

  • The residency constraint survives because it is a compliance question, not a placement question. The existing storage-region setting keeps its meaning as a boundary (data may live only in Europe, only in the United States, or anywhere) and is presented as exactly that. The placer never places outside it. A customer who must answer “where may our data live?” for a security review can still answer it; nobody is asked “where should servers run?” anymore.
  • A read-only serving view replaces the management card, and it deliberately does not name regions (decided). The view says the cache is active and served close to the account’s traffic; there are no buttons, and no region word to anchor a region request. Where data may live stays answerable through the residency setting, which is the one customer-facing place geography appears; where it does run is visible in the operator surfaces and available through support. Relocations and expansions are silent by the same logic (decided): no notification fires, the serving view is simply always current, and every transition lands in the operator audit trail.

Operator surfaces grow rather than shrink. The existing versioned service-region assignment gains an ops UI and becomes the override mechanism (pin an account to a region, suppress expansion, force a move), each with owner, reason, and history. The pin is also the per-account rollback for everything in this spec: a pinned account is invisible to the placer.

Spec #77’s opt-in act, where an administrator asks Tuist to deploy a Kura node, keeps its commercial meaning as consent to the paid model. It stops naming a region. Provisioning follows demand and placement follows traffic.

The origin signal: coarse, in-path, control-plane-only

The control plane already sees every client address on every request (it rate limits on it). This design persists strictly less than what already transits: a coarse origin label, aggregated per account per day. Attribution happens at control-plane request boundaries only, on requests clients already make:

  • Each qualifying demand event gains an origin. The request path maps the already-resolved client address to the coarsest unit that still distinguishes candidate regions: country for most of the world, state or province where one country hosts several candidate regions (country alone cannot separate US East from US West). The address itself is discarded inside the request path and never persisted.
  • A static, versioned mapping table translates origin into a candidate-region preference order (France to EU Central, Oregon to US West, Virginia to US East). The table is deployment configuration reviewed like code. There is no per-request network lookup, and no geographic database returns to Kura nodes.
  • The unit that counts toward thresholds is a run, a build or test run that used the cache, attributed once to its origin. Endpoint resolution would be the wrong unit: it is a presence signal with biases documented in Tuist.Kura.Demand itself (client-side endpoint caching under-counts, the setup launch agent over-counts), so raw resolution or request counts would let those biases move servers. Runs are robust to both, and run ingestion already sees the client address at its boundary. CI and developer-machine traffic both count; the mix is the point.
  • Retention is aggregate-only: per-account, per-origin, per-day counters. No per-user or per-device series. The aggregates are organization-level operational data and get documented in the server data-export documentation when they ship.

The static table is the v1 signal; client-measured latency is the upgrade. The CLI’s existing selectBestEndpoint probe reveals which existing instance is actually fastest, and run events already record its choice, so a systematic disagreement between the mapping table and revealed preference is observable from day one. If the disagreement rate shows the table misplacing accounts (VPN egress in the wrong country, cloud CI whose network distance defies geography), the upgrade is clients probing candidate regions and reporting measurements; that needs a client release and a telemetry channel, so it is deliberately not the v1 dependency. The same disagreement signal is also the table’s review mechanism (decided): the table is owned like the code it ships with, changed through review, and an alert on the disagreement rate replaces any calendar cadence. The alert, not a scheduled audit, is the trigger to fix entries.

Air: one instance that follows the traffic

  • Initial placement uses the origin of the demand that triggers provisioning. The first qualifying demand carries an origin; the instance is provisioned in that origin’s candidate region, within the residency constraint and subject to per-region Air capacity. No observation window precedes first provisioning: a wrong first guess is cheap because the instance starts empty either way, and relocation exists.
  • Relocation requires the traffic to have durably moved. Another candidate region must carry a majority of the account’s qualifying runs (starting value: 60 percent over 30 consecutive days), the account must clear a minimum-volume floor so noise cannot move anyone (starting value: 10 runs per day), and relocations are capped (starting value: one per account per quarter). All three are configuration, tuned during the shadow phase rather than re-litigated here.
  • Air moves cold. A move provisions the new instance empty, flips resolution, then drains and archives the old one: a miss is safe, the client rebuilds, and the Air working set is small enough that paying it once per relocation beats maintaining move machinery for the free tier.
  • Air candidates are bounded by per-region Air capacity budgets, and unmet preference is the procurement signal. A region without an Air budget is not a candidate even when it is the majority origin; the placer serves the nearest funded candidate instead and records the unmet preference. Sustained unmet preference is a capacity trigger in exactly the sense of spec #78. The 30 Air accounts currently refused for their Europe restriction stop being an invisible failure and become the quantified case for a European Air budget, served the moment one exists.

Pro and Enterprise: expansion and retirement by absolute thresholds

  • The primary follows the majority rule. Same placement and relocation policy as Air, except a paid primary moves warm: relocation reuses the existing warm-handoff move state machine so the working set survives, and the production caller that state machine still lacks (named in spec #78) becomes part of this work. Enterprise keeps the current property of never being archived by inactivity; Pro keeps its existing 90-day archival.
  • Expansion triggers on absolute demand, not share. When qualifying runs mapped to a candidate region that does not yet serve the account sustain an absolute per-day floor for 14 consecutive days, the placer provisions an instance there. Absolute rather than share-based on purpose: a heavy secondary CI site deserves a local instance even when the primary site dominates the account’s mix, and a share threshold would let a large primary permanently veto a second region. The floor prices what a warm local instance is worth (the cross-region round trip saved, per run, per day) against what it costs (a storage slot and admission weight in that region, per spec #78’s placement model).
  • The mechanism is shared; the conditions are per-plan. Pro and Enterprise use the same expansion and retirement machinery, but the floors and windows are per-plan configuration, calibrated separately during the shadow phase. Pro’s conditions for creating a new instance can be more lenient than, or simply different from, Enterprise’s; nothing in the design assumes the two plans expand at the same rate, only that both expand by measured demand rather than by request. Pro’s floor derives from the same slot-economics formula with a lower starting value, and it deliberately does not couple to billing state (decided): gating expansion on measured usage revenue would make placement depend on the commercial system’s view of the account, against the principle that billing facts stay separate. Spec #77’s pricing, once set, informs the floor’s value at the next calibration, not the mechanism.
  • Retirement is the same trigger with hysteresis. A secondary instance whose region’s demand stays below a much lower floor for 90 consecutive days is drained and archived. The gap between the two thresholds plus the long window prevents flapping. Retirement discards that region’s cached content, which is safe for the same reason cold provisioning is: a miss rebuilds.
  • A new secondary warms from its peer, not from misses. It joins the account’s replication mesh and pulls from the existing instance; spec #82 keeps the recurring cost of that sync proportional to genuinely new content. Instances serving the same account may temporarily hold different subsets without correctness risk, because content is content-addressed and presence-based sync converges them.

Claim sizing: the third dimension of the same decision

Placement answers where and how many; the claim answers how big, and today it is the least measured of the three. Regions.storage_profile/1 sizes every instance from a plan constant, and Tuist.Kura.StorageClaims exists because the constants are wrong: its doc calls the plan “a weak predictor of any individual account”, with the spread inside a single plan reaching two orders of magnitude in both directions. The overrides in that table are operators doing by hand exactly what the placer should do from telemetry, the same shape as the manual US West assignments that motivated this spec. So claims stop being plan constants and become placer outputs (decided):

  • The sizing signals already have a design. Spec #78’s eviction analytics (segment bytes and residency duration at eviction) plus exported quota occupancy are the inputs. Sustained short residency with high evicted bytes means the ring is smaller than the working set: the account churns artifacts it just paid to build. Sustained low occupancy means the claim is oversized and the slot is reclaimable.
  • Grow on churn, shrink on idle bytes, with the same hysteresis discipline. Grow when eviction churn is sustained (starting value: median segment residency under 7 days for 14 consecutive days at high occupancy); shrink when occupancy stays low for a long window (starting value: under 40 percent of the claim for 90 consecutive days). The plan constant becomes the starting value rather than the size; the floor is the validated minimum claim, and a per-plan ceiling bounds automatic growth, beyond which sizing returns to an operator override with owner and reason, exactly as today.
  • A resize is a move, which the placer already owns. The local storage class rejects in-place expansion, so growing or shrinking was always a replacement instance; spec #78 said so and the code enforces it. That makes auto-sizing nearly free once the placer exists: a resize proposal flows through the same proposals record, the same supervised-then-automatic ladder, the same warm (paid) or cold (Air) transition, and the same per-pass caps. A shrink through the warm path carries the working set that fits; the tail is evicted content, safe because a miss rebuilds. Air stays effectively static inside a narrow band: its claims are already small, and its cost bound matters more than its fit.
  • Capacity accounting improves as a side effect. Region storage budgets currently reserve constants that are admittedly wrong in both directions. Claims that track measured working sets make the storage dimension of spec #78’s capacity formulas track reality, shrink the reserved-but-empty gap on the big plans, and surface the genuinely large accounts as explicit sizing facts instead of operator folklore.

Routing across instances: the seams already exist

  • The data model already permits several instances per account, one per region; today only the manual picker exercises that.
  • Cache-endpoint resolution returns every active instance for the account, ordered by the caller’s origin (the same in-path coarse mapping, used transiently, nothing persisted), so a client that simply takes the first entry lands near itself.
  • The current CLI already probes returned endpoints and picks the lowest measured latency in selectBestEndpoint, so latency-aware clients refine the server’s ordering locally. Correct behavior requires no client release; newer clients get the sharper pick.

Where the placer lives: the lifecycle we already run

The placer is not a new service and not a Kubernetes operator. Every input it needs is a control-plane fact in Postgres (the demand aggregates, the plan, the residency constraint, pins, budgets, the current instance set), and the downstream intent channel already exists: the provisioner renders a KuraInstance per decision and the reconciler converges it. A cluster-side placer would have to import all of that upstream context through a harder interface, which is the custom-scheduler alternative considered below. The placer is therefore the existing Tuist.Kura.Lifecycle plus a “where” dimension, in four pieces:

  • Origin-aware demand extends Tuist.Kura.Demand. The buffer key widens from the account to the account plus candidate region: the request boundary maps the resolved client address through the static origin table before anything is buffered, so no address ever enters even the in-memory layer. The flush upserts day-grain counters per account and candidate region next to the lifecycle rows; the cardinality is small (hundreds of demand-active accounts, four candidate regions), so the placer’s inputs live in the same database as its outputs. Run ingestion increments the same counters. One assumption dies here: the demand flush currently resolves one region per account, and its own moduledoc says so. The lifecycle-row identity is already account plus service region and supports several rows per account; only the resolution step collapses to one today.
  • A pure decision core next to the lifecycle. Its input is one account’s counter rollup over the policy windows, the plan, the residency constraint, pins, per-region budgets, the current instances with their occupancy and eviction telemetry, and any in-flight move; its output is the desired instance set with claim sizes, plus at most one proposed transition. Every threshold and hysteresis value lives here as configuration. Purity is what makes the shadow phase nearly free (run it against production aggregates, store the output, act on nothing) and what makes threshold calibration table-testable.
  • Actuation stays exactly where it is. The lifecycle tick (reconcile/0) already provisions on first demand and already re-resolves the region at provision time; it starts asking the decision core for the target instead of AccountPolicies.resolve/1. AccountPolicies is demoted, not deleted: it becomes the constraint resolver (residency, pins) the decision core consumes, instead of the placement decider. The daily sweep/0 is where relocation, expansion, retirement, and resize decisions belong, on the same cadence as today’s archival decisions and with the same bounded-actions discipline: the lifecycle already caps provisions and archival transitions per pass, and placement transitions get their own small cap so a miscalibrated threshold cannot move the fleet in one night. Moves execute through the existing warm-handoff move state machine, and source archival is coupled into the move so nothing strands.
  • The shadow, supervised, and automatic phases are one proposals record. The sweep writes proposals; the ops UI renders them. Supervised means an operator confirms a proposal, which enqueues the transition; automatic means the sweep enqueues it directly. It is the same object either way, so graduating between rollout phases is a configuration flip, not a code-path change.

What does not change at all: the reconciler, the provisioner and KuraInstance contract, token exchange, and the CLI. Endpoint resolution changes only its ordering.

The Kubernetes levers: the placer picks regions, the cluster does the rest

The placer’s only novel decision is which account-region instances should exist and at what claim size. Every decision below that line is expressed as a standard Kubernetes scheduling input on machinery the clusters already run, so the day-to-day behavior is mostly configuration on existing levers rather than new orchestration:

  • Node choice is the scheduler’s, already. The provisioner renders a KuraInstance per account-region and pins it to its region through node selection; which box inside the region hosts it is kube-scheduler bin-packing over the plan profile’s processor, memory, and storage requests. capacity.ex documents this division deliberately: control-plane capacity accounting answers “is this region under pressure?”, never “which node?”. The placer inherits that division unchanged; it emits one more instance or one fewer and lets the scheduler pack it.
  • Pools are taints, plans are tolerations. Spec #78’s Air and paid pool separation becomes node taints on the region’s machines, with the instance’s plan supplying the toleration. Funding an Air budget in a region is then a labeling operation on machines, not new placement code, and the placer’s “is this region a candidate for this plan?” check reads the same configuration the scheduler enforces.
  • Best-effort is a priority class. Air instances run at a lower priority class than paid ones, so genuine node pressure resolves by preemption in exactly the direction the product already promises: Air is best-effort and a miss is safe. The eviction the platform performs for free replaces bespoke shed logic, and the lifecycle’s cold-return path is already the recovery for a preempted Air instance. Preemption does not replace the 60-day pressure-archival rule (decided): it reclaims processor, memory, and pod slots, never disk, and Air is storage-bound. Archival stays as the storage lever because only archival deletes the quota-enforced directory, and a preempted pod can only return to the node holding its single-writer volume anyway; the interim is safe because a miss is safe.
  • Machine resilience is anti-affinity. The machine-resilient Enterprise override (two failure-separated instances from spec #78) is pod anti-affinity across failure domains within the region, not control-plane choreography.
  • Healing is the reconciler pattern we already run. Postgres owns intent, the KuraInstance CR carries it, and the reconciler projects status from observed cluster state each tick. A placer decision is a new desired instance set; convergence, retries, and status truth are the existing loop. Node loss recreates instances through the same loop (cold, because the single-writer local volume is gone), and planned maintenance is an ordinary cordon and drain with the warm-handoff move recreating paid instances elsewhere in the region first.
  • Fleet growth is a MachineDeployment scale. Regions are CAPI machine fleets (infra/cluster-api-provider-tuist), where adoption, bootstrap, join, and labeling are already automatic and growing a fleet is scaling a MachineDeployment. The placer’s capacity triggers and unmet-preference counters emit order recommendations; bare-metal lead time keeps a human on the purchase decision, and everything after the machine exists is hands-off. Within already-provisioned headroom, the trigger can drive the scale directly under pre-approved bounds.

What stays control-plane-owned is exactly the list Kubernetes cannot know: which account-region instances should exist (the thresholds), their claim sizes, the residency constraint, operator pins, and the per-account storage quota enforcement on the node filesystem. Everything else (node choice, packing, pool enforcement, preemption, spread, healing, machine lifecycle) is delegated to levers that are already deployed.

Scope

In scope: managed public cache regions; placement, instance-count, and claim-sizing policy for every plan; origin attribution at control-plane boundaries and the per-account aggregates; removing the picker and create and destroy controls and adding the read-only serving view; the ops override surface; the pin-the-present migration; the warm-handoff production caller for paid moves; the pool taints, priority classes, and anti-affinity configuration on the region fleets.

Out of scope: private runner-cache regions (already placed automatically by runner availability) and self-hosted nodes with their free-form region labels; pricing and the commercial model (spec #77); runtime-version rollout (spec #79); procurement mechanics and admission gates (spec #78 keeps owning them; this spec feeds them better inputs); per-request geographic routing inside Kura nodes.

Trade-offs

Advantages

  • Removes a question customers cannot answer, along with both live failure modes of today’s answer path: the silent fallback to the default and the invisible refusal of Europe-restricted Air accounts.
  • Placement tracks reality as CI providers and teams move, without anyone noticing or filing a ticket, and claims track working sets instead of plan guesses that are measurably wrong in both directions.
  • Latency is the product; owning placement means owning the main latency lever with the only complete view of where traffic originates.
  • Procurement stops guessing: demand by origin, quantified unmet placement preferences, and working-set-sized claims are exactly the forecast inputs spec #78’s capacity triggers want, and the “should Air exist in Europe” question gets answered by measured refused demand instead of debate.
  • The runtime cost is low because the decisions delegate to deployed machinery: the scheduler packs, taints partition, priority preempts, CAPI heals and grows the fleets. The new code is the placer’s thresholds and the origin aggregates, not an orchestration layer.
  • The privacy posture persists strictly less than what already transits the control plane: coarse origin, in-path discard, aggregates only, no geographic machinery in Kura nodes.

Disadvantages

  • Tuist becomes accountable for placement quality. A bad placement used to be the customer’s choice; now it is our bug. The operator surfaces and the pin exist so support can see and correct placement in one step.
  • A new measurement surface with privacy obligations, even if coarse. It requires review before shipping and a data-export documentation update.
  • Corporate network egress can misplace origin: a VPN or NAT that exits in another country attributes traffic there. The revealed-preference cross-check catches systematic cases; the operator pin handles stubborn ones.
  • Paid customers lose an explicit knob some buyers ask about in security review, and the serving view deliberately does not name regions. The honest answer is that the compliance question survives as the residency constraint, precise placement is available through support, and contractual placement requirements become operator pins with an audit trail rather than a self-service button.
  • Air in more than one region costs real capacity, which is why Air candidates are budget-gated and unmet preference is a procurement signal rather than an automatic order.
  • Threshold-driven systems can churn, and auto-sizing adds a second threshold family. The two-threshold hysteresis, long windows, minimum-volume floor, relocation cap, and per-plan sizing ceilings are the guards, and the shadow phase exists to calibrate all of them before they act on anything.

Alternatives considered

Keep the picker, improve defaults and onboarding

The data says the setting simply goes unset, and better copy does not make “where does your CI run” answerable. A picker also caps out at preference among regions we already serve; it can never produce the demand signal for a region we do not serve yet, which is exactly the signal the Air-in-Europe decision needs.

Client-measured latency as the placement signal

The CLI already probes latency, but only against instances that exist. Discovering that an account would be better served from a region with no instance requires every client to probe candidate endpoints in every region and report the results: a client release, a new telemetry channel, and fleet adoption time before any placement improves. Server-side origin works for every client version immediately. Client-measured latency stays as the local routing refinement and the calibration cross-check, and is the named upgrade path for the mapping table.

Per-request geographic routing, anycast-style

Kura is terminal storage on specific machines; routing a request to the nearest entry point does not move the bytes it needs. Placement of data, not routing of requests, is the decision that matters.

Replicate every paid account to every region

Cost scales with the region count instead of with demand, and most accounts’ traffic is effectively single-site. The absolute threshold buys the same latency win only where traffic proves it is worth a slot.

A custom scheduler or scoring plugin for cross-region placement

Kubernetes schedules within a cluster and knows nothing about traffic origins, so teaching it cross-region placement means feeding it exactly the aggregates the placer already owns, through a harder interface. The clean split is the one the codebase already has: the control plane decides which instances exist and in which region, and stock scheduling primitives decide everything within the region.

Ask “where does your CI run?” during onboarding

The same unanswerable question with better framing, and the answer silently rots when CI moves. The system should observe instead of interrogate.

Rollout

  1. Privacy review, then shadow measurement. Ship origin attribution and the aggregates with no behavioral coupling. Build the dashboard comparing every account’s current placement and claim with the placer’s recommendation, including the relocations, expansions, retirements, and resizes it would perform. Calibrate the per-plan thresholds and the mapping table against the revealed-preference cross-check. Update the data-export documentation.
  2. Pin the present. Before any resolution default changes, record an explicit operator assignment for every live instance’s region, so existing placements are stated intent rather than accidents of the default chain. This is a hard precondition, not hygiene: today an unpinned resolution change strands the old instance while cold-provisioning a new one.
  3. New accounts go automatic. First provisioning uses origin-based placement. This population has no instances to strand and no expectations to migrate.
  4. Hide the controls. Replace the management card and picker with the read-only serving view. The self-service create and destroy path retires; operator surfaces take over overrides.
  5. Existing accounts converge, supervised. The placer proposes relocations, expansions, retirements, and resizes; an operator confirms each until the move tooling has a clean record, including the warm-handoff production caller for paid moves. Then confirmation flips to automatic with the churn guards active.
  6. Steady state. Placement, expansion, retirement, and sizing run unattended. Unmet placement preferences and origin-segmented demand feed spec #78’s capacity triggers, and measured unmet preference is the general test for proposing any new region (decided): Air budgets for additional regions, and Canada East’s production enablement as the first case, are decided from it rather than from debate.

Rollback at every stage is the operator pin: pinning an account to its current region disables the placer for that account without touching the instance. The regional kill switch and per-account opt-out from spec #78 keep their meaning unchanged.

References

Draft history
Revision Status Edited
Revision 7 Edited by marek@tuist.dev
Proposed
Revision 6 Edited by marek@tuist.dev
Proposed
Revision 5 Edited by marek@tuist.dev
Draft
Revision 4 Edited by marek@tuist.dev
Draft
Revision 3 Edited by marek@tuist.dev
Draft
Revision 2 Edited by marek@tuist.dev
Draft
Revision 1 Edited by marek@tuist.dev
Draft
Comments
P
pedro@tuist.dev Aug 25, 2026

I’m onboard with this direction. Moving placement out of the customer-facing surface and letting the system follow observed traffic feels like the right product model.

A few implementation details are worth making explicit, particularly cold regional relocations, the split between first-demand placement and run-based decisions, and capacity admission. None of those change my view on the proposal.

Sign in to comment

Comments are available to authenticated users.