One stable endpoint
Applications use familiar OpenAI and Anthropic interfaces while administrators change providers, accounts, models, and limits centrally.
Resilient model delivery
Route for cost, capability, and continuity—not guesswork.
Build compatible provider pools, distribute traffic across credentials, and fail over only when policy says the alternative is safe.
Priority, weighted, request-cost, and health-aware strategies
Bounded retries, fallback, and circuit availability
Capability-aware fallback that preserves required API contracts
In plain language
Your applications keep one endpoint while your options keep changing.
Instead of embedding a provider decision and API key in every application, administrators define reusable route pools. A pool states which models and credentials are eligible, how they should be ranked, and exactly when a fallback is allowed. Builders call a stable gateway alias; operators can change the supply path without asking every team to redeploy.
The practical resultProvider outages, exhausted quotas, and price changes become controlled operating events rather than emergency code changes.
For the operating team
A route is a documented decision tree, not a hidden algorithm.
Platform and operations teams control eligibility first, optimization second, and fallback last.
What administrators configure
Compatible targets
Specify provider, account, model, region, capabilities, context limits, and streaming support.
Selection strategy
Choose priority, weighted distribution, health-aware ordering, or estimated request cost.
Failure boundary
Cap attempts, time, cost, cooldown, and capability changes before a fallback is permitted.
Exact-cache boundary
Opt in per route to short-lived, workload-isolated repeat responses without durable prompt or response storage; streaming and tool calls bypass it.
Change recovery
Compare retained route releases and restore a known-good configuration as a new, reason-coded version that must pass current safety checks.
What happens to each request
- 01Authenticate
Resolve the tenant, workload key, user context, and client mode before considering a provider.
- 02Constrain
Remove targets that violate model, capability, budget, risk, or environment policy.
- 03Select
Rank eligible credentials and models by the configured priority, weight, health, or estimated request cost.
- 04Reconcile
Serve a governed exact-cache hit or call the selected provider, then record the decision and reconcile reserved budget.
See the working control
Troubleshoot from the decision, not from a prompt archive.
The live ledger shows workload, route, reason code, upstream attempt, outcome, latency, configured cost, and audit-chain integrity.
- Rendered by the real customer console and control API
- Exercised with safe OpenAI- and Anthropic-compatible simulators
- Sanitization gate rejects credential-shaped values and private owner email
The live ledger shows workload, route, reason code, upstream attempt, outcome, latency, configured cost, and audit-chain integrity.
Control surface
Useful on day one. Explainable on day one hundred.
Each control has an operating path, an owner, and evidence that can be reviewed without collecting prompt bodies by default.
Every attempt is accountable
The ledger records the requested model, selected target, retries, fallback reason, latency, reserved estimate, supported provider-reported settlement, and final result without retaining prompt bodies.
Availability without accidental spend
Hedging stays off by default. Retries and fallbacks have explicit time, attempt, cost, capability, and data-location boundaries.
Evidence, not assertions
Explain why this target handled this request.
The request ledger separates requested model, selected target, retry, fallback, upstream outcome, gateway latency, reserved estimate, and supported provider-reported settlement.
Review security boundaries and current status- Reason-coded policy, route, retry, fallback, and denial metadata
- Bounded tool names, types, MCP hosts, and authorization outcomes—without schemas, arguments, or results
- Provider-reported usage reconciliation for complete responses and supported terminal streams
- Tenant-bound credential encryption and fail-closed tenant authorization
- Signed, retryable administrative audit delivery with payloads and configuration snapshots excluded
- Clear labels separating available controls from items that still require validation
Questions teams ask
Know the boundary before you deploy.
Clear answers for buyers, administrators, and security reviewers.
Will a fallback silently remove a required capability?
No fallback is eligible unless its configured capability profile satisfies the request. Administrators still need to validate exact provider and model behavior before production use.
Can we balance several keys for the same provider?
Yes. Credentials can participate in a pool and be selected under priority, weighted, health, quota-cooldown, or request-cost rules without exposing the underlying key to the client.
Does cost routing always pick the cheapest model?
No. It picks the least-cost target remaining after policy, capability, context, budget, and health constraints. A cheap but ineligible target is never considered.
Can repeated requests avoid another provider charge?
Yes, when an administrator enables the exact-response cache for that route. Identical non-streaming JSON requests from the same workload key can reuse a bounded volatile response for 30 seconds to one hour. Current budgets, rate limits, platform billing, and response guardrails still run; tool-bearing and idempotent requests bypass it.
Can an operator undo a bad routing change?
Yes. The route console compares retained releases and can restore an earlier configuration. The restore creates a new version, records who did it and why, preserves the route's current active or disabled state, and is rejected if the older configuration no longer satisfies current credential, provider, model, or tenant safety checks.
Start safely
Start guided setupSee what the policy would do before it can block production.
Connect a provider credential, create a workload key, and begin in Observe mode. Move a tested rule to Enforce when your team is ready.