One cost vocabulary
Connect request, token, model, provider account, configured rate, workload, and execution context in an explainable metadata ledger, then inspect the latest retained window in the customer overview.
For FinOps and finance
Attribute AI spend before the invoice arrives.
Reserve provider exposure before dispatch and attribute recent gateway-observed usage across operational owners.
Recent cost ownership by workload, environment, client, data class, account, route, and model
Least-cost eligible routes, budget limits, and prepaid hard stops
Provider-reported token reconciliation for complete responses and supported terminal streams
In plain language
Attach an owner and an action to AI spend before it becomes an invoice line.
AI Gateway HQ attributes modeled and supported provider-reported usage to the workload, environment, client mode, data class, provider account, route, and model that caused it. Strict limits reserve budget before forwarding, while cost-ranked routes consider only models that already satisfy technical and policy requirements.
The finance outcomeBudget conversations can happen at the workload boundary, with a defined downgrade or stop, rather than after an unexplained provider bill.
For the operating team
Model the limit where responsibility actually sits.
Organization and workload scopes separate centrally funded use from product, team, environment, or customer-specific consumption.
What administrators configure
Ownership
Name the organization, workload, administrator, environment, and provider account behind usage.
Consumption policy
Combine monthly cost, request, RPM, TPM, concurrency, prepaid, and output-cap controls.
Unit economics
Break down the latest retained request window across seven operational dimensions, with exact usage and conservative settlement visibly separated.
What happens to each request
- 01Estimate
Estimate the maximum gateway and provider exposure from the request and configured output cap.
- 02Reserve
Atomically reserve against the applicable organization, workload, and prepaid limits before forwarding.
- 03Optimize
Choose the least-cost eligible target only after capability, policy, and health requirements pass.
- 04Settle
Reconcile the reservation against reported usage and preserve a reason-coded ledger entry.
See the working control
Put the stop condition next to the number.
Organization and workload-key limits are reserved before provider traffic instead of reported only after the bill arrives.
- Rendered by the real customer console and control API
- Exercised with safe OpenAI- and Anthropic-compatible simulators
- Sanitization gate rejects credential-shaped values and private owner email
Organization and workload-key limits are reserved before provider traffic instead of reported only after the bill arrives.
Control surface
Useful on day one. Explainable on day one hundred.
Each control has an operating path, an owner, and evidence that can be reviewed without collecting prompt bodies by default.
Control marginal cost
Free-tier and internal quotas can be enforced without AI Gateway HQ funding customer inference.
Find billing variance
Provider invoice import and variance detection are not currently included; current evidence reports gateway-observed usage.
Evidence, not assertions
Keep the estimate, reservation, and result connected.
Current evidence provides a bounded, tenant-private operational allocation—not an invoice or forecast. Durable periods, provider invoice import, discount schedules, and automated variance detection are not currently included.
Review security boundaries and current status- Reason-coded policy, route, retry, fallback, and denial metadata
- Bounded tool names, types, MCP hosts, and authorization outcomes—without schemas, arguments, or results
- Provider-reported usage reconciliation for complete responses and supported terminal streams
- Tenant-bound credential encryption and fail-closed tenant authorization
- Signed, retryable administrative audit delivery with payloads and configuration snapshots excluded
- Clear labels separating available controls from items that still require validation
Questions teams ask
Know the boundary before you deploy.
Clear answers for buyers, administrators, and security reviewers.
Can we allocate spend by team or product?
Current organization and workload-key scopes can represent a team, product, environment, or customer. Deeper hierarchy and cost-center synchronization are planned.
Does routing account for our negotiated discounts?
Administrators configure the rates used by request-cost routing. Automated provider invoice and contract-rate ingestion are not yet shipped.
What happens when prepaid credit reaches zero?
Strict prepaid mode stops eligible paid requests before forwarding. If the account owner has enabled capped auto-reload, the configured Stripe charge can restore credit; otherwise traffic remains stopped. Managed Bedrock also requires positive purchased, managed-eligible cash; promotional credit cannot fund AWS inference.
Start safely
Start guided setupSee what the policy would do before it can block production.
Connect a provider credential, create a workload key, and begin in Observe mode. Move a tested rule to Enforce when your team is ready.