Hard limits that are actually hard
Strict mode performs atomic reservations before inference and reconciles estimates after usage is known.
AI FinOps
Make budget policy part of every model decision.
Reserve spend before a request, reconcile actual usage afterward, and degrade, reroute, alert, approve, or block at the scope that owns the budget.
Request, token-cap, provider-cost, and prepaid hard limits
Organization and virtual-key budget scopes
Provider-reported token reconciliation for complete responses and supported terminal streams
In plain language
Decide whether the request can be afforded before the provider charges for it.
Dashboards explain yesterday's invoice. Hard controls must act before inference. AI Gateway HQ estimates exposure, reserves against the applicable budget or prepaid balance, selects from eligible routes, and settles the reservation when supported usage data returns.
The practical resultTeams can receive a graceful limit or approved downgrade instead of creating an uncapped vendor invoice during a traffic spike.
For the operating team
Give every dollar an owner and every limit an action.
FinOps defines the boundary; platform teams define which technically eligible alternatives can operate inside it.
What administrators configure
Budget scope
Set request, rate, token, provider-cost, organization, workload-key, and prepaid limits.
Threshold action
Alert, cap, reroute, degrade, require a different workflow, or stop before forwarding.
Rate catalog
Maintain explicit provider/model rates used for estimates and examine variance against supported reported usage.
What happens to each request
- 01Estimate
Estimate the maximum gateway and provider exposure from the request and configured output cap.
- 02Reserve
Atomically reserve against the applicable organization, workload, and prepaid limits before forwarding.
- 03Optimize
Choose the least-cost eligible target only after capability, policy, and health requirements pass.
- 04Settle
Reconcile the reservation against reported usage and preserve a reason-coded ledger entry.
See the working control
Put the stop condition next to the number.
Organization and workload-key limits are reserved before provider traffic instead of reported only after the bill arrives.
- Rendered by the real customer console and control API
- Exercised with safe OpenAI- and Anthropic-compatible simulators
- Sanitization gate rejects credential-shaped values and private owner email
Organization and workload-key limits are reserved before provider traffic instead of reported only after the bill arrives.
Control surface
Useful on day one. Explainable on day one hundred.
Each control has an operating path, an owner, and evidence that can be reviewed without collecting prompt bodies by default.
Rank eligible cost before execution
Cost-optimized pools estimate each compatible target from request size, output cap, and administrator-configured rates before forwarding.
No BYOK inference markup
Customers using their own authorized provider credentials retain their provider pricing. Optional managed Bedrock access is separately prepaid at a visible server catalog rate.
Evidence, not assertions
Trace reservation, route estimate, and settlement.
The tenant console allocates the latest retained request evidence by workload, environment, client mode, data class, provider account, route, and model while separating exact and conservative settlement. Durable period reporting, provider invoice import, and variance automation are not currently included.
Review security boundaries and current status Use the AI cost-control checklist- Reason-coded policy, route, retry, fallback, and denial metadata
- Bounded tool names, types, MCP hosts, and authorization outcomes—without schemas, arguments, or results
- Provider-reported usage reconciliation for complete responses and supported terminal streams
- Tenant-bound credential encryption and fail-closed tenant authorization
- Signed, retryable administrative audit delivery with payloads and configuration snapshots excluded
- Clear labels separating available controls from items that still require validation
Questions teams ask
Know the boundary before you deploy.
Clear answers for buyers, administrators, and security reviewers.
Can a request overshoot a strict budget?
Strict mode reserves an estimate atomically before forwarding. Estimation and provider reporting can still differ, so output caps and conservative rate configuration remain important.
Do you mark up inference when we bring our own keys?
The published BYOK model does not add an inference markup. Gateway request fees and any optional managed-model credits are separate and visible.
Can we automatically buy more credit?
Yes. An account owner can buy prepaid credit through Stripe-hosted checkout, then separately opt into an amount, threshold, and hard monthly auto-reload cap. Auto-reload stays off until the owner configures it.
Start safely
Start guided setupSee what the policy would do before it can block production.
Connect a provider credential, create a workload key, and begin in Observe mode. Move a tested rule to Enforce when your team is ready.