Hard limits that are actually hard
Strict mode performs atomic reservations before inference and reconciles estimates after usage is known.
AI FinOps
Make budget policy part of every model decision.
Reserve spend before a request, reconcile actual usage afterward, and degrade, reroute, alert, approve, or block at the scope that owns the budget.
Request, token-cap, provider-cost, and prepaid hard limits
Organization and virtual-key budget scopes
Provider-reported token reconciliation for complete responses and supported terminal streams
In plain language
Decide whether the request can be afforded before the provider charges for it.
Dashboards explain yesterday's invoice. Hard controls must act before inference. AI Gateway HQ estimates exposure, reserves against the applicable budget or prepaid balance, selects from eligible routes, and settles the reservation when supported usage data returns.
The practical resultTeams can receive a graceful limit or approved downgrade instead of creating an uncapped vendor invoice during a traffic spike.
For the operating team
Give every dollar an owner and every limit an action.
FinOps defines the boundary; platform teams define which technically eligible alternatives can operate inside it.
What administrators configure
Budget scope
Set request, rate, token, provider-cost, organization, workload-key, and prepaid limits.
Threshold action
Alert, cap, reroute, degrade, require a different workflow, or stop before forwarding.
Rate catalog
Maintain explicit provider/model rates used for estimates and examine variance against supported reported usage.
What happens to each request
- 01Estimate
Estimate the maximum gateway and provider exposure from the request and configured output cap.
- 02Reserve
Atomically reserve against the applicable organization, workload, and prepaid limits before forwarding.
- 03Optimize
Choose the least-cost eligible target only after capability, policy, and health requirements pass.
- 04Settle
Reconcile the reservation against reported usage and preserve a reason-coded ledger entry.
See the working control
Put the stop condition next to the number for cost management.
For cost management, organization and workload-key limits are reserved before provider traffic instead of reported only after the bill arrives.
- This cost management view runs in the working customer console
- Exercised with safe OpenAI- and Anthropic-compatible simulators
- The cost management capture excludes keys, prompt text, responses, and private email
For cost management, organization and workload-key limits are reserved before provider traffic instead of reported only after the bill arrives.
What administrators can change
The settings behind ai finops.
For ai finops, the console shows each setting, its owner, and its scope so administrators can review changes before they reach live traffic.
Rank eligible cost before execution
Cost-optimized pools estimate each compatible target from request size, output cap, and administrator-configured rates before forwarding.
No BYOK inference markup
Customers using their own authorized provider credentials retain their provider pricing. Optional managed Bedrock access is separately prepaid at a visible server catalog rate.
Evidence, not assertions
Trace reservation, route estimate, and settlement.
The tenant console allocates the latest retained request evidence by workload, environment, client mode, data class, provider account, route, and model while separating exact and conservative settlement. Durable period reporting, provider invoice import, and variance automation are not currently included.
Review security boundaries and current status Use the AI cost-control checklist- Budget scope: Set request, rate, token, provider-cost, organization, workload-key, and prepaid limits.
- Threshold action: Alert, cap, reroute, degrade, require a different workflow, or stop before forwarding.
- Rate catalog: Maintain explicit provider/model rates used for estimates and examine variance against supported reported usage.
Questions teams ask
Questions about ai finops.
What buyers, administrators, and security reviewers need to confirm for ai finops.
Can a request overshoot a strict budget?
Strict mode reserves an estimate atomically before forwarding. Estimation and provider reporting can still differ, so output caps and conservative rate configuration remain important.
Do you mark up inference when we bring our own keys?
The published BYOK model does not add an inference markup. Gateway request fees and any optional managed-model credits are separate and visible.
Can we automatically buy more credit?
Yes. An account owner can buy prepaid credit through Stripe-hosted checkout, then separately opt into an amount, threshold, and hard monthly auto-reload cap. Auto-reload stays off until the owner configures it.
Start safely
Start guided setupSee what the policy would do before it can block production.
Start with a provider connection, a limited workload key, and Observe mode; move to enforcement after your team confirms the result.