AI gateway security features: a CISO buyer checklist

What security features should an AI gateway have?

An enterprise AI gateway should provide one enforceable boundary for approved application and agent traffic. It should identify the caller, protect provider credentials, inspect requests and responses, enforce model and tool access, cap abusive consumption, preserve tenant boundaries, and record enough evidence to investigate a decision.

The strongest evaluation covers seven areas: identity, credential handling, request and response controls, agent and MCP permissions, spend and rate limits, audit delivery, and deployment resilience. Buyers should test these controls against the release they plan to use instead of treating a feature list as proof.

1. Caller identity and least-privilege access

Every application, agent, environment, and human administrator needs a separate identity. Look for workload keys or federated workload identity, SSO, lifecycle provisioning, fixed or custom roles, immediate revocation, and a clear separation between customer and vendor staff access.

Test whether one identity can cross a tenant, route, environment, or administrative boundary. A gateway should deny a request when required identity or policy context is missing; it should not guess or fall back to broader access.

2. Credential and data protection in both directions

Provider credentials should stay encrypted at the gateway boundary and out of application code. Request controls should detect configured injection, jailbreak, secret, personal-data, and organization-specific patterns before forwarding. Response controls should independently detect, redact, or block sensitive output before it reaches the caller.

Ask what the gateway inspects, retains, exports, and sends to another service. Full prompt retention is not a prerequisite for routine governance. Metadata can explain identity, policy, route, timing, usage, cost, and outcome while prompt and response storage remains off unless an authorized owner makes a separate choice.

3. Agent and MCP tool authorization

Agents add a second control plane: the tools they can call. A useful boundary identifies the agent and execution mode, inventories offered tools, restricts tool names and types, limits approved MCP hosts, and rejects unoffered or unauthorized calls before release.

Look beyond a simple allowlist. Mature controls should address arguments, action risk, delegation, session behavior, approval for consequential actions, and evidence that connects a model request with its later tool use. Ask which of these checks are available now and which remain planned.

4. Abuse controls with hard ceilings

Rate limits should cover requests, tokens, concurrency, and provider capacity. Financial controls should reserve a conservative amount before dispatch so a runaway loop or stolen workload key cannot continue spending after it reaches its limit.

Test concurrent requests at the boundary rather than a single request after the limit. Also test retry amplification, provider throttling, automatic reload ceilings, ambiguous upstream outcomes, and whether a fallback can increase cost or weaken policy without approval.

5. Investigation evidence and security delivery

For one request, an investigator should be able to identify the caller, configuration version, policy decision, eligible route, provider attempts, tool boundary, timing, usage, cost basis, and final outcome. Administrative changes should be linked and tamper-evident without requiring a warehouse of sensitive prompt text.

Confirm how evidence reaches existing operations. A generic signed HTTPS receiver can support a customer-owned integration, but it is not the same as a native Splunk, Microsoft Sentinel, Google Security Operations, Datadog, object-storage, syslog, or SOAR adapter. Record that distinction during procurement.

6. Tenant, deployment, and resilience boundaries

Attempt cross-tenant identifiers, keys, routes, caches, stored context, exports, and support access. Document the encryption context and the ownership of cloud accounts, network paths, logs, backups, upgrades, quotas, and incident response.

If customer-controlled deployment is required, verify that the product is actually available in a private VPC, dedicated account, Kubernetes cluster, or on-premises environment. For every deployment model, test provider failure, shared health state, bounded retry, approved fallback, backup restoration, and a clear failure when no compliant destination remains.

7. Release evidence and independent assurance

Ask for evidence tied to the exact deployed revision: source and dependency checks, infrastructure review, active web security tests, artifact signatures, configuration inventory, authentication journeys, provider canaries, billing tests, and backup or restore results.

Automated engineering checks are useful, but they do not become an independent penetration test, SOC 2 examination, compliance certification, contractual service level, or external security opinion. A credible vendor labels each category accurately and publishes unresolved gaps.

What does AI Gateway HQ support today?

The hosted service currently provides tenant-bound workload keys, encrypted provider credentials, fixed roles, self-service OIDC or SAML, SCIM lifecycle controls, model and route policy, local request and response signals, tool and MCP declaration checks, hard budget and traffic limits, health-aware approved fallback, metadata-focused request evidence, and signed administrative-event delivery to customer-owned HTTPS receivers.

Prompt and response bodies are not written to the operating ledger by default. A release-candidate buyer-owned AWS path is limited to one documented private Bedrock deployment and still needs external acceptance before general availability. Current gaps include native SIEM and SOAR adapters, mTLS and federated workload identity, cross-session behavioral detection, broad shadow-AI discovery, multi-region private deployment, an independent penetration test, and completed SOC 2 assurance.

Does an AI gateway replace a DLP, SIEM, or identity platform?

No. The gateway should enforce model-specific controls and send useful evidence to the systems a company already operates. It can reduce duplicate policy code in applications, but it does not replace endpoint protection, enterprise DLP, identity governance, SIEM or SOAR operations, application security, or accountable incident response.

Should an AI gateway store every prompt for security?

Not by default. Content may be needed for a specifically authorized investigation or evaluation, but routine governance can use identity, route, policy, signal, timing, usage, and outcome metadata. Buyers should require explicit retention settings, access control, encryption, deletion behavior, and an audit record for any content-storage decision.

What should a CISO ask vendors to show?

Ask each vendor to run the same bounded scenarios and show the resulting evidence. Use synthetic data first, record current and planned status separately, and name the owner and decision date for every unresolved exception.

• A revoked workload identity and an attempted cross-tenant request.
• A prompt attack, an outbound secret, and a sensitive-data response rule.
• An unauthorized tool call and an unapproved MCP host.
• Concurrent requests at a hard token, rate, and spending limit.
• A provider outage with one approved fallback and then no eligible fallback.
• A request investigation and export to the customer's security stack.
• The exact evidence for the deployed release and every external-assurance gap.