Automatic AI provider failover without silent policy drift

Detect the condition

The gateway observes provider outcomes on real governed requests. Rate-limit responses and exhausted quota headers create a target-specific cooldown. Repeated upstream server failures open a short-lived health circuit. Newer successful evidence can clear an older passive circuit without erasing a separate quota cooldown.

Share the decision across instances

Cooldown and circuit records are tenant-scoped, payload-free operational metadata in the shared control store. Another gateway instance can honor the same state before it sends a duplicate request to an already unhealthy target.

Choose only an eligible alternative

The route removes paused, cooling, unhealthy, incompatible, over-capacity, policy-blocked, and over-budget targets before applying priority, weight, or request-cost strategy.

• Attempts and timeouts remain bounded per target.
• The request ledger records the skipped target, fallback reason, and upstream attempts.
• Operators can drain and later resume a target without deleting its configuration.
• A route with no valid alternative returns an explicit error before another provider charge.