Public AI Gateway Reference Architecture

Role-based topology; deployment identifiers, endpoints, credentials, and tenant data are omitted.

Public AI Gateway Reference Architecture Role-based topology; deployment identifiers, endpoints, credentials, and tenant data are omitted. API Clients · OpenAI-compatible requests · Architecture component · public contract API Clients OpenAI-compatible requests public contract Secure Edge · TLS, admission, tunnel · Architecture component · trust boundary Secure Edge TLS, admission, tunnel trust boundary AI Gateway · auth, policy, routing · Architecture component · replicated AI Gateway auth, policy, routing replicated Model Providers · provider-native APIs · Architecture component · external Model Providers provider-native APIs external Provider Relay · session normalization · Architecture component · single writer Provider Relay session normalization single writer State Routers · leader/master aware · Architecture component · stable endpoints State Routers leader/master aware stable endpoints Relational State · replicated, quorum-backed · Architecture component · metadata Relational State replicated, quorum-backed metadata In-memory State · replicated, monitored · Architecture component · routing state In-memory State replicated, monitored routing state HTTPS / SSE admitted request normalized request provider call state access leader route master route Legend Backend Database Cloud Security External