Consumers

A consumer is the calling application’s identity — create it in the console to set routing, credentials, and model policies, then copy Connect snippets for your apps.

A consumer is the runtime identity of a calling application. It holds credentials and routing configuration. Its slug is the first path segment on the proxy URL shown in the Connect tab.

Each consumer belongs to one gateway and has a type:

Type	Traffic
LLM	Chat / Responses / Messages / embeddings / images / files / rerank / models (default).
MCP	MCP tool endpoints.

Consumers list

Open Agent Gateway → Consumers.

Capability	Notes
Search / filter	By name; protocol filter LLM / MCP.
Columns	Name, Protocol, Auth method, Policies, Status (Active / Paused), Created.
Row actions	Open detail, edit, delete.
Pause / Resume	Temporarily stop authentication without deleting.

Create a consumer

1. Open Consumers → New Consumer.
2. Work through the create tabs:
   • General — Name, Protocol (LLM or MCP), optional initial target registry.
   • Auth — method + entity (see below). Copy API keys when shown (once only).
   • Routing — mode + strategy + registries/models (or MCP target).
   • Policies — optional Add Policy or Clone policies from another consumer.
3. Save. The success step and the detail Connect tab show ready-to-use snippets.

Detail tabs

Tab	Purpose
General	Name, protocol, slug / route metadata.
Auth	Link Identity auth entities; create/regenerate API keys (LLM).
Routing	Routing mode, strategy, registries, smart tiers, fallback, roles.
Policies	Attach, detach, clone targeted policies (globals still apply).
Connect	Base URL, headers, model hints (auto vs explicit), client snippets.

Routing mode

Mode	UI label	Behavior
inline	Static	Strategy, registries, fallback, and model policies live on the consumer.
role_based	Identity-based	Roles selected from OIDC/OAuth claims decide registries and models. Attach roles under Routing; manage role definitions under Identity.

Strategies (Static routing)

Strategy	When to use
Simple routing	Send traffic to one or more registries without load balancing.
Fallback	Try registries in order until one succeeds.
Smart routing	Route by complexity label (Simple / Medium / Hard) across tiers and models (recommended multi-model path).
Round robin	Cycle evenly across pool members.
Weighted	Send more traffic to members with higher weights.
Least connections	Prefer the member with the fewest in-flight requests.
Random	Pick a pool member at random.

Load-balancing strategies (including smart routing) are grouped under Load balancing in the Strategy dropdown; Simple routing and Fallback under Direct.

MCP consumers always bind registries/tools directly — they do not show the Strategy selector or load-balancing algorithms.

Providers editor

For simple routing and non-smart load balancing, use Add Registry, set optional weight (weighted strategy), choose all models/tools or restrict via the model browser, and set a default model when needed.

Smart routing

Use Complexity tiers (at least two): Simple / Medium / Hard, each with a registry and model. Same registry can appear on multiple tiers with different models.

Details: Load balancing, Smart routing, Model resolution, Fallback.

Model policies

For each bound registry you can:

• Allow all models, or filter to a subset.
• Set a default model used when the client does not name one.

When the consumer uses load balancing or smart routing, Connect snippets use "model": "auto" so the gateway picks the registry and model. With multiple registries and no load balancing, clients must specify the model themselves.

Connect tab

Open a consumer → Connect for:

• Gateway host (SaaS or your Private LLM URL)
• Consumer slug
• Auth header examples (X-AG-API-Key, and X-AG-Gateway-Slug on Private)
• Language snippets (cURL, Python, Node, …)
• Same host and key for embeddings, images, files, rerank, and models

The provider credential remains in TrustGate. Keep the consumer API key secret.

Edit routing later

Open the consumer → Routing to change strategy, registries, smart-routing tiers, fallback chain, or identity-based roles without recreating the consumer.