Load balancing

Distribute traffic across a consumer’s registries from the console — smart routing, round-robin, weighted, least-connections, or random.

When a consumer can reach more than one registry, load balancing picks which one serves each request. Configure it on the consumer under Routing → Strategy → Load balancing.

Strategies

Strategy in the UI	How it picks
Smart routing	Routes by prompt complexity across Simple / Medium / Hard tiers (registry + model per label). Recommended multi-model path.
Round robin	Cycles evenly across pool members.
Weighted	Sends more traffic to members with higher weights.
Least connections	Prefers the member with the fewest in-flight requests.
Random	Uniform random pick.

Simple routing and Fallback live under the Direct group in the same Strategy dropdown — they are not load-balancing algorithms. Details for label-based complexity routing are on the Smart routing page.

Configure load balancing

1. Open Consumers → select a consumer → Routing.
2. Set Routing mode to Static.
3. Open Strategy and choose an algorithm under Load balancing.
4. Add pool members (registries / models). For Weighted, set each member’s weight.
5. For Smart routing, configure Complexity tiers (at least two) — see Smart routing.
6. Save.

Connect snippets for load-balanced consumers use "model": "auto" so clients do not need to name a model — see Model resolution.

Weights

For Weighted, each pool member carries a weight from 1 to 100. A member at 70 receives roughly 70% of the share of one at 30. Set weights in the registry / member rows in the Routing editor.

Health checks

LLM registries can define health checks in the Registry editor. Unhealthy registries are skipped by the load balancer until they recover.

Pair load balancing with fallback so a failed pick can retry another path instead of erroring immediately.