Skip to main content
A guardrail inspects the traffic passing through the gateway and either lets it through, refuses it, or rewrites it. Six of them are available, and they differ in ways that matter before you pick one.

​
Streamed responses

TrustGuard, OpenAI Moderation, Google Model Armor and Regex Replace check a streamed response as it is generated. The gateway holds each stretch of text briefly, checks it, and only then lets it through. A violation stops the stream at that point, but the text released before it has already reached the reader. A response short enough to fit in the hold is refused with 403 before streaming starts. TrustGuard also holds streamed tool calls when it inspects the response. On an OpenAI-shaped Chat Completions stream, and on Mistral, it keeps the arguments of one choice whole until the verdict, then releases the held chunks together. OpenAI Moderation, Google Model Armor and Regex Replace leave argument fragments to stream as they arrive. What TrustGuard refuses, and the hold cap, are on the TrustGuard page. AWS Bedrock Guardrail does not check streamed responses, and Azure Content Safety checks requests only. Put them on the request leg, or rely on non-streamed responses, when you need them to stop something.

​
Choosing one

Four questions usually settle it. Do you already run TrustGuard? Then reach for its guardrail. Its findings line up with Telemetry alerts, so a detection here becomes part of the same picture as everything else you watch. No provider’s moderation API can do that. Do you need the content cleaned rather than the request refused? TrustGuard, Bedrock and Model Armor can rewrite the content in place and let the call continue. Azure and OpenAI Moderation return a score, so all they can do is allow or refuse. Is the rule something you can write down exactly? An internal hostname, a ticket reference, a key prefix — that is Regex Replace. It calls nothing, scores nothing, and does exactly what the pattern says. For anything needing judgement, you want one of the others. Do you already own the policy somewhere else? Bedrock and Model Armor reuse a guardrail or template you built in your own cloud account, so the definition lives in one place rather than two. That is their real argument, and the cost is a setup that happens in the provider’s console before this one is any use.

​
Two things worth knowing before you commit

A guardrail that cannot be reached does not stop your traffic, but content it cannot check does. When the provider is unreachable, too slow, rate limiting, or rejects the credentials, or the policy’s own configuration is the problem, OpenAI Moderation, Azure Content Safety, Bedrock and Model Armor forward the call uninspected and record it as Failed Open, in Enforce as well as Observe. The trade is that a provider outage leaves that traffic unguarded rather than refused, so watch for failed_open in the event if the guard is something you rely on. TrustGuard does the same by default, but its If the guardrail cannot evaluate field can be set to Block, which rejects those requests in Enforce and only records them in Observe. See TrustGuard. When the content itself is the cause, such as content larger than the provider accepts, a body that cannot be read, or a mask that cannot be applied over sensitive data the guard found, Enforce refuses the call with 403 and records it as Failed Closed (or blocked, for a mask). Observe only records it and never blocks. Requests on routes that carry no chat (images, audio, files) and upstream error responses are not inspected by any of them: the call is forwarded and recorded as not inspected, never refused. Azure never sees the model’s answer. It runs on requests only. If what you need to inspect is what the model said, this is not the one.

​
Combining them

A chain may hold several, and they run in policy-priority order. TrustGuard on the request and a Bedrock guardrail anonymising sensitive data on the response is a common pair: the first refuses what should not go out, the second cleans what comes back. A guardrail that rewrites content changes what any guardrail after it sees.