Quality failures in model outputs

A language model does not produce its answer the way a calculator produces a sum. Its reply is generated text, and that text can go wrong in many different ways. It might come back empty, break the format an application depends on, state something untrue, or contain content that no product should show to a customer. An application that passes every reply straight through to the user is trusting the model completely, and that trust is rarely justified in a production system.

An output guardrail is the checkpoint placed between the model and the user. Its purpose is to examine what the model produced before the application acts on it. A guardrail is not a model improvement and does not make the model smarter. It is application logic that inspects results and decides what should happen when a result is not acceptable.

The guardrail does not need to know how the model works internally. It only sees the output and the expected standard. This is why guardrails can be added to an application without changing the model itself.

A guardrail has two responsibilities, and they depend on each other. The first is detection: the guardrail must catch outputs that fail to meet the standards the application sets. The second is response: once a failure has been caught, the guardrail must apply a policy that says what to do about that particular kind of failure. Detection without a policy leaves the application stuck with a bad reply. A policy without detection never gets triggered.

It helps to picture a response moving through a sequence of decisions. The model's output enters the guardrail, which runs its checks. A response that passes goes on to the user. A response that fails is matched to a failure category, and the policy for that category determines the next step. The diagram below shows this route. Pay attention to the fork at the check, because the same response can take two very different paths depending on what the check finds.

Read the diagram as a route, not a single step. A passing response leaves the guardrail immediately. A failing response is classified, and the category's policy decides whether it loops back for another attempt or leaves the automated path entirely. The new attempt returns to the same check, so every retry is inspected again before it can reach the user.

Failures look different across applications, so the first skill in designing a guardrail is recognizing what failure looks like for a specific product. The source sorts common failures into two families. Quality failures concern whether the output is good and correct. Security failures concern whether the output is safe and whether it exposes something it should not.

Detection tells the guardrail that something went wrong. The policy tells it what to do. Each failure mode needs its own policy, because a empty reply and a toxic reply should not be handled the same way. The map below pairs example failure modes with an example policy for each. The pairings are illustrations of the design decision, not fixed rules from the source.

Many failures can be reduced with simple retry logic. The reason it works is that AI models are probabilistic. Sending the same query again may produce a different response. If the reply came back empty, the application can try again a fixed number of times or until a nonempty reply arrives. If the reply was malformatted, it can keep trying until the format is correct. A retry policy is therefore a rule that converts a failure into another attempt, with a defined stopping condition.

A retry policy is not free. Every retry is another round of API calls, which means extra latency and extra cost. If the retry happens only after a failure has been observed, the user waits for the first attempt and then for the second, so the latency the user perceives can roughly double. Parallel calls avoid waiting, but they add redundant API calls, which still cost money.

Retries assume another model response will be good enough. For some requests that assumption is weak, and the policy should send the conversation to a person instead. Human fallback handles tricky requests, and its trigger is only as good as the detection behind it. If the detection misses a tricky request, no fallback will ever run.

Tightening a guardrail to block more security failures sounds like pure improvement. But every check that flags more content also flags some content that was acceptable. A reply that was safe and useful gets refused, and the user receives a fallback they did not need. This is a false refusal. The guardrail refused something it should have passed.

This is why both numbers have to be measured together. A team that tracks only security failures can keep tightening the check and never notice that the application is refusing more and more acceptable replies. Tracking the two side by side shows whether a stricter policy is actually improving the product or only shifting the errors.