Instruction Following in Foundation Models

Many AI applications need more than readable text. A program that reads a model's answer may expect a JSON object with specific keys, a YAML document with a fixed layout, or a CSV row with the right number of columns. If the model's output breaks that format, the downstream program fails, even when the content of the answer is reasonable.

This article looks at the main routes practitioners use to get a model to produce output in an expected shape. Each route intervenes at a different point: the instructions the model receives, the text it produces, the choices it makes token by token, or the model's internal parameters. Keeping those intervention points in mind makes the trade-offs between approaches much easier to reason about.

Think of a response as passing through a pipeline. The prompt goes in, the model selects tokens one at a time to build a finished text, and that text is what a program eventually receives. The model's weights sit underneath every one of those choices, because they are the internal parameters that determine how the model behaves.

Prompting acts at the start of the pipeline, by changing what the model receives. Post-processing acts at the end, by changing the finished text. Constrained sampling acts in the middle, at the token selection step. Finetuning acts on the weights, which changes the choices the model makes everywhere in the pipeline. Each approach is a different answer to the question of where to intervene.

A useful way to sort the approaches is by how much they change the system. Prompting and post-processing make small adjustments and are inexpensive to try, but they offer no hard promise. Constrained sampling and finetuning change how generation itself works or retrain the model, so they demand more effort but can provide stronger guarantees.

Prompting is the cheapest starting point. You write instructions that describe the required format, and the model tries to follow them. Models are steadily improving at following instructions, but prompting carries no guarantee of format. A failure rate of a few percent may sound small, yet it can be unacceptable for an application that processes thousands of outputs.

Some teams add a second model query that checks or repairs the first output. This is an instance of the AI as a judge pattern. The cost is that each output now requires at least two model calls, one to generate and one to validate. The validation layer can raise validity substantially, but the added expense and delay can make it impractical for some systems.

Post-processing takes a different route. It leaves the model alone and repairs predictable errors after generation. If a generated JSON object is missing a closing bracket and the same omission appears again and again, a short script can append that bracket whenever it is missing. A reported case from LinkedIn describes a defensive YAML parser built around known model errors that raised the share of correct YAML outputs from 90% to 99.99%. The lesson is that a parser designed around the errors a model is known to make can recover most of what would otherwise be lost.

Post-processing has a boundary. It works only when the mistakes are easy to fix, which usually means the model's outputs are already mostly well formed and the remaining errors are small and predictable. If the text is badly broken or the errors vary widely, a repair script becomes a fragile collection of special cases.

Constrained sampling works inside the generation loop. At each step, the model proposes a probability for every possible next token, and a grammar filters out the tokens that would make the output invalid. The model then picks from what remains. Because the filter runs at every step, the finished output stays within the format.

The grammar carries the real knowledge. The source gives an example: in JSON, after an opening brace you cannot immediately place another opening brace unless it sits inside a string. The filter must know rules of this kind at every step, which is why the grammar is the heart of the method.

Constrained sampling is only as good as the grammar it relies on. Writing a grammar for a format and connecting it to the sampling process is not trivial. Each format, whether JSON, YAML, regular expressions, or CSV, needs its own grammar. The method is therefore less general than it first appears: it can be used only for formats whose grammars are already supported by external tools or built by your team.

Grammar verification also adds generation latency, so the method trades speed for validity. Some practitioners argue against constrained sampling for this reason, believing that the same resources would be better spent training models to follow instructions more faithfully.

Finetuning is the heaviest approach. It collects examples of the desired format and trains the model, so the model's internal weights change. Prompting, by contrast, leaves the weights untouched. Only the instructions change, and the model's habits stay as they were.

Finetuning can update the whole model end to end, or update only part of it. For classification, a classifier head can be attached to the foundation model so that its outputs are restricted to a fixed list of labels. With three classes, the model can only answer with one of those three. End-to-end training demands more computing resources but tends to offer better performance.

Prompting and finetuning sit at opposite ends of the spectrum. Prompting costs little to set up because it only requires writing instructions, but its reliability depends on the model's existing instruction-following ability. Finetuning requires collecting examples and training, which costs more, yet it changes the model's habits and offers stronger consistency.

Choosing between them starts with a question: how often can a format failure be tolerated? If failures are rare and easy to repair, post-processing or prompting with validation may be enough. If the output must be right and a few percent failure is unacceptable, the heavier methods become worth their cost. Constrained sampling is attractive only when a grammar for the format already exists, because building one is not trivial.