Iterating on Prompts

Imagine you have a language model task and you need a prompt that works well. One option is to spend an afternoon rewording the instructions by hand, testing each version, and keeping the best one. Many people who hear about prompt engineering tools assume the next step is simple: hand the task to a tool, let it run, and receive a finished prompt that is guaranteed to be the best one available. This article examines that belief and shows why it is incomplete.

The starting point is a fact about the search problem itself. For any task, the number of possible prompts is effectively unlimited, and manual prompt engineering is slow. Automation exists to speed up that search. Automation does not, however, turn an unreliable process into a reliable one. The rest of this article explains the difference.

Suppose you type a request into a model and receive a reply that technically responds but does not do what you wanted. This is an ordinary moment in prompt work, not a sign that you have failed. Writing a prompt is closer to a conversation with an unfamiliar collaborator than to filling in a form correctly on the first attempt. You read what came back, notice where it fell short, and adjust your wording so the next reply moves closer to your goal.

A model's response is best treated as evidence about how your prompt was interpreted. When the reply is off target, the useful question is not only whether the model is wrong but what your wording led it to assume. In the video game case, the model treated a request for a winner as a request for a judgment about contested opinions. A revised prompt that says a pick is wanted even if opinions differ tells the model to commit to an answer while acknowledging the disagreement. The revision is a targeted response to what the first reply revealed.

Iteration is therefore a deliberate cycle. You write a prompt, run it, read the reply, judge it against your goal, and revise. Then the cycle repeats. Stopping is also a decision you make on purpose, when the response meets your goal. A single attempt is rarely the finished instruction, and treating the first version as a draft is what makes the evidence in each reply useful.

Optimization tools take a task description and turn it into a search. You supply the input and output formats, the evaluation metrics, and the evaluation data. The tool then proposes candidate prompts, measures each one against your metrics on your data, and keeps the candidates that score best. Some tools search over a single prompt, and others search over a chain of prompts that are run in sequence.

A helpful comparison comes from automated machine learning. Automated ML tools search for the best hyperparameters of a classical model, such as a learning rate or a tree depth, by trying settings and scoring them. Prompt optimization tools do the same kind of search, except the thing being tuned is the text of a prompt rather than a numeric setting. The search ends with a prompt, and that prompt is only as trustworthy as the metrics and data behind its score.

A second route to automation uses the language model itself as the author of prompts. The simplest version is to ask a model directly for a prompt. For example, you could ask it to help write a concise prompt for an application that grades college essays on a scale from 1 to 5. Models can also be asked to critique a prompt you already have, to improve it, or to generate in-context examples that you then include in the prompt.

These uses have the same underlying idea: the model reads a prompt as text and produces a changed version of that text. A model's critique is also a kind of evidence, but it is evidence you must still check. A suggested rewrite is a candidate, and it earns a place in your work only after it is tested on your data.

Promptbreeder is an example of an AI-powered optimization tool built on an evolutionary strategy. Its approach is to selectively breed prompts, so that good prompts produce variations and the search gradually moves toward prompts that satisfy your criteria. The process begins with an initial prompt. An AI model then generates mutations of that prompt. The mutation step is guided by a set of mutator prompts, which shape how the model changes the text.

The tool takes the most promising mutation, generates mutations of it, and repeats. The loop stops when a prompt satisfies your criteria. Read this as a loop rather than a straight line. The tool does not generate one prompt and stop. It keeps returning to the mutation step, using the most promising candidate so far as the new parent, until the stopping condition is met.

A tool that searches for prompts can fail you in three main ways. The sections below take them one at a time, because each one needs a different kind of attention.

The pitfalls point toward a set of practices. The first is to follow the keep-it-simple principle: before using any tool, write your own prompts without one. This builds a clearer understanding of the underlying model and of what your requirements actually are, and that understanding lets you judge a tool's output when you do use one.

Refinement goes faster when you understand the model you are working with. Each cycle has five parts: you write a prompt, run it, read the reply, judge whether the reply meets your goal, and revise the prompt based on what the reply showed. The stopping point is a decision you make, not something that happens automatically after one attempt.

Models differ from one another, and those differences show up in what they handle well and where they need more guidance. The source notes that models have distinct tendencies, such as differing strengths and preferences for where instructions appear in a prompt. Learning these tendencies is part of the work, because a prompt that succeeds on one model can underperform on another.

This knowledge feeds back into prompt design. When you notice that a model responds better to an instruction placed in a certain position, or that it hedges on a particular kind of request, that observation shapes the next revision. Each revision becomes more informed, so later prompts start closer to the goal than your first draft did.

Once you start revising prompts, a second problem appears: it becomes hard to know which change caused which result. The source recommends making changes in a systematic way rather than editing freely and hoping for the best. It names three supporting practices.

The practice comes down to one controlled comparison. You save a new version, run it and the previous version against the same evaluation data, and record the results so they can be compared against a baseline. The test data and metrics stay fixed while the prompt changes, so any difference in score can be attributed to the prompt.

A prompt is often one part of a larger system, and improvement in one place can hide damage somewhere else. A prompt might improve the model's performance on one subtask while making the whole system perform worse. A change that looks like a win when you examine one behavior should still be checked against the outcome the system as a whole needs to produce.