Prompt Engineering and Context Construction
Prompt engineering is the process of getting AI models to express desirable behaviors from the input alone, without changing the model weights.
A Model That Seems Fixed
When people first use an AI model, they often assume its performance is fixed by the model itself. Either the model can do a task or it cannot. That view leaves out an important lever: the input you give the model. The way a request is written can change how well the model performs, even when the model itself stays exactly the same.
This article introduces that lever, called prompt engineering. It looks at the evidence for its effect on measured performance, and at a related idea, context construction, which matters when a task is complex and involves long context. Keep one question in mind as you read: what can be changed without retraining the model, and what can that change achieve?
Working From the Input Alone
Prompt engineering is the process of getting AI models to express desirable behaviors from the input alone, without changing the model weights.
The key phrase is from the input alone. A prompt engineer does not reach into the model and adjust its internal settings. The work happens in what is sent to the model: the instructions, the description of the task, and the framing of the request. The right instructions can get a model to perform the task you want, in the format of your choice. Both parts of that statement are about the input. The task and the output format are specified in the prompt, and the model is asked to do what the prompt describes.
The diagram shows the central point. Only the prompt is a variable you control in this flow. The model weights sit in the middle, unchanged, and every output is produced by passing the input through them. Prompt engineering works at the entry point, and its reach is determined by what the model is given.
Input Versus Weights
The weights are the part of the model that prompt engineering leaves alone. The definition draws its boundary exactly here: the technique works without changing the model weights. That boundary separates prompt engineering from any approach that modifies the model itself. When you change a prompt, you change what the model receives, and the model's weights stay the same.
Read the two columns side by side. In prompt engineering, the input changes and the weights do not. In the other approach, the weights themselves are modified, so the model that produces the output is a different model from the one before. Both approaches can change what a model does, but only one of them changes what the model receives while leaving the model unchanged.
What do you think happens?
If you rewrite a prompt so the model answers in a different format, have the model's weights changed?
Reveal answer
No, only the input changed
Prompt engineering is defined as getting desired behavior from the input alone, without changing the model weights. Rewriting the prompt changes what the model receives, and the weights remain the same.
Measured Performance on MMLU
Prompt engineering is not only a matter of wording. The source uses an evaluation to show that it affects measured model performance. MMLU is a benchmark used to evaluate models. On this benchmark, Gemini Ultra's performance went from 83.7% to 90.04% when a different prompt engineering technique was used.
| Setup | Model | MMLU score | What changed |
|---|---|---|---|
| Earlier prompting | Gemini Ultra | 83.7% | Baseline input |
| Different prompt engineering technique | Gemini Ultra | 90.04% | Only the prompting method |
The same model, with the reported MMLU scores for two prompting approaches
The gap between the two scores is 6.34 percentage points, which is 90.04 minus 83.7. The model is the same in both rows. What differs is the method used to write the input, so the change in the measured score is attributed to the input rather than to any change in the model. This is the reason the evaluation matters: it shows that a reported number depends partly on how the model was prompted.
Why Long-Context Tasks Need More Than an Instruction
A simple instruction covers an easy case. A short question with a clear task needs little more than a clearly written request. The source goes further and notes that complex work requires more care in what the model is given. It singles out tasks with long context: tasks where a large amount of material bears on the answer, so the model's input is no longer a short sentence.
Context construction is the process of building the context a model works from for complex tasks with long context. It is framed as a further step that goes beyond writing a good instruction.
The pipeline shows the idea. The pieces of a complex request do not reach the model as separate loose items. They are assembled into one input, and that assembled input is what the model reads. The instruction is only one of the parts. The material and the framing around it determine how much of the relevant information the model actually has when it responds.
Mistakes Beginners Make
Common mistake Assuming a model's performance is fixed no matter how the request is written
What it looks likeDeciding a model cannot do a task after one vague request
Why it failsThe way a request is written can change how well the model performs, even with the same model.
Do this insteadTry a clearer instruction, with the task and output format stated, before concluding the model cannot do the task.
Common mistake Thinking a prompt change modifies the model
What it looks likeBelieving that editing a prompt updates what the model has learned
Why it failsPrompt engineering works without changing the model weights. The change affects only what the model receives.
Do this insteadKeep the two ideas separate: changing the input is a different action from changing the weights.
Common mistake Treating a long-context task as if one instruction is enough
What it looks likePasting a long document and writing one sentence of request
Why it failsThe source notes that complex tasks with long context may also require constructing context, not just writing a good instruction.
Do this insteadDecide which instructions, material, and examples belong in the input, and assemble them into one clear context.
Common mistake Comparing benchmark numbers without noticing the prompting method used
Practice Sorting Changes
For each change below, decide whether it is prompt engineering (changing the input, weights unchanged) or a change to the model weights. Change 1: you add a sentence asking for a bulleted list. Change 2: you update the model's parameters on new training data. Change 3: you restructure the instructions so the task is stated before the material. Explain your answer for each one.
Hints
- Ask whether the change affects what the model receives or what the model itself is.
- Changes 1 and 3 alter the input; change 2 alters the weights.
A user wants a model to summarize a long set of meeting notes and follow a specific format. Describe the parts you would put into the input, in order, and say which part is the instruction and which part is the context. Then explain why writing only one clear instruction might not be enough.
Hints
- Consider the instructions, the material, the format, and the request itself.
- Think about how much relevant material the model would need to see at once.
Key Takeaways
- Prompt engineering is the process of getting AI models to express desirable behaviors from the input alone, without changing the model weights.
- Changing a prompt changes what the model receives; changing the weights changes the model itself, so the two are different levers.
- The reported MMLU result for Gemini Ultra rose from 83.7% to 90.04% when a different prompting technique was used, showing that measured performance can depend on the input.
- For complex tasks with long context, writing a good instruction may need to be followed by context construction, which assembles the material and instructions into one input.