How Large Language Models Generate Text
The architecture behind modern LLMs is what makes this kind of selective focus possible.
Choosing What to Focus On
Read this sentence: The trophy would not fit in the suitcase because it was too big. Pause on the word it. Ask yourself what it refers to. Most readers instantly link it to the trophy, not the suitcase, even though both nouns appear earlier in the sentence. You did not reach that answer by reading each word in isolation. Your understanding depended on which earlier words mattered for the word it, and which ones could be set aside.
Large language models face a version of this same problem constantly. To produce a sensible continuation of a text, a model has to decide which parts of the input deserve the most weight at that moment. The architecture behind modern LLMs is what makes this kind of selective focus possible. This article follows that idea step by step: first where the architecture sits in the model, then what selective attention means, then how information from earlier parts of an input can shape a single prediction, and finally which claims about attention are easy to overstate.
What do you think happens?
In the trophy sentence, which earlier noun does the word it most likely refer to?
Reveal answer
trophy
Most readers link it to the trophy. Both nouns appear earlier, so the link is not decided by position alone. It depends on which earlier words matter for the word being interpreted. The same idea applies to an LLM: its predictions depend on which parts of the input it focuses on.
Where the Transformer Fits
The transformer is the architecture that LLMs use to process their input. Architecture here means the overall structural design of the model: the way its components are arranged and how information is passed between them. When you hear that a model is a transformer, you can read that as a statement about how it is built rather than about what task it performs.
The source gives the transformer a specific job in the chain of events. Input comes in. The transformer lets the model pay selective attention to different parts of that input. The model then makes its predictions. The predictions depend on that selective focus, which is why the architecture is central to how LLMs work. The diagram below shows this order: the transformer sits between the input and the predictions, and it is the stage where the model decides how to weight the input.
What Selective Attention Means
Selective attention means the model does not treat every part of its input as equally important when it makes a prediction. Some parts receive more of its focus than others, and the choice depends on what the model is trying to predict. The word selective is doing real work here: the focus is chosen, not spread evenly across everything.
Why does this matter? The source ties this ability to a benefit. Because the model can focus on the parts of the input that matter, it becomes especially good at handling the nuances and complexities of human language. Human language often depends on relationships between distant words, on ambiguity, and on context. A model that can weigh different parts of an input differently has a better chance of capturing those relationships than one that treats every part the same.
Tracing One Prediction Through the Input
Follow a single prediction from start to finish. Assume a short piece of text has been given to the model as input. The input is not used in a single pass without distinction. The transformer lets the model direct its focus toward some portions of that input more than others. Only after that selective focus has shaped the model's internal processing does the model produce its prediction.
Notice what changes between the start and the end of this trace. The input stays the same, but the model's emphasis on different parts of it shifts depending on the prediction being made. A different prediction over the same text can lead the model to attend to a different set of parts. This is the core difference between selective attention and treating all input uniformly.
Uneven Weight in the Trophy Sentence
Return to the trophy sentence and treat the word it as the prediction being made. The input is the whole sentence. The question is which earlier words should carry the most weight when the model interprets it. A simple illustration is to show uneven emphasis across three candidates. This is a teaching illustration built from the sentence, not a measured output of any real model: no numbers are given, only the qualitative pattern that the source describes.
The diagram shows relative emphasis only. The suitcase is not ignored, since both nouns are still part of the input, but the trophy receives the stronger focus for this particular interpretation. If the prediction were about a different word, the pattern of emphasis could look different, even though the sentence has not changed.
Mistakes When Reading Selective Attention
Selective attention is easy to misread because it sounds like a complete account of how a model thinks. The source supports a narrower claim. The comparison below separates what the idea establishes from what it does not.
| Claim | Supported by the source? | Why |
|---|---|---|
| Selective attention means uneven emphasis across the input | Yes | The source states it directly and says it is not the same as ignoring most of the input. |
| Selective attention means the model ignores most of the input | No | The source explicitly says selective attention is not ignoring most of it. |
| The focus depends on what the model is trying to predict | Yes | The source says the choice depends on the prediction being made. |
| The focus is fixed for a given input, regardless of prediction | No | The source says a different prediction over the same text can lead to attending to a different set of parts. |
| Selective focus explains why the model chose a particular output step by step | Not claimed | The source ties selective focus to handling nuance and complexity, but does not describe it as a full explanation of the model's reasoning. |
What selective attention does and does not claim
Common mistake Reading selective attention as the model ignoring most of its input
What it looks likeConcluding that the suitcase plays no role in the trophy sentence because the trophy gets more focus.
Why it failsThe source says selective attention means uneven emphasis, not ignoring most of the input.
Do this insteadTreat less-focused parts as weighted less, not removed.
Common mistake Assuming the focus is the same for every prediction on the same text
What it looks likeThinking the trophy always receives the strongest focus whatever word the model is predicting.
Why it failsThe source says the focus depends on what the model is predicting, and a different prediction can shift which parts are attended to.
Do this insteadAsk which prediction is being made before asking which parts of the input matter.
Common mistake Treating the transformer as a task rather than an architecture
What it looks likeSaying a model is a transformer, so it must be a translator or summarizer.
Why it failsThe source uses transformer to name how the model is built, not the job it performs.
Do this insteadRead the word transformer as a statement about structure and how information is passed between components.
Checking Your Understanding of Focus Shifts
Using the trophy sentence, explain in your own words what stays the same and what can change when the model makes a different prediction over this same input. Then say whether the model has stopped using the suitcase when its focus shifts toward the trophy.
Hints
- Separate the input from the model's emphasis on the input.
- The source says the focus is chosen according to the prediction being made.
- Uneven emphasis is not the same as ignoring part of the input.
Summary of the Mechanism
An LLM takes input and processes it through the transformer, the architecture that lets the model pay selective attention to different parts of that input. The focus is uneven and chosen according to the prediction being made, which is what allows the model to handle the nuances and complexities of human language. Selective attention is not the same as ignoring most of the input, and it does not by itself explain the model's reasoning.
At a glance
- The transformer is the architecture LLMs use to process input; it sits between the input and the model's predictions.
- Selective attention means uneven emphasis across the input, chosen according to what the model is predicting.
- Information from an earlier word can influence a prediction by shaping the selective focus that feeds internal processing.
- The same input can lead to different focus patterns for different predictions.
- Selective attention does not mean ignoring most of the input, and it does not by itself explain why a model produced a given output.