Encoder and Decoder in the Transformer

The self-attention mechanism lets a transformer weigh the importance of different words or tokens in a sequence relative to each other.

Programming

Why Words Need Other Words

When you read a sentence, you constantly decide which other words matter for understanding a given word. A pronoun such as 'it' is meaningless until you connect it to something earlier in the text. A word near the end of a long passage may depend on a phrase introduced many words before it. Language is full of these links, and they do not always sit next to each other.

Transformers and large language models (LLMs) are built around a component called the self-attention mechanism, which is designed to handle exactly this problem. Its job is to let the model weigh the importance of different words or tokens in a sequence relative to each other. Because the weighing happens between the words of the input itself, the mechanism is described as self-attention. This article builds a picture of what that weighing accomplishes and where it fits in the larger architecture.

Weighing Tokens Against Each Other

Start with the idea of relative importance. When a model processes one token, not every other token in the sequence is equally useful for interpreting it. Some neighbours carry most of the meaning, some carry little, and some far-away tokens carry a crucial clue. Self-attention is the mechanism that expresses this unequal relevance, so that the representation of each token can draw more heavily on the tokens that matter most to it.

The diagram is a picture of the judgement self-attention makes, not a measurement. It shows that 'it' is connected to several tokens with different strengths, and that the strongest connections may lie far from 'it' itself. The source does not give numerical weights for this sentence, so the strength labels here are an illustration of relative importance only.

Splitting the Translation Work

The source describes the transformer as a deep learning model architecture for language translation. In the simplified depiction it gives, the transformer is made of two parts: an encoder and a decoder. Self-attention appears within this larger structure, and understanding the two parts helps place it.

In the original transformer architecture, the encoder processes the complete input text and produces an embedding representation. The decoder then uses that representation to generate output one word at a time. The two parts have distinct jobs: one reads and represents the input, and the other builds the output from that representation.

Part	Input it receives	What it produces	How it is described in the source
Encoder	The complete input text	An embedding representation	Processes the complete input text
Decoder	The embedding representation, plus words it has already produced	One output word at a time	Uses the representation to generate output one word at a time

The encoder reads and represents; the decoder generates.

Following One Translation Through the Pipeline

The source follows the original transformer through one translation. The input text is the sentence 'This is an example', and the partially translated sentence is 'Das ist ein'. The final stage is the decoder generating only the last word, 'Beispiel', to complete the translation. Read the pipeline from the input on the left through to the complete output.

Worked example

Tracing the handoff for one sentence

Trace how the English sentence 'This is an example' becomes the German sentence 'Das ist ein Beispiel', using only the roles the source describes.

1
Step 1: The encoder reads the complete input
The whole sentence 'This is an example' is processed by the encoder, not one word at a time in isolation.
2
Step 2: The encoder produces a representation
The output of the encoder is an embedding representation of the input text.
3
Step 3: The decoder generates the first words
The decoder uses the representation to produce 'Das ist ein', which is the partially translated sentence.
4
Step 4: The decoder generates the last word
The final stage requires the decoder to produce only 'Beispiel' to complete the translation.

Result

The encoder turns the English input into a representation, and the decoder uses it to generate 'Das ist ein Beispiel' word by word.

What do you think happens?

In the partial translation 'Das ist ein', which word is still missing to complete the German sentence?

Reveal answer

Answer:

Beispiel

The source describes the decoder generating only the last word, 'Beispiel', to complete the translation of 'This is an example'.

Long-Range Links and Context

Self-attention is what allows a model to relate every position in an input to other positions, rather than reading each word in isolation. The source identifies two closely linked benefits. First, the mechanism enables the model to capture long-range dependencies, meaning relationships between words that sit far apart. Second, it captures contextual relationships within the input data, so that a token's role is shaped by its surroundings.

The contrast in the diagram is conceptual. Under self-attention, the two distant positions can be weighed against each other directly. A step-by-step model would have to carry information from position 1 through every intermediate position before it reaches position 20. The source does not describe that sequential alternative in detail, so the diagram shows only the general contrast.

Mistakes Beginners Make With the Big Picture

Common mistake
Thinking self-attention only looks at the neighbouring word
What it looks like
In 'The animal didn't cross the street because it was tired', the word directly before 'it' is 'because', which carries little meaning for 'it'.
Why it fails
Self-attention weighs each token against every other token, so distant tokens such as 'animal' can carry the crucial clue.
Do this instead
Think of relevance across the whole sequence, not just adjacency.
Common mistake
Assuming the encoder and decoder both generate the whole translation at once
What it looks like
The decoder produces 'Das ist ein' and then 'Beispiel' as separate steps.
Why it fails
The source describes the decoder generating output one word at a time.
Do this instead
Track the translation as a sequence of single-word generation steps.
Common mistake
Assuming the weighing of words is the entire translation process
Why it fails
The source places self-attention inside the larger encoder and decoder architecture, which handles the handoff of information between its two parts.
Do this instead
Keep self-attention as one component inside the encoder-decoder structure.

Where This Introduction Stops

What you can take forward is the picture: self-attention weighs tokens against each other, the encoder turns the input into a representation, and the decoder uses that representation to generate the output word by word.

Trace the Roles in Your Own Words

EASY

Using the translation 'This is an example' to 'Das ist ein Beispiel', explain in your own words which part of the transformer reads the complete English input, which part produces the German words, and what information passes between them.

Hints

Look for which part produces an embedding representation from the input.
Consider which part uses that representation to generate output one word at a time.
The decoder also uses the words it has already produced.

Your attempt (not saved)

MEDIUM

For the sentence 'The animal didn't cross the street because it was tired', name two tokens that the word 'it' should be weighed against most strongly, and explain why a model needs to consider more than the word directly before 'it'.