Preparing Text Inputs for Neural Networks
The original transformer is a deep neural network architecture introduced in 2017, developed for machine translation, and now underlies most modern LLMs.
Why the Original Transformer Was Built for Translation
Most modern large language models are built on the transformer architecture. The transformer is a deep neural network architecture that was introduced in a 2017 paper titled Attention Is All You Need. Because so many current systems descend from this design, it helps to understand the architecture as it was first presented, rather than jumping straight to the large models that followed.
The original transformer was not built for chatbots or text completion. It was developed for machine translation, specifically translating English texts into German and French. That origin shapes everything about how its parts are arranged. The design is organized around reading one language and writing another, and that goal explains why the architecture is split into two halves.
Guessing Which Half Writes the Output
Before reading the mechanics, try a quick prediction. Suppose you give the original transformer an English sentence and want a German translation. Think about which part of the system should be responsible for producing the German words, and which part should only be responsible for taking in the English.
What do you think happens?
In a translation task, which submodule is responsible for writing the German output text?
Reveal answer
The decoder
The encoder never writes German. It turns the input text into numerical vectors. The decoder takes those vectors and generates the target-language text, so the decoder is the part responsible for the German output. The two halves still work together, because neither submodule does the whole job alone.
Two Submodules and Their Jobs
The transformer is made of two submodules. The first is the encoder, which takes in the input text. The second is the decoder, which produces the output text. Neither submodule does the whole job alone. The encoder's result is what the decoder works from, so the two parts operate as a pipeline in which one feeds the next.
The diagram above shows the structure from the top down. The transformer contains two submodules. Each submodule is not a single flat block but consists of many layers, which is why both the encoder and the decoder are drawn as stacks of layers rather than as one box each.
The encoder's job is to turn input text into numerical vectors that capture context. The encoder does not produce words. The decoder's job is to generate output text from those vectors. It does not read the original text in the way the encoder does. It begins from the vectors that the encoder produced.
The handoff between the two submodules is the vectors. The encoder produces them, the decoder consumes them, and that handoff is the central design idea of the architecture.
Tracing a Sentence from Encoder to Decoder
Now follow one sentence through the whole system. Read the diagram below from left to right as a single path. The source sentence enters the encoder, the encoder produces vectors, those vectors pass to the decoder, and the decoder generates the target-language text.
- 1The source-language sentence enters the transformer as input text and is taken in by the encoder.
- 2The encoder processes the source language and turns the input into numerical vectors that capture its context. It does not produce words at this stage.
- 3The vectors are handed to the decoder. The decoder does not read the original sentence directly the way the encoder did. It starts from these vectors.
- 4The decoder uses the vectors to generate the output text in the target language, such as German or French.
Following one English sentence toward German
Trace the path of a short English sentence through the original transformer during a translation into German. Identify which component handles each stage.
- 1Input arrives
The English sentence is the input text. It enters the encoder, the first submodule, which takes in the input text.
- 2Encoder produces vectors
The encoder processes the English and outputs numerical vectors that capture the context of the sentence. At this point, no German has been written.
- 3Vectors are handed over
The encoder's vectors are what the decoder works from. The handoff between the two halves is the vectors.
- 4Decoder writes German
The decoder takes the encoded vectors and generates the target-language text. Because this is a German translation, the decoder is the component producing the German words.
The English sentence goes in as text, becomes vectors inside the encoder, passes to the decoder, and comes out as German text. The encoder never writes German. The decoder is responsible for the German output.
Self-Attention as the Connective Tissue
Both the encoder and the decoder consist of many layers, and those layers are connected to one another by a mechanism called self-attention. The source describes self-attention as the connective tissue inside each submodule that links the layers together. It is the mechanism that joins the layers within the encoder stack and within the decoder stack.
Mistakes Beginners Make with the Two Halves
Common mistake Thinking the encoder writes the translated text
What it looks likeAssuming that the English-reading half also produces the German sentence.
Why it failsThe encoder turns input text into numerical vectors that capture context. It does not produce words.
Do this insteadThe encoder produces vectors. The decoder generates the target-language output from those vectors.
Common mistake Assuming the decoder reads the source text directly
What it looks likeBelieving the decoder scans the English sentence word by word, just as the encoder does.
Why it failsThe decoder begins from the vectors produced by the encoder rather than reading the original text in the way the encoder does.
Do this insteadTreat the vectors as the handoff between the two halves.
Common mistake Treating each submodule as a single block
What it looks likeDrawing the encoder as one box with no internal structure.
Why it failsBoth the encoder and the decoder are made of many layers connected by self-attention.
Do this insteadPicture each submodule as a stack of layers, linked internally by self-attention.
Common mistake Confusing the 2017 original design with later large language models that descend from it
Trace It Yourself
A French translation task uses the original transformer. Describe, in order, the path a French source sentence takes through the system, naming which submodule performs each stage, what passes between the two submodules, and what role self-attention plays inside each submodule.
Hints
- Start with which submodule takes in the input text.
- Think about what the encoder outputs, and what the decoder works from.
- Recall the phrase connective tissue when describing the layers.
Key Takeaways
At a glance
- The original transformer, introduced in 2017, was developed for machine translation, such as translating English into German and French.
- The transformer has two submodules: the encoder takes in input text and turns it into numerical vectors that capture context, and the decoder generates output text from those vectors.
- In a translation task, the source sentence enters the encoder, the encoder's vectors are handed to the decoder, and the decoder produces the target-language text.
- Both submodules are made of many layers, and self-attention is the mechanism that links those layers together inside each submodule.