Understanding Large Language Models
The early modules introduce large language models and the book itself, then explain what these models are in the context of artificial intelligence and machine learning.
Starting From a Familiar Habit
Many people have used a phone keyboard that suggests the next word as they type. You start a message, and the keyboard offers a few words that are likely to come next. That habit of guessing what follows a given stretch of text is the same habit at the heart of a large language model. The difference is one of scale and ambition. An LLM is built to handle a far wider range of language, and it learns that skill from a very large body of text rather than from a small list of common phrases.
This article builds the definition of an LLM from the ground up. It starts with what an LLM is, then looks at what the word 'large' is doing in its name, then follows the training task that shapes the model. Along the way it also gives you a map of where this material sits inside the course, so that each later lesson has a place to land.
How the Course Is Organized
The course is organized as a sequence of modules, and each module is made up of one or more chapters. A module groups related material under a shared theme, and a chapter narrows that theme to a specific set of topics. Read the diagram from the top down: the course contains modules, and each module contains chapters.
What a Large Language Model Is
A large language model is a deep neural network designed to understand, generate, and respond to humanlike text.
Three parts of that definition do most of the work. The first is that an LLM is a neural network, which means it is a learned mathematical system made of many connected adjustable pieces. It is not a hand-written list of grammar rules. The second is that it handles text, so both its inputs and its outputs are language. The third is that it is designed to work in a humanlike way, meaning its output is meant to read naturally and respond to what was written.
The phrase 'deep neural network' signals that the network has many layers of processing. That depth is what allows the model to pick up subtle patterns across a long passage, rather than only the relationships between neighboring words.
The Kind of Text an LLM Works With
An LLM works with language expressed as text. It reads text as input and produces text as output, and everything it learns comes from text as well. The model is trained on massive amounts of text. In some cases that training text can cover large portions of the entire publicly available text on the internet. The scale of the data is part of what distinguishes these models from smaller language systems. A model that saw only a few thousand sentences could not learn the wide variety of patterns that the largest models pick up.
Two Meanings of Large
The word 'large' in the name carries two separate meanings. Both are about scale, but they measure different things. The first is the number of parameters in the model. The second is the immense dataset used to train it. Once you can tell these apart, it becomes easier to reason about what makes a model powerful and what goes into building one.
The diagram shows the relationship between the two meanings. The parameters are the adjustable weights that make up the network. The dataset is the material the network learns from. Training uses the text to adjust the parameters, so the two kinds of scale are connected, but they are not the same thing: one describes the size of the structure, and the other describes the size of the experience used to shape it.
Parameters as Adjustable Weights
Parameters are the adjustable weights inside the network. They are optimized during training so the model can predict the next word in a sequence. An analogy from the source helps here. Picture a large mixing board in a recording studio with thousands of sliders. Each slider sets how much one signal contributes to the final sound. Training is the process of moving the sliders until the output matches what you want.
In the mixing board analogy, the sliders are the parameters, and the thing you want the output to match is the next word in a sequence. No single slider has a meaning on its own. The sound comes from how all the sliders are set together, which is the same idea behind an LLM: its knowledge is spread across many adjustable values rather than stored in one labeled place.
Next-Word Prediction as a Training Task
The task used to train these models is next-word prediction. The model is shown a sequence of words and asked to guess the word that comes next. Language is naturally sequential: each word arrives after the ones before it, and meaning builds up as the sequence unfolds. Next-word prediction takes advantage of that order. Learning to guess well forces the model to pick up context, grammatical structure, and the relationships between words.
What do you think happens?
When the model trains on a passage, where does the correct answer for each guess come from?
Reveal answer
The next word in the text itself
The model does not need a teacher to label each sentence. The word that actually follows in the text serves as the answer, so every position in a passage becomes a small prediction exercise. This is why the training text can be so plentiful.
The simplicity of the goal is what makes the task powerful. A guessing game sounds far too modest to produce a system that can understand and respond to text. Yet a task this simple has produced models that are remarkably capable. The source notes that this surprised many researchers.
Tracing One Prediction Into a Training Signal
The training loop is easiest to understand by following one prediction from start to finish. Read the diagram from left to right. The text supplies context, the model makes a guess, and the guess is compared with the word that actually follows in the text. The result is an error signal that feeds back into the parameters, nudging them so that future guesses improve. Repeat this across an enormous amount of text and the parameters gradually settle into values that capture a great deal about language.
Tracing a Single Prediction (illustrative)
The training text contains the fragment 'The cat sat on the mat.' Trace one prediction, where the context is 'The cat sat on the' and the actual next word in the text is 'mat'.
- 1Provide the context
The model sees the text so far, 'The cat sat on the', and nothing after it.
- 2Make a guess
Using its current parameters, the model produces a guess for the next word. Suppose it guesses 'roof'.
- 3Compare with the actual word
The word that actually follows in the text is 'mat'. The text itself supplies this answer, so no teacher is needed.
- 4Produce the error signal
The guess and the actual word do not match, so the comparison yields an error signal describing how far off the guess was.
- 5Adjust the parameters
The error feeds back into the parameters, nudging them so that a similar future prediction is more likely to land on 'mat'.
- 6Repeat across the text
Every other position in the passage gives another context-and-answer pair. Many such small predictions, across an enormous amount of text, shape the parameters over time.
One prediction produces one training signal. The model does not learn from a single example in isolation; it learns from the accumulated effect of countless predictions, each one nudging the sliders a little.
Mistakes Beginners Make With LLM Basics
Common mistake Treating the training text and the model as the same thing
Why it failsThe text is the material the model learns from during training, while the network is the structure that holds what it learned.
Do this insteadDescribe the text as the learning material and the network as the structure that stores the result.
Common mistake Thinking 'large' only means the number of parameters
Why it failsThe word 'large' refers to both the number of parameters and the immense dataset used to train the model.
Do this insteadName both meanings and explain how training connects them.
Common mistake Assuming next-word prediction needs a human to label each sentence
Why it failsThe next word in the text itself serves as the answer, which is why plentiful training text is usable.
Do this insteadExplain that every position in a passage becomes a small prediction exercise.
Common mistake Dismissing next-word prediction as too simple to matter
Why it failsLearning to guess well forces the model to pick up context, grammatical structure, and relationships between words.
Do this insteadConnect the simple goal to the richer patterns it drives the model to learn.
Common mistake Expecting this orientation article to teach installation or deep technical detail
Practice Tracing a Prediction
Take the training fragment 'The sky is blue.' Set the context to 'The sky is' and the actual next word to 'blue'. Describe, in order, what the model sees, what it guesses, how the guess is compared with the actual word, what the error signal represents, and how the parameters change. Then explain in one sentence why this single prediction is not the whole story of training.
Hints
- Start from the text so far, then ask what the model produces from it.
- The comparison is always between the guess and the word that actually follows in the text.
- Think about what happens after one prediction versus after the whole passage.
Key Takeaways
- 1The course is a sequence of modules, each containing one or more chapters, and the Getting Started module contains a single chapter that holds this orientation article.
- 2A large language model is a deep neural network designed to understand, generate, and respond to humanlike text, trained on massive amounts of text.
- 3The word 'large' refers both to the number of parameters, which are adjustable weights, and to the immense dataset used for training.
- 4Next-word prediction drives training: the text supplies the answer, and each prediction produces a training signal that nudges the parameters.
At a glance
- The course is organized as modules containing chapters, and the Getting Started module holds this orientation article as its single chapter.
- A large language model is a deep neural network designed to understand, generate, and respond to humanlike text, trained on massive amounts of text.
- The word 'large' refers both to the number of parameters, which are adjustable weights, and to the immense dataset used to train the model.
- Next-word prediction drives training: the text supplies the correct next word, and each prediction produces a training signal that adjusts the parameters.