Training Large Language Models

Imagine you want a program that can write fluent sentences. You could try to write rules for grammar and word choice by hand, but language is too varied for that to work. A different approach is to let a model learn patterns from a large body of text and then use what it learned to generate new text. The process of building that learned starting point is called pretraining. It is the stage where a model is first trained on text before anyone asks it to do a specific job.

Before a language model can learn anything from text, that text has to be broken into pieces. Those pieces are called tokens, and the process of producing them is called tokenization. This article focuses on one particular tokenization scheme, byte pair encoding, usually abbreviated BPE. It also places that scheme inside the larger path that leads from raw data to a model you can evaluate.

The easiest way to see the roles of each component is to follow a piece of text from start to finish. Raw text goes into the BPE tokenizer. The tokenizer produces tokens. Those tokens are part of what the language model learns from during training. Each arrow in this trace marks a change of role, not a change of content, and keeping track of which component owns which step is the key skill for the rest of this article.

The source introduces BPE as a more sophisticated tokenization scheme than the simpler approaches that come before it. The word sophisticated is the key signal. BPE is not a basic way of cutting text apart; it is a scheme built on a specific concept, namely byte pair encoding. So the safe classification to hold onto is this: BPE is one way to split text into tokens, and it is considered more capable than the simpler schemes introduced earlier in the same chapter.

The reason BPE deserves attention is its use in practice. The source states that a BPE tokenizer was used to train large language models such as GPT-2, GPT-3, and the original model used in ChatGPT. In other words, BPE was not only a topic in theory. It was part of the pipeline that prepared text for training some of the best-known language models.

The tokenizer and the language model are different parts of the pipeline. The tokenizer's job is to turn text into tokens. The language model's job is to learn from those tokens during training. Because the two are separate, you can say what each one is responsible for without mixing them up. The tokenizer produces the tokens; the model is trained on them.

In this course, pretraining is treated as the second major stage of work. The first stage prepares the ingredients: data preprocessing and the attention mechanism at the core of modern LLMs. The second stage puts those ingredients together, showing how to code and pretrain a GPT-like model that can generate new text, and then how to evaluate the result. Keeping these stages in mind helps you see why each topic exists.

A helpful way to hold the whole topic is to follow one path. Start with raw data and ask what must happen before a model can learn from it. Then ask what the model needs in its design, and what training does with that design. Finally, ask how you judge whether the trained model is any good, and whether you should train it yourself at all or start from weights someone else already produced.

Stage 1 of the course is where the foundations are laid. Pretraining depends on two of them: data preprocessing and the attention mechanism. Neither is the training itself, but each one has to be understood before the training stage makes sense.

There are two routes to a trained model. The first is to pretrain an LLM from scratch. The second is to load openly available model weights, which means starting from a set of learned values that someone else already produced. The source treats loading weights as an alternative way to reach a trained model without running the full training yourself.

The cost is the reason the course narrows its goal. Pretraining at full scale demands thousands to millions of dollars in computing costs for GPT-like models, a range wide enough that full-scale pretraining is a major commitment rather than a routine exercise. So stage 2 focuses on implementing training with a small dataset for educational purposes.

The third topic in stage 2 is evaluating LLMs, which the source describes as essential for developing capable NLP systems. A trained model is a set of learned values, and those values alone do not tell you whether the model produces good text. Evaluation is the step that measures what the model does, so that you can compare versions and decide what to change.

Evaluation applies to both routes. A model you pretrain on a small dataset needs to be evaluated, and so does a model whose weights you loaded from an open source. Because the weights in the second case came from elsewhere, you cannot assume they behave as you expect in your own setting. Measuring is how you find out.