Tokenizing Text for Large Language Models
Python is a general-purpose programming language, and the examples throughout the course are Python programs or Python-based code that builds up to working language models.
Why Reading Code Is Not Enough
This course is built around Python code, and running the examples is essential to understanding them. Concepts such as tokenization, attention, and text generation are easier to grasp when you can run a small piece of code and see what it produces. For that reason, each lesson assumes that you will try things rather than only read about them.
Reading a code example and running it are two different experiences. Reading tells you what the code is supposed to do. Running it tells you what it actually does on your system. When the two match, you have confirmation that your setup works and that you understand the lesson. When they do not match, that mismatch is a useful signal to check your environment or re-read the example carefully.
What do you think happens?
Before running anything, what do you think a tiny program that stores the text 'Hello, LLM' and prints it along with its character count will show on screen?
Reveal answer
The text Hello, LLM and then the number 10
The string 'Hello, LLM' has 10 characters: 'Hello,' is 6 characters, the space is 1, and 'LLM' is 3. The program prints the stored text on one line and the length on the next line.
Where the Course Code Runs
The code in this course is written in Python. Python is a general-purpose programming language, and the examples throughout the course are Python programs or Python-based code that builds up to working language models. To follow along, you need a place where Python code can be executed and its results displayed to you. That place is your learning environment. Once you have one, you can paste a snippet into it, run it, and check the output against what the lesson describes.
The diagram below traces the path a script takes. You write the script in an editor, the Python interpreter reads and executes it inside your learning environment, and the results come back to you as printed output. Your environment is the part of the path that makes the whole sequence possible.
Tracing a First Successful Run
A first successful run means that you executed a program, it finished without an error, and the output matched what you expected. The program is deliberately simple. Its only job is to confirm that your environment can run Python code and show you a result. The example below is an original illustration written for this article in the same spirit as the course's first program: it stores a piece of text, prints it, and then prints how many characters it contains.
text = "Hello, LLM"
print(text)
print(len(text))- 1Line 1 creates a name called text and binds it to the string Hello, LLM, using double quotation marks around the text.
- 2Line 2 asks Python to display the value of text, which produces the first line of output: Hello, LLM.
- 3Line 3 asks Python for the number of characters in text and displays that number, which produces the second line of output: 10.
- text = stores the string in a name called text
print(text)displays the stored stringlen(text)counts the characters in the string
Hello, LLM
10Matching each output line to its code line
You run the three-line program and see two lines on screen. How do you confirm that your environment is working?
- 1Count the output lines
There are two lines of output, even though the program has three lines. The first line of code only stores a value and prints nothing.
- 2Match the first line
The first output line repeats the stored text, so it comes from the
print(text)line. - 3Match the second line
The second output line is a number, the character count, so it comes from the
print(len(text))line. - 4Judge the result
Both lines match what the lesson expects, so your environment executes basic Python and displays output as expected.
Your environment is working for basic Python when the two output lines match the two print statements.
Reading Errors From a Failed Run
If your output does not match, or you see an error message instead, the source gives a clear first step: read the last line of the message first, since it usually names the problem. At this early stage, a missing or misspelled name, or a missing quotation mark, are common causes.
If the second line of the program were written as print(lenn(text)) with a misspelled function name, the error message would name the unknown name in its last line. The fix is to correct the spelling so it reads len, then run the program again and compare the result with the lesson.
Which Tools You Need Now
You do not need to install everything on the first day. The course introduces its tools in the order that it needs them. Early chapters focus on the overall picture of large language models and on the conceptual building blocks, so a basic Python environment is enough to begin.
Two later tools wait for their own chapters. Some later chapters rely on a deep learning library, and the course includes a dedicated chapter on Introduction to PyTorch that covers its components and its installation. Other chapters use a separate tool for evaluating models, which is introduced in the chapter where it becomes relevant.
How the Course Builds From Foundations
The course moves through a sequence of chapters, and each chapter builds on the ones before it. The first chapter, Getting Started, is the one you are reading now. The early material then introduces large language models in general and places them in the context of artificial intelligence and machine learning. From there, the course looks at how these models are applied, how they are built and trained, and how text is turned into the numerical form a model can process. That last step is where tokenization begins to matter.
The middle of the course covers the ideas that make modern language models work, including word embeddings, tokenization and token IDs, attention mechanisms and self-attention, and the transformer and GPT architectures. Later chapters address pretraining, fine-tuning, evaluation of model responses, and methods for controlling generated text such as temperature scaling and top-k sampling.
Mistakes Beginners Make on the First Run
Common mistake Treating a successful read-through as proof that the code works
What it looks likeYou understand the three-line program on the page but never run it.
Why it failsReading tells you what the code is supposed to do, not what it does on your system.
Do this insteadRun the program and compare its output with what the lesson describes.
Common mistake Skipping the prediction step
What it looks likeYou run the program immediately and only then look at the output.
Why it failsWithout a prediction, a mismatch between expected and actual output is hard to notice and interpret.
Do this insteadWrite down what you expect each line to show, then run the program and compare.
Common mistake Reading the first line of an error message instead of the last
What it looks likeYou focus on the opening traceback text and miss the line that names the problem.
Why it failsThe source notes that the last line of an error message usually names the problem.
Do this insteadRead the last line first, then check for a misspelled name or a missing quotation mark.
Common mistake Installing every tool on the first day
Why it failsThe course introduces tools in the order it needs them, and early chapters only need a basic Python environment.
Do this insteadSet up the basic environment now and add the deep learning library or evaluation tool when its chapter asks for it.
Your First Session Routine
A productive routine for this course is simple. Read the lesson, predict what each code example will do, run it, and compare the result with your prediction. When something differs, figure out why before moving on. Keep a short note of any setup steps you took, so that you can repeat them if you change machines or need to help someone else. These habits matter more than speed, because later chapters reuse the same ideas in larger programs.
Create a three-line Python program that stores a short text of your choice, prints it, and prints its character count. Before you run it, write down the two lines you expect to see. Then run it, compare the output with your prediction, and note any setup steps you took.
Hints
- Use the same three-line pattern shown earlier: assign the text, print the text, print the length of the text.
- If the output differs from your prediction, check the spelling of the function name and the quotation marks first.
- Count the characters in your text by hand before running the program, so you can check the second line of output.
Key Takeaways
- The course is built around Python code, and running examples lets you check that what you read matches what your system actually does.
- A basic Python environment is enough to begin, and the deep learning library and evaluation tool are introduced in the chapters that need them.
- A first successful run is one that finishes without error and produces the output you predicted; when output differs, read the last line of any error message first.
- The course builds from Getting Started through language model basics, tokenization, attention, architectures, pretraining, and fine-tuning, so each chapter depends on the ones before it.