Large Language Models

When you read that a model can understand language, you probably picture something that grasps meaning, notices intent, and knows what its words refer to in the world. That picture is natural, because it is how we describe people. This article starts from that assumption on purpose, tests it, and then replaces it with a more accurate picture of what large language models, usually called LLMs, actually do. The goal is a precise mental model you can reason with, not a vague sense that the software is clever.

The book's introduction places the turning point for LLMs in 2022, with the release of ChatGPT. From that point, large language models have taken the world by storm and have changed how many people work. The source describes them as incredibly versatile, aiding in tasks such as checking grammar, composing emails, and summarizing lengthy documents. The author attributes this versatility to their ability to parse and generate human-like text, a skill the source says matters in fields ranging from customer service and content creation to technical domains such as coding and data analysis.

The source asks you to read the word 'understand' narrowly. An LLM is evaluated by the text it generates. If the output is coherent, stays consistent across sentences, and fits the context it was given, the model is described as understanding. This is a statement about observable output quality. It says nothing about inner experience, consciousness, or comprehension in the human sense.

Follow a piece of text through an LLM at the level the source describes. The source says the model parses and generates human-like text, and that its output is judged by how coherent it is and how well it fits the context. It does not, in this section, walk through the internal mechanism. The book builds up to the transformer architecture and the other components in later chapters, so this section keeps to the observable sequence.

A common belief is that LLMs are a black box: they produce impressive output, but nobody can explain how. The belief is easy to form. These systems are very large, the public sees only their answers, and their internal workings are rarely shown to ordinary users. The defining feature the author gives for LLMs is scale. As the name implies, LLMs are large, encompassing millions to billions of parameters. Parameters are the adjustable values inside a model, so that count means the model contains a very large number of them.

The author's central point is that scale and understandability are separate properties. A model can be large and still be understood at a fundamental level. The stated goal of the book is that, by the end, the reader will have a solid understanding of how an LLM, like the ones used in ChatGPT, works on a fundamental level. The author's route to that understanding is to build one step by step, because the best way to understand LLMs is to code one from scratch.

LLMs belong to a family of fields that nest inside one another. Artificial intelligence is the broadest of these. Machine learning is a part of it. Deep learning is a further part of machine learning, and it is the one centered on neural networks. The source defines deep learning as a subset of machine learning and artificial intelligence that focuses on neural networks. Large language models are deep neural network models, so they sit at the innermost layer of this nesting.

Before LLMs, many language tools were built around a single job. Some classification work, such as flagging email spam, could be handled by handcrafted rules or simpler models. Those methods worked well for narrow categorization and straightforward pattern recognition, but they tended to underperform on tasks that demanded more, such as following detailed instructions, analyzing context, and producing coherent original text. The source gives a pointed example: earlier generations of language models could not write an email from a list of keywords, a task that is routine for contemporary LLMs.

The source attributes the success of LLMs to two things working together. The first is the transformer architecture, the design that underpins many LLMs. The second is the very large amount of data the models are trained on. Together they let an LLM capture a wide variety of linguistic nuances, contexts, and patterns. The source notes that encoding these patterns by hand would be very difficult. Deep learning is the ingredient that makes training on such volumes practical, and training on that much text lets the model pick up deeper contextual information and subtleties of language than earlier approaches could capture.

The source describes the transformer architecture only as the design underpinning many LLMs and as a key enabler of their success. It does not walk through its internal parts in this material. Chapter 1 of the book gives a high-level introduction to it, and later chapters cover preparing text, attention mechanisms, implementing a GPT-like model, pretraining, and fine-tuning. For now, treat the transformer as the structural foundation that, combined with large-scale training data, has changed how NLP is done.