Transformers vs. LLMs

Read a recent machine learning paper, a product announcement, or a forum thread, and you will often see the words transformer and LLM used as if they meant the same thing. The habit is understandable. The language models that have attracted the most attention in recent years are built on the transformer architecture, so the two labels tend to travel together. But they name different things. One is a neural network architecture, which is a design for how a network's computations are organized. The other is a category of AI model defined by what it does, namely processing language.

This article is about keeping those two ideas apart. Along the way you will trace how information moves through two different designs, because seeing the internal mechanics is the fastest way to understand why the terms overlap and why they do not match perfectly.

Before the transformer enters the picture, consider the recurrent design, which is one of the alternatives used for some language models. A recurrent model reads a sequence one token at a time. At each step it combines the current token with a hidden state, which is a running summary of what the model has seen so far. The updated hidden state is then passed on to the next step. Information from early tokens can only reach later steps by being carried inside that hidden state.

The reason the labels blur is that today's LLMs are constructed from transformers. Overlap, however, is not identity. Some transformers are not LLMs, and some LLMs are not transformers. The diagram below shows how the purpose-driven category and the architectural category relate to each other.

The clearest way to test the relationship is to classify concrete systems. For each one, ask two questions: is it built on the transformer architecture, and is its purpose processing language?

The most common error here is a pair of mirror-image assumptions. Each one feels reasonable because the two sets overlap heavily in practice, but each fails for a different reason.

If transformer-based LLMs are the dominant design, why would anyone build LLMs on recurrent or convolutional foundations? The source gives a direct answer: the main motivation behind these alternatives is to improve the computational efficiency of LLMs. The goal is to make language models cheaper to run in terms of computation, not to create a different kind of language model. The purpose stays the same. Only the underlying design changes.

A convolutional design takes a different route from the recurrent one. Instead of passing a hidden state step by step, it applies a filter over fixed windows of neighboring tokens. Each position in the sequence is processed using a small window around it, and stacking layers lets the windows cover more of the sentence. Both designs are alternatives to the transformer, and both are used to build language models that serve the LLM purpose.