Tensors and Shapes in PyTorch

Suppose you want a language model to continue a short piece of text. The concrete goal here is a prompt, the string "Hello, I am", which must be prepared as input for the generate_text_simple function. A model does not read characters the way a person does. Before the model can use the prompt, the text has to pass through two conversions: first into token IDs, and then into a tensor. Each conversion produces something the next stage can consume, and the whole path is a chain in which every stage depends on the one before it.

This article follows that chain one stage at a time. You will see what each stage receives, what it returns, and how the shape of the data changes along the way. By the end you should be able to look at a printed list and a printed shape and say which stage a problem most likely came from.

The preparation takes four lines of logic. One line assigns the prompt to a variable. The second encodes it. The third converts the encoded result into a tensor with a batch dimension. The remaining two lines print what the encoding and tensor stages produced, so you can inspect each intermediate result instead of trusting the final output alone.

The first stage is tokenizer.encode(input_text). Its input is a string and its output is a list of integers. Those integers are the representation the rest of the pipeline works with. The raw text is not passed to the tensor step; only the list of token IDs is. This means that everything downstream depends on what the encoder returned, so the printed list is the first thing to check.

The second stage is torch.tensor(encoded), which converts the Python list of token IDs into a PyTorch tensor. A list of four integers becomes a one-dimensional tensor with four elements. The values stay the same; what changes is that the data now lives in the tensor type that PyTorch operations expect.

The third stage is the call .unsqueeze(0), which adds a new dimension at index 0. This is the batch dimension. The model expects its input to carry a batch dimension, so this step is required even though the token values themselves do not change. Only this final stage adds the batch dimension, which is why its effect appears in the shape printed at the end of the trace.

The diagram is a pipeline, so each stage consumes the output of the previous one. A problem at any stage propagates to every later stage. If the encoder returns the wrong integers, the tensor will faithfully hold the wrong integers. If the tensor is built correctly but the batch dimension is missing, the model receives the right values with the wrong shape.

A shape tells you how many values sit at each level of nesting. Each number in a shape corresponds to one level of brackets. A shape of $(4,)$ describes a single bracketed group containing four values, such as a flat list of four token IDs. A shape of $(1, 4)$ describes an outer group containing exactly one inner group, and that inner group holds four values. The first number is the outermost level and the last number is the innermost.

The method call .unsqueeze(0) adds a new dimension at index 0. Index 0 is the front of the shape, so the new dimension becomes the outermost level of nesting. The token values themselves are not altered and not reordered. The call only changes how those values are grouped. This is why the source describes the step as adding a batch dimension: it wraps the existing sequence so that the model sees a batch containing one item.

Reading the diagram from left to right shows the transformation. The four token IDs survive intact. What changes is that a new outer level of size 1 now encloses them, and that level sits at index 0 because the call specifies 0. Calling unsqueeze with a different index would place the size-1 dimension somewhere else, but the batch dimension described in the source is the one created at index 0.

The encoder is the stage that converts text into the integer representation the rest of the pipeline uses. The source states the key contract: tokenizer.encode(input_text) converts a string into a list of integer token IDs. Each integer stands in for a piece of the input. The exact integer values depend on the tokenizer, so the number you see for a given word is a property of that tokenizer rather than something you can derive from the text alone.

The source gives a practical way to separate two kinds of failure. Checking the printed encoded list and the printed tensor shape tells you whether the problem is in encoding or in tensor preparation. The following mistakes are the ones most often confused with each other.