Tensors and PyTorch Fundamentals

A tensor is the basic data container you will work with throughout PyTorch. Before you can compute anything with one, you need the first practical skill: creating one. In PyTorch that skill rests on a single function, torch.tensor(). You pass it a Python value, and it returns a tensor that you can assign to a name.

Everything you need to understand about creation is visible in a handful of short lines. The goal of this article is to read those lines carefully and understand why each one produces the tensor it does. Along the way you will form a rule you can apply to inputs you have never seen before.

The source listing begins by importing the library and then makes four assignments. Each assignment follows the same pattern: a Python value is passed to torch.tensor(), and the returned tensor is stored under a name. The first two assignments are shown below exactly as they appear in the source.

The source describes the first result as a zero-dimensional tensor, also called a scalar. A scalar holds one value and has no axes of its own. The second result is described as a one-dimensional tensor, or vector. Its three values sit side by side in a single row of data. Notice what changed between the two lines: only the input. Moving from a bare integer to a list is what moved the result from zero dimensions to one.

The third and fourth lines of the listing extend the same idea by nesting lists inside lists. A list of lists produces a two-dimensional tensor, and a list of lists of lists produces a three-dimensional tensor. The following lines are an original illustration of that pattern, using new values so you can see the structure without relying on the listing's exact numbers.

Every line in the listing follows the same path. You write a Python value, pass it to torch.tensor(), and assign the returned tensor to a name. The diagram below shows that path in order. The important point is that the shape of the resulting tensor is decided at the moment of creation, by the value you pass in. You do not reshape the tensor afterwards to get the dimensions you want; the structure of the input determines them.

The listing gives you a reliable way to predict dimensions. Count the levels of square brackets around the innermost numbers. A bare number has no brackets and yields a zero-dimensional tensor. One level of brackets, as in a flat list, yields a one-dimensional tensor. Two levels yield a two-dimensional tensor, and three levels yield a three-dimensional tensor. Each added pair of brackets adds one dimension.

The diagram below turns that counting rule into a map. Start at the root, count the bracket levels in your input, and follow the branch that matches. Every branch corresponds to one of the four tensors in the listing.

Sometimes an input looks like it should produce one dimension count, but its brackets say otherwise. The useful habit is to read the brackets rather than the numbers. Count the opening brackets from the outside in, then check whether each level of brackets actually wraps something. The first level that does not match your expectation is where the tensor's dimensions differ from what you thought.