Manipulating Tensor Shapes

A tensor is the basic data container you will work with throughout PyTorch, so the first practical skill is creating one. Everything you need to understand about creation can be read from a short listing that creates four tensors using one function, torch.tensor(). Before reading on, take a guess: what do you think the input to torch.tensor() controls?

The core definition is short. torch.tensor() creates a PyTorch tensor from a Python value that you pass to it as an argument. Every creation follows the same path: you write a Python value, pass it to torch.tensor(), and assign the returned tensor to a name. The shape of the tensor is decided at the moment of creation, by the value you pass in. This article shows how to read that value so you can predict the dimensions before you run the code.

The listing begins by importing the library and then makes four assignments. Read the lines from top to bottom and notice what changes from one line to the next.

The first line, tensor0d = torch.tensor(1), passes a single Python integer. The result is a zero-dimensional tensor, also called a scalar. A scalar holds one value and has no axes of its own. A Python integer goes in and a tensor comes out.

The second line passes a Python list, [1, 2, 3]. The result is a one-dimensional tensor, or vector. The three values sit side by side in a single row of data. The third and fourth lines extend the same idea by nesting lists inside lists. A list of lists produces a two-dimensional tensor, and a list of lists of lists produces a three-dimensional tensor. The four inputs are the exact inputs you can check against the code.

Read the four inputs from left to right. Each step adds one level of nesting, and the dimension count rises with it. The leftmost input has no brackets at all, so it is the only one that is not a list.

The listing gives you a reliable method for predicting dimensions. Count the levels of square brackets in the input before you run the code. A bare number has no brackets and yields a zero-dimensional tensor. One level of brackets, as in a flat list, yields a one-dimensional tensor. Two levels yield a two-dimensional tensor, and three levels yield a three-dimensional tensor.

Reading the input from left to right, the outer pair of brackets wraps the whole value, and each inner pair wraps one row. The number of bracket levels that wrap the innermost numbers is the dimension count. In this input the innermost numbers sit two levels deep, so the tensor is two-dimensional.

Sometimes the tensor you get is not the one you planned. Bracket counting helps you locate the exact point where the input departs from your expectation. The rule is the same as before: the number of bracket levels around the innermost numbers sets the dimension count, so any extra or missing pair of brackets changes the result.

The same method works in the other direction. If you expected a two-dimensional tensor but the input has only one level of brackets, the missing pair is where the structure diverges. Always compare the bracket depth of the input with the depth you intended, and look for the first level where the two disagree.