Tensor Transpose
The starting tensor is built from a list of two inner lists, each holding three numbers.
Why Early Assumptions Matter
This article works through a short PyTorch program in which one small tensor is created, inspected, reshaped, transposed, and multiplied by its own transpose. Each of these steps is simple on its own. They are worth practicing together because a single wrong assumption early on, such as a shape you did not expect, carries forward into every later line. It can then look like a failure somewhere else entirely.
The habit to build is prediction. Before you look at each printed result, decide what you expect it to be. The goal is not to memorize outputs. It is to notice the moment when your expectation and the actual result disagree, because that moment tells you where your understanding (or your code) needs attention.
Tracing the Starting Tensor
The starting tensor is built from a list of two inner lists, each holding three numbers. PyTorch reads that nested structure and creates a two-dimensional tensor with two rows and three columns. Its shape attribute reports how many entries lie along each dimension. Checking the shape is the first thing to do whenever a later operation behaves strangely.
torch.Size([2, 3])
tensor([[1, 2, 3],
[4, 5, 6]])- The outer list has two inner lists, so the tensor has two rows.
- Each inner list holds three numbers, so each row has three entries.
- The shape attribute reports (2, 3): two entries along the first dimension and three along the second.
Reshape Regroups the Same Values
The reshape method takes a new shape and returns a tensor with that shape. The starting tensor holds six values, so any new shape must also account for exactly six entries. Asking for a shape with a different total count cannot succeed. For example, a request for a shape that holds eight entries cannot be satisfied by six values.
The important point is the order of the values. Reshaping does not shuffle or recalculate anything. The six values keep their sequence, and the new shape determines where each row begins and ends. Reading the same six values in order into three rows of two produces a different grouping of the same numbers. The values themselves are not changed.
What do you think happens?
Before reading on: if the six values 1 through 6 are reshaped into three rows of two, which pairs appear as the rows?
Reveal answer
The rows are 1 and 2, then 3 and 4, then 5 and 6.
Reshape keeps the original sequence 1, 2, 3, 4, 5, 6 and simply reads it into rows of two. The same numbers appear in the same order, only grouped differently.
tensor([[1, 2],
[3, 4],
[5, 6]])- The argument (3, 2) is a new shape with six entries, matching the original count.
- The output reads the original sequence in order, two values per row.
If a reshaped result shows values that have been reordered or multiplied, treat that surprise as a signal to check the input, not as normal reshape behavior. Reshape only regroups. The source also notes that .view() is the more common command for the same kind of change. Both change the shape of a tensor, and PyTorch offers several syntax options for the same computation. The difference between them lies in how each handles the layout of the underlying data. The source explains that .view() requires contiguous data and fails otherwise, while .reshape() copies data when necessary to produce the requested shape.
Transpose Trades Rows and Columns
The transpose attribute, written .T with no parentheses, flips a two-dimensional tensor so that its rows become its columns and its columns become its rows. The first row of the original becomes the first column of the result, and the same pattern continues for every row. Because a transpose swaps the two dimensions, the shape changes as well: a two-by-three tensor becomes three-by-two.
torch.Size([3, 2])
tensor([[1, 4],
[2, 5],
[3, 6]])- A.T has no parentheses because it is an attribute, not a method call.
- The shape changes from (2, 3) to (3, 2).
- The first printed row, 1 and 4, is the first column of A read top to bottom.
Matrix Multiplication with matmul
The matmul method multiplies two tensors as matrices. In this program the tensor is multiplied by its own transpose. The original is two by three and the transpose is three by two. Matrix multiplication requires the inner dimensions to agree: the number of columns in the first tensor must equal the number of rows in the second. Each output entry combines one row of the first tensor with one column of the second.
- 1The first tensor has m rows and n columns.
- 2The second tensor must have n rows, so its inner dimension matches the first tensor's n columns.
- 3The result keeps the outer dimensions m and p, giving a tensor with m rows and p columns.
torch.Size([2, 2])
tensor([[14, 32],
[32, 77]])- The inner dimensions are 3 and 3, so the product is defined.
- The outer dimensions are 2 and 2, so the result has shape 2 by 2.
Checking one entry of the product by hand
Verify the top-left entry of A.matmul(A.T), where A is the tensor with rows 1, 2, 3 and 4, 5, 6.
- 1Identify the row and column
The top-left entry combines the first row of A, which is 1, 2, 3, with the first column of A.T. That column is the first row of A read as a column, which is also 1, 2, 3.
- 2Multiply matching positions
Pair the entries by position: 1 times 1, 2 times 2, and 3 times 3.
- 3Add the products
$14 = 1 \cdot 1 + 2 \cdot 2 + 3 \cdot 3$, which matches the top-left value in the printed result.
The top-left entry is 14. Checking a single entry this way is a fast test when a whole result looks wrong.
The printed product is the two-by-two tensor with rows 14 and 32 in the first row and 32 and 77 in the second. The off-diagonal entries are equal, which reflects the fact that the product is formed from a tensor and its own transpose. Use the other entries as further checks: the bottom-right value 77 is the sum 4 times 4, plus 5 times 5, plus 6 times 6.
Reshape Versus Transpose Side by Side
Reshape and transpose both change the layout of the same six values, yet they treat element order differently. Reshape keeps the flat sequence and regroups it. Transpose swaps the roles of rows and columns, so the elements move to new positions in the grid.
| Operation | What it changes | Element order | Example result on the same six values |
|---|---|---|---|
| reshape(3, 2) | Groups the values into a new shape | Kept: values read in the same sequence | Rows 1 2, 3 4, 5 6 |
| .T | Swaps rows and columns | Changed: each column becomes a row | Rows 1 4, 2 5, 3 6 |
Same six values, two different operations
Mistakes Beginners Make
Common mistake Assuming reshape reorders or recalculates the values
What it looks likeExpecting the six values to come out shuffled or multiplied after reshape(3, 2).
Why it failsReshape only regroups the existing sequence into a new shape. The six values keep their order.
Do this insteadRead the output as the same numbers in the same sequence, split at new row boundaries. Treat any reordering as a sign that the input differs from what you think.
Common mistake Requesting a shape with the wrong total count
What it looks likeAsking for a shape that holds more or fewer than six entries when the tensor has six values.
Why it failsThe new shape must account for exactly the same number of entries as the original.
Do this insteadMultiply the dimensions of the new shape and confirm the product equals the original total before running the call.
Common mistake Confusing .T with a method call
What it looks likeWriting A.T() when the transpose is an attribute.
Why it failsThe transpose is written .T with no parentheses, while matmul and reshape are methods that take parentheses.
Do this insteadUse A.T for the transpose and A.matmul(A.T) for the product.
Common mistake Multiplying tensors without checking inner dimensions
What it looks likeMultiplying a 2 by 3 tensor by another 2 by 3 tensor directly.
Why it failsThe columns of the first tensor (3) must equal the rows of the second (2). They do not agree, so the product is not defined.
Do this insteadCompare the inner dimensions first. Transposing the second tensor makes the inner dimensions match in the source example.
Common mistake Believing the first error you see is the root cause, when an earlier shape assumption is the real problem
Finding Where a Result Goes Wrong
When an output differs from what you expected, move forward through the program one operation at a time and check the shape and values after each line. The first line where the observed result departs from your prediction is the failure location. Everything after that point may be a consequence of that line rather than a separate cause.
Given the tensor A with rows 1, 2, 3 and 4, 5, 6, predict the shape and first row of A.reshape(3, 2) and A.T before running either line. Then check your predictions against the printed outputs. If your prediction differs, identify the first line where it went wrong.
Hints
- Multiply the dimensions of the new shape and confirm the total is six.
- For the transpose, the first row of the result is the first column of A.
- The reshape keeps the sequence 1 through 6 and only changes the row boundaries.
Key Takeaways
- Reshape changes how a fixed sequence of values is grouped into rows and columns. The total number of elements stays the same and the values keep their order.
- The .T attribute swaps the two dimensions of a two-dimensional tensor, so a two-by-three tensor becomes three-by-two and each column becomes a row.
- Matrix multiplication with .matmul() requires the inner dimensions to agree. Each output entry combines one row of the first tensor with one column of the second, so a 2 by 3 tensor times its 3 by 2 transpose gives a 2 by 2 result.
- When a result looks wrong, check shape and values line by line. The first line where the output departs from your prediction is where the problem begins.