Tensor Data Types
Tensor rank counts axes; shape records the size along each axis; dtype identifies the contained data type.
Three Questions About One Tensor
When describing a tensor, ask three different questions. How many axes does it have? How large is it along each axis? What kind of data does it contain? These questions correspond to rank, shape, and data type, or dtype. Together, they describe both the tensor's structure and its contents.
Rank and shape describe structure. Dtype describes the kind of data contained in that structure.
Counting Axes as Rank
A tensor's rank is the number of axes in the tensor. In Python libraries such as NumPy, this quantity is also called ndim. Rank describes the tensor's structure, not the total number of values stored in it. A one-axis tensor has a different structure from a two-axis tensor, and a three-dimensional tensor has three axes.
Finding Rank from a Shape
A tensor has shape (3, 3, 5). What is its rank?
Read the shape: The shape contains the three entries 3, 3, and 5.
Count the entries: Counting the entries gives three, so the tensor has three axes.
State the rank: The tensor's rank is 3. In a Python library such as NumPy, its ndim is also 3.
The tensor has rank 3, or ndim 3.
Reading a Shape Tuple
A tensor's shape records its size along each axis. It is written as a tuple of integers. The number of entries in the shape equals the tensor's rank. Therefore, a shape with two entries describes a two-axis tensor, while a shape with three entries describes a three-axis tensor.
| Tensor structure | Shape example | Rank |
|---|---|---|
| Scalar | () | 0 |
| Vector | (5,) | 1 |
| Matrix | (3, 5) | 2 |
| 3D tensor | (3, 3, 5) | 3 |
The number of entries in a shape gives the tensor's rank.
The parentheses and comma matter for special cases. A vector can have the single-entry shape (5,), while a scalar has the empty shape (). These shapes show that a vector has one axis and a scalar has no axes.
Identifying the Data Type
The dtype identifies the kind of data contained in a tensor. Examples include float32, uint8, and float64. Unlike rank and shape, dtype does not describe how many axes the tensor has or how large it is along those axes. It describes the tensor's contents.
Separating Shape from dtype
A tensor is described by shape (3, 5) and dtype float32. What does each attribute tell you?
Interpret the shape: The shape has two entries, 3 and 5. This means the tensor has two axes, with sizes 3 and 5 along those axes.
Find the rank: Because the shape has two entries, the tensor's rank is 2.
Interpret the dtype: The dtype float32 identifies the kind of data contained in the tensor.
Shape (3, 5) describes structure; dtype float32 describes the contained data type.
Practical Interpretation
Read tensor descriptions in a fixed order. First count the shape entries to determine rank. Next read each shape entry as the size along its corresponding axis. Finally inspect dtype to determine the kind of data contained. This order keeps structural information separate from information about the tensor's values.
The source also identifies a rare char tensor as another possible data type. It notes that string tensors do not exist in NumPy, or in most other libraries, because tensors live in preallocated, contiguous memory segments and variable-length strings would prevent this implementation.
Common Interpretation Mistakes
Treating shape as the data type
The shape records sizes along axes. The dtype identifies the contained data type.
Fix:
Read (3, 5) as structural information and look separately for a dtype such as float32, uint8, or float64.Counting values instead of axes to find rank
Rank counts the entries in the shape, which correspond to axes, not the total number of values.
Fix:
Count the shape entries. The shape (3, 5) has two entries, so its rank is 2.Ignoring the empty shape of a scalar
A scalar has the empty shape ().
Fix:
Recognize that the empty shape represents a tensor with no axes.Confusing a one-entry shape with an empty shape
The shape (5,) describes a vector with one axis, while () describes a scalar with no axes.
Fix:
Count the entries: (5,) has rank 1, and () has rank 0.
Check Your Understanding
A tensor has shape (3, 3, 5) and dtype uint8. Identify its rank, state the size along each axis, and explain what uint8 describes.
Hints
- Count the entries in the shape to find rank.
- Match each shape entry to one axis.
- dtype describes the kind of data contained.
Compare a vector with shape (5,) and a scalar with shape (). Which has an axis, and what is the rank of each?
Hints
- The number of shape entries gives rank.
- A one-entry shape and an empty shape represent different structures.
Key Takeaways
- A tensor is described by rank, shape, and dtype.
- Rank counts the tensor's axes and is also called ndim in Python libraries such as NumPy.
- Shape is a tuple of integers recording the size along each axis.
- The number of entries in a shape equals the tensor's rank.
- dtype identifies the kind of data contained and is separate from shape and rank.
Key Takeaways
- Rank counts axes; it does not count the total number of tensor values.
- Shape records the size along each axis, and its number of entries gives rank.
- A vector can have shape (5,), a scalar has shape (), a matrix can have shape (3, 5), and a 3D tensor can have shape (3, 3, 5).
- dtype identifies the contained data type, such as float32, uint8, or float64.
- Shape and rank describe structure, while dtype describes contents.