NumPy arrays
A tensor is a container for data, almost always numerical data.
From Numbers to Tensors
When a machine-learning system works with numerical data, it needs a way to hold that data in an organized form. A tensor provides that container. The most useful way to approach a tensor is not as a mysterious new mathematical object, but as a broader category that includes matrices and extends the same organizing idea to any number of dimensions.
A tensor is a container for data, almost always numerical data.
Dimensions and Axes
A tensor's dimensions describe how its numerical data is organized. Tensors generalize matrices to an arbitrary number of dimensions, so a matrix is one particular case: a matrix is a two-dimensional tensor. In tensor terminology, a dimension is often called an axis.
A Worked Shape Example
Adding dimensions to numerical data
Compare three generated arrangements of numerical data and identify what changes as dimensions are added.
One dimension: Consider the generated sequence [2, 4, 6]. It is organized along one dimension, so it can be described as a one-dimensional tensor.
Two dimensions: Now organize generated values into rows: [[2, 4, 6], [8, 10, 12]]. This is a matrix, and therefore it is a two-dimensional tensor.
Three dimensions: Add another layer of rows: [[[2, 4, 6], [8, 10, 12]], [[14, 16, 18], [20, 22, 24]]]. This arrangement has three dimensions, so it is a three-dimensional tensor.
Name the dimensions: Each dimension in the three-dimensional arrangement can be referred to as an axis. The values are still numerical data; the added dimensions describe how that data is organized.
The key change is the number of dimensions used to organize the values. A matrix is already a tensor; adding dimensions produces tensors with more axes.
This example illustrates the generalization rather than a special category of values. The numerical contents can vary, while the tensor idea remains the same: data is held in a container and organized across one or more dimensions.
Why Machine Learning Uses Tensors
Machine-learning systems need a way to hold the numerical data they work with. Tensors provide that kind of container, which makes them the basic data structure of current machine-learning systems. Their importance is practical: machine learning relies on tensors to represent the numerical data its systems use.
Tensors are not merely an optional way to organize numbers. They are central because machine-learning systems rely on them to represent the numerical data those systems use.
Common Conceptual Mistakes
Treating a tensor as something completely separate from a matrix.
A matrix is a two-dimensional tensor.
Fix:
Treat a matrix as one specific member of the broader tensor category.Confusing an axis with a new kind of numerical value.
In tensor terminology, a dimension is often called an axis.
Fix:
Use axis as the name for one dimension of the tensor's organization.Thinking that tensors are important only because they have many dimensions.
The practical importance of tensors comes from their role as the basic data structure of current machine-learning systems.
Fix:
Remember both parts: a tensor organizes data, almost always numerical data, and machine-learning systems rely on it to represent the numerical data they use.
Practice Check
Explain the relationship among a matrix, a tensor, a dimension, and an axis in your own words. Then describe what changes when a two-dimensional numerical arrangement becomes a three-dimensional one.
Hints
- Start with the fact that a matrix has two dimensions.
- Use the tensor definition as a container for numerical data.
- Remember that a dimension is often called an axis.
What do you think happens?
A matrix is two-dimensional. Is it also a tensor?
Reveal answer
Answer: Yes
A matrix is a two-dimensional tensor. Tensors generalize matrices to an arbitrary number of dimensions.
Essential Takeaways
- A tensor is a container for data, almost always numerical data.
- Tensors generalize matrices to an arbitrary number of dimensions.
- A matrix is a two-dimensional tensor.
- A tensor dimension is often called an axis.
- Tensors are central to current machine-learning systems because those systems rely on them to represent the numerical data they use.
Key Takeaways
- A tensor is a container for numerical data.
- Matrices are two-dimensional tensors, while tensors can have an arbitrary number of dimensions.
- Each tensor dimension is often called an axis.
- Tensors are the basic data structure used by current machine-learning systems to represent numerical data.