Mathematical Foundations of Neural Networks
A tensor is a container for data, almost always numerical data.
From Numbers to Tensors
Neural networks work with numerical data, so they need a consistent way to hold that data. A tensor provides that container. The central idea is not that a tensor is an entirely mysterious new object. It is a broader category that includes matrices and extends the same organizing idea to an arbitrary number of dimensions.
A Tensor as a Numerical Container
A tensor is a container for data, almost always numerical data.
Think of the tensor as the container and the numerical values as the contents. The container may organize a small collection of values or a much more elaborate arrangement. What makes the idea useful is that the same general notion can cover data with different numbers of dimensions.
Reading a Tensor as Organized Data
Imagine a numerical container that begins with one arrangement of values and then gains additional dimensions.
Start with one value: A single numerical item is the smallest illustration of data being held in a container.
Arrange values along one dimension: The values now have one organizing direction. This illustrates a one-dimensional arrangement.
Arrange values along two dimensions: The values now have two organizing directions. A matrix is a two-dimensional tensor.
Continue adding dimensions: The same organizing idea can extend beyond two dimensions because tensors generalize matrices to an arbitrary number of dimensions.
A tensor is best understood as a general numerical container whose data can be organized across different numbers of dimensions.
Dimensions and Axes
In tensor terminology, a dimension is often called an axis. This gives two ways to describe the same structural feature: you can talk about how many dimensions a tensor has, or how many axes organize its numerical data.
The axis language is useful because it focuses attention on organization rather than on a special name for each possible number of dimensions. A matrix has two dimensions, so it has two axes in this terminology. A tensor with more dimensions has more axes while remaining within the same general category of numerical container.
Treating tensor and matrix as unrelated kinds of objects
The source defines a matrix as a two-dimensional tensor.
Fix:
Use tensor as the broader category and matrix as the specific two-dimensional case.Using dimension and axis as completely unrelated terms
In tensor terminology, a dimension is often called an axis.
Fix:
When discussing tensor structure, treat dimension and axis as two terms for the organizing directions.Thinking tensors are important only because they have many dimensions
The defining idea is that a tensor is a container for data, almost always numerical data.
Fix:
Keep both parts in view: the tensor contains numerical data and organizes it across dimensions.
Why Neural Networks Use Tensors
Machine-learning systems need a way to hold the numerical data they work with. Tensors provide that kind of container, which is why they are the basic data structure of current machine-learning systems. Their importance is practical: machine learning relies on tensors to represent the numerical data used by its systems.
| Term | Meaning in this topic | Relationship |
|---|---|---|
| Tensor | A container for data, almost always numerical data | Broad category |
| Matrix | A two-dimensional tensor | Specific tensor case |
| Dimension | An organizing direction of tensor data | Often called an axis |
| Axis | Another term for a tensor dimension | Describes tensor organization |
Check Your Understanding
Explain in your own words why a matrix can be described as a tensor. Then explain why the word axis may be used when discussing a tensor's dimensions.
Hints
- Start with the definition of a tensor as a container for data.
- Recall how many dimensions a matrix has.
- Use the terminology that connects a dimension with an axis.
A machine-learning system must work with numerical data. Give a short explanation of why tensors are central to that system rather than merely an optional way to arrange numbers.
Hints
- Focus on what machine-learning systems need to hold.
- Connect that need to the role of tensors as containers for numerical data.
Key Takeaways
- A tensor is a container for data, almost always numerical data.
- Tensors generalize matrices to an arbitrary number of dimensions.
- A matrix is a two-dimensional tensor.
- In tensor terminology, a dimension is often called an axis.
- Tensors are central to current machine-learning systems because those systems need containers for the numerical data they use.