Manipulating Tensors in NumPy
Tensor slicing is the selection of specific elements in a tensor.
From Dataset to One Sample
A data tensor often contains multiple samples. In the MNIST example, those samples are images of digits. The practical question is how to work with one particular image when all of the images are stored together. Tensor slicing provides the answer: it selects specific elements from the tensor.
What do you think happens?
The tensor train_images contains many digit images. What does train_images[i] select when the first axis is the samples axis?
Reveal answer
Answer: One complete digit image
The index i is applied along the first axis. Because that axis identifies samples, the indexed position identifies one digit image.
Axis 0 as the Samples Axis
For data tensors used in deep learning, axis 0 is the samples axis. This means that positions along axis 0 correspond to complete data samples. In the MNIST example, moving from one position on axis 0 to another means moving from one digit image to another.
The important meaning of an index is determined by the axis it addresses. An index applied along the samples axis identifies a sample, rather than automatically identifying a pixel or another internal part of that sample.
Reading a Slice as a Selection
Tensor slicing is the selection of specific elements in a tensor.
A slice should be understood as an operation that chooses part of a larger tensor. It does not automatically mean the entire dataset. The selection may identify one sample or a group of samples, depending on how the samples axis is selected.
Tracing One MNIST Selection
Interpret train_images[i] when train_images stores multiple digit images and axis 0 is the samples axis.
Start: Begin with the complete data tensor train_images, which contains multiple digit-image samples.
Apply the index: Apply the index i along the tensor's first axis, axis 0.
Identify the result: The indexed position identifies one complete digit image, because axis 0 is the samples axis.
train_images[i] selects one digit image from the data tensor.
Samples and Data Batches
| Concept | Meaning | Relationship to axis 0 |
|---|---|---|
| Sample | One complete item in the data tensor, such as one digit image | One position along the samples axis identifies it |
| Data batch | A group of samples considered together | The grouped samples are organized along the samples axis |
A sample and a batch are related but not interchangeable. A sample is one complete data item. A data batch contains a group of samples considered together. Because the samples are organized along axis 0, selecting one position on that axis identifies one sample, while selecting multiple positions identifies a group of samples.
Common Misreadings
Assuming that a slice always represents the entire dataset.
The selection identifies a particular position along axis 0, so it selects one sample in the MNIST example.
Fix:
Read the selection as a choice from the larger tensor. With the samples axis involved, one indexed position identifies one sample.Ignoring the meaning of axis 0.
In the data tensor described here, axis 0 is the samples axis.
Fix:
Identify the axis first. Then interpret an index along axis 0 as a selection among complete samples.Confusing one sample with a data batch.
A sample is one data item, while a data batch is a group of samples considered together.
Fix:
Use sample for one complete item and batch for a group selected or considered together.
Trace every selection in three stages: start with the complete data tensor, identify the axis being selected, and describe the resulting sample or group of samples. This gives the brackets an explicit meaning instead of treating them as unexplained notation.
Trace the Selection
A tensor contains multiple MNIST digit images, and its axis 0 is the samples axis. Explain what happens when a selection identifies one position along axis 0. Then explain how the result differs when the selection identifies a group of positions along axis 0.
Hints
- Start by naming what axis 0 represents.
- One position on the samples axis identifies one complete sample.
- A group of positions identifies multiple samples considered together.
Checking a Selection
Explain the result of selecting one position along axis 0 from a tensor of digit images.
Name the axis: Axis 0 is the samples axis.
Follow the selection: The selected position identifies one complete sample in the tensor.
Classify the result: Because the result is one digit image, it is a sample rather than a data batch.
The selection produces one sample from the larger data tensor.
Key Takeaways
- Tensor slicing selects specific elements from a tensor.
- For the data tensors described here, axis 0 is the samples axis.
- Selecting one position along axis 0 identifies one complete sample, such as one MNIST digit image.
- Selecting a group along axis 0 identifies multiple samples considered together.
- A sample is one data item, while a data batch is a group of samples.
Key Takeaways
- Tensor slicing is the selection of specific elements in a tensor.
- Axis 0 is the samples axis in the data-tensor example.
- An index along axis 0 can identify one complete sample.
- A group selected along axis 0 represents multiple samples considered together.
- A sample and a data batch are different: one is a single item, and the other is a group of items.