Data Representation in Deep Learning
Tensor slicing is the selection of specific elements in a tensor.
From Dataset to Sample
A deep learning data tensor often contains multiple samples. In the MNIST example, the samples are images of digits. The complete tensor represents the available collection, but many operations need to work with one particular image or with a selected group of images. Tensor slicing provides the selection mechanism.
Tensor slicing is the selection of specific elements in a tensor. When the first axis is the samples axis, selecting along that axis identifies which sample or samples are being selected.
The First Axis as a Sample Selector
Axis 0 is the samples axis in the data tensors described here. This means that the first position in a tensor index identifies a position among complete samples. In the MNIST example, applying the index i to train_images selects one digit image. The key idea is that i is not simply selecting an unexplained position: it is selecting a sample because the first axis has the samples-axis role.
A sample is one individual data item in the tensor. In the MNIST example, one sample is one digit image.
A data batch is a group of samples considered together. Those samples occupy multiple positions along the samples axis.
Tracing a Selection
Selecting One Digit Image
Interpret the selection train_images[i] when train_images contains multiple digit images.
Start: Begin with the complete train_images data tensor, which contains multiple samples.
Apply the index: Apply i along the tensor's first axis. Because the first axis is the samples axis, the index identifies a sample position.
Reach the result: The selection arrives at one complete sample: one digit image.
train_images[i] selects one digit image from the tensor.
The same mental model applies when the selection identifies a group rather than one position. For example, the generated notation tensor[2:5] represents a selection of positions 2 through 4 along axis 0. That selection refers to a group of samples, so it can be understood as a data batch rather than as one sample. The important point is that slicing specifies a selection; it does not automatically mean the entire dataset.
Reading Tensor Selections Correctly
When interpreting a tensor selection, trace it in three parts: begin with the complete data tensor, identify the axis receiving the index or slice, and determine whether the result is one sample or a group of samples. For the MNIST expression train_images[i], the first axis is axis 0, axis 0 is the samples axis, and the result is one digit image.
Treating train_images[i] as an arbitrary element with no data meaning.
The index is applied along the first axis, and that axis is the samples axis.
Fix:
Interpret the indexed position as identifying one complete digit-image sample.Confusing one sample with a data batch.
A sample is one individual data item, while a batch is a group of samples considered together.
Fix:
Ask whether the selection identifies one position on axis 0 or multiple positions on axis 0.Assuming every slice means the entire dataset.
Slicing means that a particular selection is being made.
Fix:
Trace which elements or sample positions the selection identifies.
Practice the Mental Trace
Suppose a data tensor contains multiple digit-image samples. Without executing anything, explain what is selected by tensor[0] and what is selected by tensor[2:5] when axis 0 is the samples axis.
Hints
- Start by identifying the axis receiving the selection.
- Remember that axis 0 identifies sample positions.
- Decide whether each selection identifies one position or a group of positions.
What do you think happens?
What kind of result should you expect when selecting one index along axis 0 of a tensor whose axis 0 is the samples axis?
Reveal answer
Answer: One complete sample
A single index selects one position along the samples axis. In the MNIST example, that position identifies one digit image.
Key Takeaways
- Tensor slicing selects specific elements from a tensor.
- Axis 0 is the samples axis in the data representation described here.
- Selecting one index along axis 0 identifies one complete sample, such as one MNIST digit image.
- Selecting multiple positions along axis 0 identifies a group of samples.
- A sample is one data item, while a data batch is a group of samples considered together.
Key Takeaways
- Tensor slicing is the selection of specific elements in a tensor.
- Axis 0 serves as the samples axis in the described data tensors.
- An index such as train_images[i] selects one complete digit-image sample.
- A range selection along axis 0 can identify a group of samples, which forms a data batch.
- To interpret a selection, trace the complete tensor, the selected axis, and the resulting sample or group.