Preprocessing Data for Neural Networks
Reshaping reorganizes a tensor's rows and columns into a target shape.
From Raw Data to Usable Input
A neural network cannot learn directly from raw sound, images, text, or an unprepared table. Preprocessing changes raw data into numerical forms that a neural network can accept and learn from. The main preparation steps in this topic are vectorization, normalization, reshaping, and handling missing values.
Think of preprocessing as preparing the representation of information, not changing the underlying information itself. A useful representation has a suitable tensor structure, numerical values that are easier for the network to use, and a clear treatment of missing entries.
Reshaping Without Losing Data
Reshaping reorganizes a tensor's rows and columns into a target shape. The target shape is the new arrangement requested by the reshape operation. Reshaping changes the tensor's structure, but it does not change the total number of coefficients stored in the tensor. It is therefore best understood as a change in arrangement rather than a change in content.
tensor.reshape((new_shape))Rearranging a six-coefficient tensor
A tensor has shape 2 by 3 and contains the coefficients 1, 2, 3, 4, 5, and 6. Consider reshaping it to a target shape of 3 by 2.
Read the original shape: The original arrangement has 2 rows and 3 columns, so it contains 2 multiplied by 3, or 6, coefficients.
Read the target shape: The requested arrangement has 3 rows and 2 columns, so it also contains 3 multiplied by 2, or 6, coefficients.
Compare the totals: Because both shapes account for six coefficients, the target shape preserves the tensor's total amount of data.
Interpret the result: The rows and columns are reorganized. The coefficients remain the same, so the operation changes arrangement rather than content.
A 2 by 3 tensor can be reshaped into a 3 by 2 tensor because both shapes contain six coefficients.
Checking the Target Shape
Before accepting a proposed target shape, multiply its dimensions and compare that result with the number of coefficients in the original tensor. The target arrangement is valid only when it accounts for the same total number of coefficients. Different rows and columns are expected; a different total is the problem.
Vectorizing Data for the Network
Vectorization produces tensors from sources such as text, images, and tabular data. It changes raw information into numerical structure that can be supplied to a neural network. In a digit-image example, reshaping is part of preprocessing before the data enters the network. The important question is what the requested dimensions represent: one dimension can represent the collection of images, while another represents the contents of one image after rearrangement.
Neural-network inputs and targets must be represented as tensors because preprocessing and network operations work with tensor structure. The input tensor carries the prepared examples, while the target tensor carries the corresponding outputs used during learning. Their shapes must represent the same collection of examples and their associated target values.
Keeping Numerical Values Usable
Normalization keeps values small and helps avoid problems caused by incompatible feature ranges. Neural-network data should usually contain small, relatively homogeneous values so that one feature's scale does not make the other features difficult to use.
| Preparation method | Main idea | What to remember |
|---|---|---|
| Scaling | Place values into a small range | The focus is on keeping numerical values small and making feature ranges more compatible |
| Standardizing | Transform each feature to a mean of 0 and a standard deviation of 1 | The focus is on the distribution of each feature |
Handling Missing Values
A dataset may lack a value for a feature in some training or test records. One safe approach is to represent a missing value as 0 only when 0 does not already have a meaningful interpretation for that feature. If 0 is meaningful, using it as the missing marker would make two different situations look identical: a genuine zero and an absent value.
Common Preprocessing Mistakes
Assuming that a different shape means the tensor contains different data
Reshaping reorganizes rows and columns while preserving the total number of coefficients when the target shape is valid.
Fix:
Compare the products of the dimensions before interpreting the visual difference.Accepting a target shape without checking its coefficient count
The proposed target accounts for only four coefficients.
Fix:
Multiply the target dimensions and verify that the result equals the original coefficient count.Sending raw data directly to the network
Raw data must first be changed into numerical tensor forms that the network can accept and learn from.
Fix:
Vectorize the data and prepare its tensor structure before it enters the network.Using zero as a missing marker when zero is meaningful
The network cannot distinguish the two situations from their representation.
Fix:
Use a representation that distinguishes missing entries from genuine zero values.Treating scaling and standardizing as identical descriptions
Scaling places values into a small range, while standardizing gives each feature a mean of 0 and a standard deviation of 1.
Fix:
Name the preparation goal precisely.
Practice the Preparation Check
A tensor has shape 4 by 5. A proposed preprocessing step asks for a target shape of 2 by 10. Decide whether the reshape preserves the total number of coefficients. Then identify whether the operation changes the tensor's content or only its arrangement. Finally, state one question you would ask before representing a missing value as 0.
Hints
- Multiply the original dimensions and then multiply the target dimensions.
- Reshaping is described as a change in arrangement rather than a change in content.
- Ask whether 0 already has a meaningful interpretation for the feature.
Practice solution
Compare the original shape 4 by 5 with the proposed target shape 2 by 10.
Count the original coefficients: The original shape contains 4 multiplied by 5, which is 20 coefficients.
Count the target coefficients: The target shape contains 2 multiplied by 10, which is also 20 coefficients.
Interpret the reshape: Because the totals match, the target shape preserves the tensor's total number of coefficients. The rows and columns are reorganized rather than the content being changed.
Check the missing marker: Before using 0 for a missing value, check whether 0 is already a meaningful value for that feature.
The reshape is valid under the coefficient-preservation rule, and 0 is safe as a missing marker only when it is not already meaningful for the feature.
Preparation Checklist
- Convert raw sources such as text, images, and tables into numerical tensor forms through vectorization.
- Choose a tensor arrangement that suits the neural network, using reshaping when the target shape preserves the total number of coefficients.
- Keep numerical values small and relatively homogeneous through appropriate normalization, scaling, or standardizing.
- Represent inputs and targets as tensors whose shapes describe corresponding examples and outputs.
- Handle missing values consistently, using 0 as a missing marker only when 0 has no meaningful interpretation for that feature.
Key Takeaways
- Reshaping reorganizes rows and columns into a target shape while preserving the total number of coefficients.
- A proposed target shape must account for exactly the same total number of coefficients as the original tensor.
- Vectorization changes raw text, images, and tabular data into numerical tensor forms that a neural network can use.
- Scaling keeps values small, while standardizing gives each feature a mean of 0 and a standard deviation of 1.
- Use 0 for missing values only when 0 is not already a meaningful value for that feature.