Overfitting in Deep Learning
Data augmentation creates varied training examples from existing images.
Why One Dataset Is Not Enough
When a computer vision model is trained with a small dataset, the original collection may not provide enough variety. Data augmentation addresses this by using existing images as starting points and presenting them in varied forms. The goal is to create a larger and more diverse set of training examples, helping prevent overfitting.
From One Image to Many Variations
Think of the original image as the input to a preparation process. An augmentation configuration determines which transformations are available, such as rotation, shifting, or flipping. Those transformations produce varied training examples from the original image. Across a small collection of images, this process increases both the apparent size and the diversity of the training dataset.
Following One Image Through Augmentation
Suppose a training image is prepared with an augmentation configuration that makes rotation, shifting, and flipping available. What changes in the training data?
Start with the existing image: The original image is the starting point rather than a newly collected image.
Apply configured transformations: The configuration determines which transformations are available while the training data is prepared.
Present varied examples: The model receives varied forms of the existing image, increasing the diversity of the training examples.
The original collection can supply a larger and more diverse set of training examples for model training.
Why Variation Helps Generalization
The purpose of augmentation is to increase the diversity of the training data and help prevent overfitting. Training with varied forms of existing images gives the model a broader set of training examples than the original collection alone. This encourages the training process to use the available variety instead of depending only on the limited forms in the small dataset.
The connection is indirect but important: augmentation increases dataset diversity, and that increased diversity is used to help prevent overfitting.
Reading the Generator Configuration
ImageDataGenerator is configured by passing named augmentation parameters into its call. The resulting configured generator is stored under the name datagen in the source configuration. To read this kind of statement, move from left to right: identify the object being called, inspect each named argument and its assigned value, and then identify the name receiving the result.
| Configuration element | Meaning in the configuration |
|---|---|
| ImageDataGenerator | The object being called to configure augmentation |
| rotation_range | A named augmentation parameter supplied to ImageDataGenerator |
| horizontal_flip | A named augmentation parameter supplied to ImageDataGenerator |
| Assigned value | The value written after a parameter name |
| datagen | The name receiving the configured generator |
Elements explicitly identified in the supplied source description
Tracing Data Through ImageDataGenerator
The configured ImageDataGenerator represents the selected augmentation configuration. Original images are supplied as the starting material, the configured transformations are applied while the training data is prepared, and the resulting varied examples are made available for model training. The important distinction is between the generator configuration and the later training process: a problem in the named arguments is different from a problem elsewhere in training.
When debugging, trace the configuration from left to right. First identify the object being called. Next inspect every named argument and its assigned value. Finally check the name receiving the result. This helps separate a configuration problem from a problem elsewhere in the training process.
Limits of the Supplied Configuration
Treating a parameter name as if it were its value
The source emphasizes separating each named parameter from the value assigned to it.
Fix:
Identify the name first, then inspect the value associated with that name.Assuming augmentation creates an unrelated image collection
Augmentation starts with existing images and applies transformations to them.
Fix:
Trace each varied example back to an existing training image and the configured transformations.Ignoring the name receiving the configured generator
The source configuration stores the configured generator as datagen, and debugging requires checking where the result is assigned.
Fix:
After reviewing the named arguments, check the receiving name as the final step of the left-to-right trace.Claiming that augmentation guarantees prevention of overfitting
The stated purpose is to help prevent overfitting, not to guarantee that it cannot occur.
Fix:
Describe augmentation as a way to increase diversity and help prevent overfitting.
Check Your Understanding
A configuration passes named arguments rotation_range and horizontal_flip into ImageDataGenerator, and the resulting object is assigned to datagen. Explain which items are parameter names, which items are assigned values, and which name receives the configured generator. Then explain why this configuration can help with a small image dataset.
Hints
- A parameter name identifies a configurable aspect of augmentation.
- An assigned value appears beside its parameter name.
- The receiving name is the name assigned the result of the ImageDataGenerator call.
- Connect the purpose to increased diversity and help with preventing overfitting.
Practice Solution
Classify the elements in the described configuration.
Identify the called object: ImageDataGenerator is the object being called to receive augmentation parameters.
Identify parameter names: rotation_range and horizontal_flip are the named parameters explicitly identified in the configuration.
Identify assigned values: Each parameter has a value assigned to it in the configuration. The value is distinct from the parameter name.
Identify the receiving name: datagen is the name used to store the configured generator.
Explain the purpose: The selected transformations can create varied training examples from existing images, increasing dataset diversity and helping prevent overfitting.
Read the configuration as object being called, named arguments with assigned values, and the name receiving the configured result.
Key Takeaways
- Data augmentation creates varied training examples from existing images.
- Applying transformations can artificially increase the size and diversity of a small training dataset.
- ImageDataGenerator accepts named augmentation parameters, including rotation_range and horizontal_flip.
- Read a configuration by separating each parameter name from its assigned value and then checking the name receiving the result.
- The purpose of augmentation is to increase training-data diversity and help prevent overfitting.
Key Takeaways
- Data augmentation uses transformations of existing images to create varied training examples.
- The resulting training data is larger and more diverse than the original collection alone.
- ImageDataGenerator is configured with named parameters such as rotation_range and horizontal_flip.
- A configuration should be read from left to right: called object, named arguments and values, then receiving name.
- Increased data diversity is intended to help prevent overfitting.