Preventing Overfitting
A Keras callback is an object passed to fit that is called at various points during training.
Training Without Guesswork
When training begins, you usually do not know in advance how many epochs will be needed to reach the best validation loss. One approach is to train long enough for overfitting to begin, determine a suitable number of epochs, and then start another training run from scratch. That approach wastes time. Keras callbacks let the training process observe what is happening and respond during the same run.
What a Callback Is
A Keras callback is an object passed to fit that is called at various points during training.
The important idea is timing. A callback does not replace the training process. Instead, it is given opportunities to respond at selected points while training is in progress. Those opportunities can be associated with batches, epochs, validation results, or the end of training. This lets a callback observe training behavior and take an action without requiring a separate trial run.
Metric Monitoring
A training-control callback needs a signal that describes training behavior. A monitored metric supplies that signal. For example, the callback may watch validation loss. After the relevant training progress is reported, the callback compares the current metric behavior with the behavior it has already observed. Its action depends on what the callback is designed to do and on how long the metric has failed to improve.
Patience is the configured period during which the monitored metric may fail to improve before the callback acts. A shorter patience causes action sooner; a longer patience allows more training to continue before action. The exact action depends on the callback. Patience therefore controls timing, not the purpose of the callback.
Reading a Patience Setting
A callback monitors validation loss. The loss improves, then fails to improve for the configured patience period. What should you expect the callback to do?
Observe: The callback receives training information at a selected training event and examines the monitored validation loss.
Compare: It checks whether the current metric has improved compared with the behavior it has already tracked.
Count: When the metric does not improve, the callback increases its count of unimproved training periods. When improvement occurs, that count is reset for the purpose of continuing to monitor the metric.
Act: When the configured patience period is reached, the callback performs its own designed action: preserving weights, interrupting training, or changing the learning rate.
Patience determines when a callback acts after insufficient improvement; it does not determine which action the callback performs.
Three Built-In Choices
| Callback | Primary response | Use it when |
|---|---|---|
| ModelCheckpoint | Saves weights during training and can keep only the best model according to a monitored metric. | Preserving the best model weights matters. |
| EarlyStopping | Interrupts training when a monitored metric stops improving for the configured patience period. | Continuing to train is no longer useful. |
| ReduceLROnPlateau | Changes the learning rate when validation loss stops improving. | You want to change the learning rate rather than immediately stop. |
Choosing by Desired Change
Training behavior suggests that the best validation result may occur before the final epoch. You want to preserve that result, stop when further training is no longer useful, or give training a chance to improve after validation loss stalls. Which callback matches each goal?
Preserve: Choose ModelCheckpoint when the important response is saving weights and keeping only the best model according to a monitored metric.
Stop: Choose EarlyStopping when the important response is interrupting training after the monitored metric has stopped improving for the configured patience period.
Adjust: Choose ReduceLROnPlateau when validation loss has stopped improving and the desired response is to change the learning rate instead of immediately stopping.
The correct callback is determined by the action you want to take, not merely by the fact that training behavior has changed.
Custom Callback Design
Built-in callbacks do not cover every possible training action. For a behavior that is not provided by a built-in callback, create a custom callback by subclassing keras.callbacks.Callback and implementing methods that run at selected training events.
A custom callback connects an event to an action that you define. The event might be an epoch start, a batch end, or training end. The callback receives training information at that point and can use its access to model information and training logs as part of the response. The key design choice is selecting the event and then implementing the callback method associated with that event.
Common Selection Mistakes
Choosing a callback only because the metric has stopped improving.
The same training behavior can call for different responses.
Fix:
First decide whether you need to preserve weights, interrupt training, or change the learning rate. Then choose ModelCheckpoint, EarlyStopping, or ReduceLROnPlateau.Treating patience as the callback's purpose.
Patience controls how long insufficient improvement is tolerated; it does not determine the callback's action.
Fix:
Separate the timing rule from the action. Identify the callback first, then understand how its patience setting delays that action.Assuming that a built-in callback can perform every possible training action.
Built-in callbacks do not cover every possible training action.
Fix:
Create a custom callback by subclassing keras.callbacks.Callback and implementing methods for selected training events.Ignoring the monitored metric when interpreting callback behavior.
Callback behavior is tied to the monitored metric and whether that metric improves.
Fix:
Name the monitored metric and explain how its improvement or lack of improvement leads to the callback's response.
Practice the Decision
For each goal, choose ModelCheckpoint, EarlyStopping, ReduceLROnPlateau, or a custom callback. Then explain which monitored training behavior and which response led to your choice: preserve the best weights; stop after a monitored metric fails to improve for the configured patience period; change the learning rate when validation loss stops improving; or perform an action not covered by the built-in callbacks.
Hints
- Separate the observed training behavior from the desired response.
- Patience tells you when a response should happen, not which response the callback performs.
- If no built-in callback covers the required action, consider subclassing keras.callbacks.Callback.
Key Takeaways
- A Keras callback is an object passed to fit that receives control at selected points during training.
- A monitored metric gives a callback the training signal it uses to decide whether behavior has improved.
- Patience counts how long insufficient improvement is tolerated before the callback acts.
- Use ModelCheckpoint to preserve weights, EarlyStopping to interrupt unhelpful training, and ReduceLROnPlateau to change the learning rate when validation loss stalls.
- Subclass keras.callbacks.Callback when the desired training response is not covered by a built-in callback.
Key Takeaways
- Callbacks let Keras observe training and respond without requiring a separate trial run.
- Metric monitoring and patience determine when a callback should act.
- ModelCheckpoint preserves important weights, EarlyStopping ends unhelpful training, and ReduceLROnPlateau changes the learning rate when validation loss stalls.
- Custom callbacks connect selected training events to user-defined actions and can use available model information and training logs.