Training and Prediction in Python
Machine learning is a field of study that trains algorithms to make predictions or decisions based on data.
A Program That Must Answer an Unseen Case
Suppose a program has to decide something about a case it has never seen before, such as whether a new number belongs in a group. The program cannot look up the answer, because the answer is not yet known. This is the situation that machine learning is built for. Before any code appears, it helps to hold one definition in mind, because nearly every idea in this course is a refinement of it.
What do you think happens?
If a program has seen some labeled examples but has not yet extracted any rule from them, can it make a useful decision about a brand new case?
Reveal answer
No, it needs something learned from the examples before it can decide about new cases
The source describes prediction as applying a trained model to a new case. A model is what training produces, so the learned result has to exist before prediction can use it.
Defining Machine Learning as a Field
Machine learning is a field of study that trains algorithms to make predictions or decisions based on data.
The central activity in this field is training. An algorithm is adjusted using data, and the adjustment is what makes it useful for predictions or decisions. The source points out that this field has two goals. The first is to understand the fundamental concepts and applications of machine learning, which is the focus of this article. The second is to learn the advantages and shortcomings of widely used machine learning algorithms. That second goal is only previewed here, since the source names those advantages and shortcomings without spelling them out in this introduction.
Training and Prediction as Two Moments
A machine learning system goes through two distinct moments. Training uses data that already has answers to shape the algorithm. The result of training is a trained model, which holds what was learned. Prediction then applies that trained model to a new case whose answer is not yet known. Mixing these two moments up is one of the most common early confusions, so it is worth keeping them clearly separate in your head.
Read the diagram from left to right. Example data goes into training, training produces a trained model, and that model is applied to a new case to produce a prediction or decision. The new case matters most for the purpose of the system, because it is data the algorithm was not trained on. The labeled examples are used only during training, while the new case arrives only at prediction time.
A Threshold Learner in Python
The following example is an illustration written for this article, not material taken from the source. It uses a deliberately simple algorithm that learns a single number, a threshold, from labeled examples. Each example is a pair of a number and a label. Study the code first, then follow how the training step changes the state of the program.
False
True- examples holds the labeled training data and is not changed by training
- threshold starts as None, meaning no learned value exists yet
- train reads the labeled examples and returns a single learned number
- predict uses whatever value is currently stored in threshold
- The call to train produces the learned number that predict depends on
Tracing the Threshold Through Program State
The before and after states show what training actually does in this example. The examples never change. What changes is the threshold variable, which moves from holding no value to holding a learned number. Once that number exists, the prediction step can use it on any new value, such as 5 or 7, without looking at the original examples again.
threshold = Nonethreshold holds no value
threshold = train(examples)threshold holds the learned number 7
Tracing a prediction after training
After the program has run the training line, what does predict(5) return, and which value does it compare against?
- 1Check the state
Training has already run, so threshold holds the learned number. Tracing the training step gives 7, the smallest value labeled 'yes'.
- 2Apply the trained value
predict(5)evaluates 5 >= threshold, which becomes 5 >= 7. - 3Read the result
5 is not at least 7, so the comparison is False.
predict(5) returns False. The examples were not consulted during this call; only the stored threshold was used.
Mistakes Beginners Make About Learning from Data
Common mistake Treating training and prediction as one step
What it looks likeThinking that calling predict on 5 is the same as running the labeled examples through the algorithm again.
Why it failsTraining uses data with known answers to produce a trained model. Prediction applies that model to a case whose answer is unknown.
Do this insteadName the two moments separately and ask which one you are in: is the program shaping the algorithm, or applying what was already shaped?
Common mistake Assuming prediction can run before training has produced anything
What it looks likeCalling predict in the threshold example before train has assigned a value to threshold.
Why it failsPrediction depends on the trained model. Before training, the threshold holds no learned value.
Do this insteadCheck that the learned result exists before using it, the same way the threshold is assigned before predict reads it.
Common mistake Believing the new case is part of the training data
Why it failsThe source emphasizes that the new case is data the algorithm was not trained on.
Do this insteadKeep the labeled examples and the new input in separate roles: examples for training, new cases for prediction.
Common mistake Confusing machine learning, the field, with one specific algorithm
Practice: Changing the Training Data
Keep the predict and train functions from the threshold example unchanged. Replace the examples list with [(1, 'no'), (4, 'no'), (6, 'yes'), (9, 'yes')]. Before running anything, trace the program state: what does threshold hold after training, and what does predict(5) return? Then predict(6) and predict(7). Finally, run the program and compare your trace with the output.
Hints
- Training returns the smallest value labeled 'yes', so find that value in the new list first.
- A prediction returns True when the new value is at least as large as threshold.
- The examples list is read only during training; after that, only threshold matters.
Key Takeaways
- Machine learning is a field of study that trains algorithms to make predictions or decisions based on data.
- Training adjusts an algorithm using data with known answers, and the result is a trained model that holds what was learned.
- Prediction applies the trained model to a new case whose answer is not yet known, and the labeled examples are not needed again for that step.
- In the threshold example, the training step changes the threshold variable from holding no value to holding a learned number, which the prediction step then uses.