Introduction to Machine Learning Algorithms

Machine learning is a field of study that trains algorithms to make predictions or decisions based on data.

Programming

Two Moments in a Learning System

Many introductions to machine learning begin by showing what a method does and how it behaves, leaving the underlying mathematics for later. That approach lets a newcomer build intuition about the most common algorithms before worrying about every equation. This article picks up where such an introduction tends to stop and asks a simple question: if the algorithms can be understood intuitively, what mathematics is sitting underneath them?

What do you think happens?

After a machine learning algorithm has been trained, does the system hold learned values shaped by the training data, or does it only hold the original examples it was given?

It only holds the original examples
It holds learned values shaped by the examples

Reveal answer

Answer:

It holds learned values shaped by the examples.

Training adjusts an algorithm using data, and the result is a trained model that holds what was learned. The examples are used to shape the model; the trained model is what is kept for later use.

Machine Learning as Training Then Predicting

Definition

Machine learning is a field of study that trains algorithms to make predictions or decisions based on data.

Machine learning is a field of study. Its central activity is training algorithms so that they can make predictions or decisions based on data. Nearly every later idea in this course is a refinement of that definition, so it is worth holding it in mind before looking at any code. The definition also contains two distinct moments. Training uses data that already has answers to shape an algorithm. Prediction then uses that shaped algorithm on a case whose answer is not yet known.

What data goes into the training step, what comes out, and how does the prediction step reuse that output on a new case?

Read the diagram from left to right. Example data goes into training, training produces a trained model, and the model is then applied to a new case to produce a prediction or decision. The new case is the one that matters for the purpose of the system, because it is data the algorithm was not trained on. Training and prediction are two different moments in the life of a machine learning system, and mixing them up is one of the most common early confusions.

Watching a Learned Threshold Appear

The following example is an illustration written for this article, not material taken from the source. It uses a deliberately simple algorithm that learns a single number, a threshold, from labeled examples. Study the code first, then follow how the training step changes the state of the program.

python

labeled_examples = [(2, 'small'), (3, 'small'), (8, 'large'), (9, 'large')]

threshold = None

def train(examples):
    small_values = [x for x, label in examples if label == 'small']
    large_values = [x for x, label in examples if label == 'large']
    return (max(small_values) + min(large_values)) / 2

threshold = train(labeled_examples)

def predict(value):
    if value >= threshold:
        return 'large'
    return 'small'

print(predict(5))
print(predict(7))

labeled_examples = [(2, 'small'), (3, 'small'), (8, 'large'), (9, 'large')]

threshold = None

def train(examples):
    small_values = [x for x, label in examples if label == 'small']
    large_values = [x for x, label in examples if label == 'large']
    return (max(small_values) + min(large_values)) / 2

threshold = train(labeled_examples)

def predict(value):
    if value >= threshold:
        return 'large'
    return 'small'

print(predict(5))
print(predict(7))

Output (expected)

small
large

Notes on the code

labeled_examples holds the training data, each pair being a value and its known label
threshold starts as None, meaning no learned value exists yet
train reads the examples and returns a number between the largest small value and the smallest large value
The assignment threshold = train(labeled_examples) is the training step
predict uses the stored threshold; it never reads labeled_examples
5 is below the threshold 5.5 so it is predicted small; 7 is above it so it is predicted large

What is stored in the program before training, what changes during training, and how does the stored value alter later behavior?

The before and after states show what training actually does in this example. The examples themselves never change. What changes is the threshold variable, which goes from having no value to holding a learned number. Once that number exists, the prediction step can use it on any new value without looking at the original examples again.

The Three Mathematical Foundations

The source gives an answer to the question of what lies underneath many introductory models: principles from three areas, probability theory, linear algebra, and optimization. Think of a machine learning model as a machine built from several parts. Some parts describe how uncertain or noisy the data is, some organize the data into a structured form that can be manipulated, and some decide how the model should change to become better at its task. Each of the three labels names a body of mathematics that supplies tools for one of these roles.

How do the three mathematical areas connect to one another and to the parts of a machine learning model they support?

The diagram is a simplification. It does not claim that every model uses every foundation equally, or that these three areas are the only mathematics involved. It reflects the statement that many models use principles from all three, so a single model can draw on more than one of them at once.

Why Theory Helps a Practitioner

The source states that a practitioner does not need to master every implementation detail of these algorithms, and that understanding the theory behind them is valuable though not required before using them intuitively. Knowing which foundation a model leans on tells you what kind of uncertainty it assumes, how its data is organized, and what it is searching over when it learns. That knowledge helps when a model behaves unexpectedly.

Predicting on New Cases

Worked example

Using the stored threshold on new values

After the training step in the threshold example, the program holds threshold = 5.5. Classify the new values 5 and 7 using only the stored threshold.

1
Check the state
The variable threshold holds 5.5. The list labeled_examples is no longer needed to answer the question.
2
Classify 5
5 is less than 5.5, so the condition value >= threshold is false and predict returns small.
3
Classify 7
7 is greater than 5.5, so the condition is true and predict returns large.
4
Confirm nothing changed
Neither prediction assigns anything. The stored threshold is still 5.5 after both calls.

Result

predict(5) returns small and predict(7) returns large, matching the printed output small then large.

Beginner Misconceptions About Learning From Data

Most early confusions come from treating training and prediction as one action, or from assuming the examples are kept and reread at prediction time. The table below separates what beginners often expect from what the threshold example actually does.

Beginner expectation	What actually happens
Training and prediction are the same step	Training produces a trained model; prediction applies that model to a new case
Prediction rereads the original examples	predict uses only the stored threshold, not labeled_examples
The examples change during training	The examples stay the same; the threshold variable gains a learned value
The new case was part of training	The new case is data the algorithm was not trained on

Misconceptions about training and prediction compared with the threshold example

Common mistake
Believing the theory must be mastered before any algorithm can be used
Why it fails
The source says understanding the theory is valuable but not required before using algorithms intuitively.
Do this instead
Start with intuition about what an algorithm does, then return to the foundations as needed.
Common mistake
Thinking a prediction changes the model
Why it fails
Prediction applies the trained model; the stored threshold stays at 5.5 after predict is called.
Do this instead
Treat training as the only step that assigns learned values.
Common mistake
Assuming the data is perfectly clean and certain
Why it fails
Data collected from the world almost always contains variation, which is why probability theory is one of the foundations.
Do this instead
Expect uncertainty in both the data and the predictions it produces.

Check Your Understanding

EASY

In the threshold example, the training examples are replaced with a different set of labeled pairs. Without running the code, describe which variable changes during training, which variable is read during prediction, and whether the prediction step still needs the original examples.

Hints

Look at which variable is assigned inside the line threshold = train(labeled_examples).
Check whether predict refers to labeled_examples anywhere in its body.

Your attempt (not saved)

Key Takeaways

Machine learning is a field of study that trains algorithms to make predictions or decisions based on data.
Training uses data with known answers to produce a trained model that holds what was learned; prediction applies that model to a new case.
Many machine learning models use principles from probability theory, which describes uncertainty, linear algebra, which organizes data as collections of numbers, and optimization, which searches for better model settings.
Understanding the theory behind algorithms is valuable but not required before using them intuitively.
Training changes stored program state; after training, prediction can use the learned value without rereading the original examples.

Related concepts

More concepts starting with I