Inspecting and Visualizing the Data

Inspecting data before building a model can show whether the task is easily solvable without machine learning, or whether the needed information is not in the data.

Programming

A Spreadsheet Before a Model

Imagine you are handed a spreadsheet of flower measurements and asked to build a model that predicts each flower's species. It is tempting to open a learning algorithm right away and start training. The source material for this topic recommends a different first move: look at the data before you build anything.

The reason is that the data determines what is possible. If the answer can be read straight from the table, a model may be unnecessary. If the information you need is simply not in the table, no model will recover it. And if the numbers themselves contain errors, a model will learn those errors along with everything else.

Two Outcomes Inspection Can Reveal

The source names two outcomes that inspection can reveal. The first is that the task may already be easy. If a simple look at the data answers the question, building a learned model adds complexity without adding value. A rule or lookup that reads the answer off the table is the simpler and more honest solution.

The second outcome is that the desired information might not be contained in the data at all. In that case the problem is not the choice of algorithm. The dataset simply lacks what you are trying to predict, and you need to collect different data or reframe the question. No amount of model tuning can supply a column that was never recorded.

Both outcomes come from the same habit: reading the data critically before committing to a method. Inspection is cheap compared with training a model and then discovering that the model cannot succeed for reasons that were visible in the table from the start.

What sequence of checks tells me whether a simple rule or lookup already answers the question, or whether the data lacks the information needed to answer it at all?

Where Inspection Fits in the Workflow

Inspection sits before any modeling decision. You load the data, look at it, and check it for abnormalities. Only after that do you decide whether a simple rule answers the question or whether a model is worth building. The second purpose of inspection, covered in the next section, is closely tied to this step: it is where unusual values are caught before a model can absorb them.

Where does data inspection fit in the overall workflow, and what comes before and after it in building a model?

Spotting a Value in the Wrong Unit

The second purpose of inspection is to find abnormalities and peculiarities. The source gives a concrete picture of one: some of the iris measurements may have been recorded in inches rather than centimeters. Nothing in the column header or the number itself has to look wrong at a glance. A value of 2.0 is a perfectly ordinary number. Its meaning depends entirely on the unit it was recorded in, and a mixed set of units can quietly distort every calculation that uses those values.

The source also makes a broader point: inconsistencies in data and unexpected measurements are very common in the real world. So inspection is not a special rescue for unusual datasets. It is a routine check you should expect to need, because real data rarely arrives perfectly clean.

The problem is hard to notice by eye. In the illustrated column below, four values cluster around 5 and one sits far below them. The lone value is still a valid-looking number. Only a comparison against the rest of the column, or against knowledge of how the measurement was taken, reveals that it does not belong. This example is generated for teaching and is not taken from the source's data.

How can a value recorded in the wrong unit, such as inches mixed into a column of centimeters, stand out from the rest of the measurements when they are laid side by side?

A Simple Python Check

Here is a small generated example of inspection in Python. Five sepal length values are stored in a list. The list stands in for a real dataset, but the habit it demonstrates is the one the source recommends: look at the range of values before trusting them. We begin by asking for the smallest and largest values.

python

sepal_lengths = [5.1, 4.9, 4.7, 2.0, 5.0]
print(min(sepal_lengths), max(sepal_lengths))

sepal_lengths = [5.1, 4.9, 4.7, 2.0, 5.0]
print(min(sepal_lengths), max(sepal_lengths))

Output (expected)

2.0 5.1

What do you think happens?

Before running it, what do you think the smallest and largest values printed will be?

Reveal answer

Answer:

2.0 5.1

min() returns the smallest value in the list and max() returns the largest. The 2.0 stands out as far below the other four values, which is exactly the kind of peculiarity inspection is meant to surface.

A range printout alone does not say which value is wrong. To surface suspicious values for closer examination, you can test each value against an expected range. The range below is an assumption chosen for this generated example, and in practice it should come from knowledge of the measurement itself.

python

sepal_lengths = [5.1, 4.9, 4.7, 2.0, 5.0]
for value in sepal_lengths:
    if value < 4.0 or value > 8.0:
        print("Suspicious value:", value)

sepal_lengths = [5.1, 4.9, 4.7, 2.0, 5.0]
for value in sepal_lengths:
    if value < 4.0 or value > 8.0:
        print("Suspicious value:", value)

Output (expected)

Suspicious value: 2.0

Walk through it

1
sepal_lengths holds the five measurements.
2
The for loop takes one value at a time from the list.
3
The condition value < 4.0 or value > 8.0 is true only when the value falls outside the expected range.
4
The print call flags the value for closer examination rather than changing or deleting it.

Notes on the code

The list is loaded and each value is visited in turn.
The if test compares each value against the assumed lower and upper limits.
Values outside the range are printed as suspicious, so a person can examine them further.

Resolving a Flagged Value

The check above only raises a flag. It does not decide what the value means. The source is clear that a flagged value should be resolved with evidence about its unit or origin, rather than deleted or kept on assumption. Looking at the flagged 2.0, you might find that it came from a different measurement system, that a data entry step went wrong, or that it is genuinely unusual. Each of these possibilities needs evidence before you act on it.

Mistakes Beginners Make with Inspection

Common mistake
Training a model first and inspecting later
What it looks like
Feeding the flower measurements straight into a learning algorithm and only checking the data after the model performs poorly.
Why it fails
Any errors in the numbers, including values recorded in the wrong unit, are learned along with everything else, and a model may be unable to succeed for reasons visible in the table from the start.
Do this instead
Look at the data, check its ranges and units, and only then decide whether a model is needed.
Common mistake
Assuming an ordinary-looking number is correct
What it looks like
Treating 2.0 as a valid sepal length because it looks like a normal decimal.
Why it fails
A number's meaning depends on its unit. A value that looks perfectly ordinary can be inches rather than centimeters.
Do this instead
Compare each value against the rest of its column and check how the measurement was taken.
Common mistake
Deleting a flagged value without evidence
What it looks like
Removing every value outside an assumed range without looking into where it came from.
Why it fails
The source says flagged values should be resolved with evidence about unit or origin, not deleted on assumption.
Do this instead
Investigate the flagged value's unit or origin, and decide with that evidence in hand.
Common mistake
Assuming a model can recover information that the dataset never recorded

Practice Spotting the Odd Value

EASY

A generated column of heights, recorded in meters, reads 1.72, 1.68, 170, and 1.80. Which value stands out, what unit do you suspect it was recorded in, and what evidence would you look for before deciding what to do with it? Then write a short Python check that flags any height outside an assumed range of 1.0 to 2.5.

Hints

Compare each value against the others in the same column before deciding it is wrong.
A flagged value should be resolved with evidence about its unit or origin, not deleted on assumption.
Reuse the loop and if-test pattern from the earlier Python check.

Your attempt (not saved)

Key Takeaways

Inspecting data before building a model is a useful first step because the data determines what is possible.
Inspection can show that a task is easy enough to solve with a simple rule or lookup, or that the needed information is missing from the data.
Abnormalities such as values recorded in the wrong unit can look perfectly ordinary, so they must be found by comparing values against the rest of the column or against knowledge of how they were measured.
A simple Python check, such as testing each value against an expected range, can surface suspicious values for closer examination.
Flagged values should be resolved with evidence about their unit or origin, rather than deleted or kept on assumption.

Related concepts

Choosing Between Simple Rules and Machine Learning

More concepts starting with I