Preprocessing for Supervised Algorithms
Unsupervised learning can also serve as a preprocessing step, where a new representation of the data is passed to a supervised algorithm.
Two Jobs for Unsupervised Learning
Unsupervised learning is easy to describe in the abstract, but a working data scientist usually asks a more practical question: what are these methods actually for? In practice, unsupervised algorithms appear in two main roles. The first is as a tool for looking at data itself, helping a person build an understanding of what a dataset contains. The second is as a helper inside a larger pipeline, where the output of an unsupervised step feeds into a supervised algorithm.
This article walks through both roles, follows a single hypothetical scenario through them, and ends with the pitfalls that catch people out. The central idea to keep in mind is that the second role depends on a new representation of the data, and that a new representation can change how well a supervised algorithm performs, sometimes for the better and sometimes not at all.
Exploring Data With an Unsupervised Lens
Unsupervised algorithms are often used in an exploratory setting, where a data scientist wants to understand the data better rather than run them inside a larger automatic system.
A large share of unsupervised work happens in this exploratory setting. The goal is not to hand a decision to an automated system. Instead, the unsupervised algorithm acts as a lens: it helps a person see structure in the dataset, and the results inform that person's judgment about the data. This is why these methods are typically used on their own rather than as one component of a larger automatic system.
Comparing the Two Roles
The same family of algorithms can play two quite different parts. The table below contrasts the exploratory role with the preprocessing role, so you can tell which one you are looking at in a given project.
| Aspect | Exploratory use | Preprocessing use |
|---|---|---|
| Main goal | Understand the data better | Produce a new representation for a supervised algorithm |
| Who uses the result | The data scientist, to inform judgment | The supervised algorithm inside a pipeline |
| Typical setup | Used on its own | One component of a larger pipeline |
| Where the output goes | To a person reading it | Forward into the supervised step |
Two roles for an unsupervised algorithm
The Unsupervised Step as Preprocessing
The second common application treats an unsupervised algorithm as a preprocessing step. Rather than feeding the raw data straight into a supervised algorithm, the pipeline first passes the data through an unsupervised step that learns a new representation of it. The supervised algorithm then works with this transformed version of the data.
The flow of information is what matters here. The data enters in its original form, the unsupervised step reshapes it, and the reshaped data is what the supervised algorithm learns from. The diagram below shows that movement from left to right.
Notice what gets passed between the steps. The unsupervised step does not hand the supervised algorithm the raw data; it hands over the new representation. That representation is the only thing the supervised algorithm sees from this stage onward.
Tracing One Dataset Through the Pipeline
A hypothetical dataset passed through a preprocessing step
A data scientist has a dataset of records with many descriptive features and wants a supervised algorithm to learn from them. They consider adding an unsupervised step first, so the supervised algorithm learns from a new representation instead of the raw features.
- 1Start with the raw data
The records enter the pipeline in their original form, with the original features.
- 2Pass the records through the unsupervised step
The unsupervised step learns a new representation of the same records. The data is reshaped, not discarded.
- 3Feed the new representation to the supervised algorithm
The supervised algorithm learns from the transformed version of the data, not from the raw features.
- 4Measure each outcome separately
Compare accuracy, memory use, and time use against a run on the raw data. Each one is measured on its own, because none of the gains is guaranteed.
The extra step is only worth keeping if the measurements show a benefit. Adding the unsupervised step does not by itself promise a better model.
The before and after view below makes the change concrete. The unsupervised transform takes the features in their original form and produces a new representation, which is what the supervised algorithm then receives.
What do you think happens?
After adding the unsupervised step, will the supervised model always become more accurate?
Reveal answer
No, accuracy gains are not guaranteed
The source says a new representation can sometimes improve the accuracy of supervised algorithms, but neither accuracy nor resource savings is guaranteed to appear. The only safe approach is to measure the result.
Accuracy, Memory, and Time Trade-offs
The reason to add an unsupervised step to a supervised pipeline is that a new representation can change how well the supervised algorithm performs. The source identifies two kinds of benefit. One is accuracy. The other is resource use, which covers memory and time consumption.
| Outcome | What a new representation can do | How to evaluate it |
|---|---|---|
| Accuracy | Sometimes improves the accuracy of the supervised algorithm | Compare the supervised model's accuracy on the raw features and on the new representation |
| Memory use | Can lead to reduced memory consumption | Measure memory for the raw-data run and the transformed-data run separately |
| Time use | Can lead to reduced time consumption | Measure time for the raw-data run and the transformed-data run separately |
Three outcomes to measure separately
Order of Steps in a Pipeline
A pipeline is a sequence, and the order of its steps determines what each step is given. The sequence diagram below shows the messages passed in order: the dataset goes to the unsupervised step, and the new representation goes on to the supervised algorithm.
Mistakes That Catch People Out
The source notes that the practical pitfalls are where people usually get caught. The following mistakes come up when unsupervised methods are applied in practice.
Common mistake Assuming a new representation always improves accuracy
What it looks likeAdding an unsupervised step and expecting the supervised model to score higher without ever checking.
Why it failsA new representation can sometimes improve accuracy, but the gain is not guaranteed.
Do this insteadMeasure the supervised model's accuracy on the raw features and on the new representation, then compare the two.
Common mistake Treating memory and time savings as automatic
What it looks likeConcluding that a pipeline is efficient because it feels faster, without a measurement.
Why it failsResource savings are separate from accuracy and are not guaranteed to appear at all.
Do this insteadMeasure memory and time separately for the raw-data run and the transformed-data run.
Common mistake Bundling the accuracy and resource results together
What it looks likeReporting that the transformed version is better overall after measuring only one of the outcomes.
Why it failsAccuracy and resource use are separate outcomes, so a benefit in one says nothing certain about the other.
Do this insteadReport each outcome on its own, with its own measurement.
Common mistake Using an exploratory result as an automatic decision
What it looks likeFeeding the output of an exploratory analysis straight into another system without a person reviewing it.
Why it failsExploratory use is meant to inform a person's judgment about the data rather than run inside a larger automatic system.
Do this insteadKeep a person in the loop when the goal is understanding the data, and treat the output as input to judgment.
Common mistake Leaving the comparison setup unclear
Why it failsIf it is not clear what each step was given, a measured difference cannot be attributed to the representation itself.
Do this insteadWrite down the data each step sees before comparing results.
Applying the Pipeline Reasoning
A colleague says their new representation made the supervised model both more accurate and faster, so the unsupervised step must be worth keeping. Which claim in this statement needs its own separate measurement, and why?
Hints
- Think about whether accuracy and resource savings are the same outcome.
- Recall what the source says about whether these gains are guaranteed.
- Ask whether the colleague measured each outcome on its own.
Key Takeaways
- Unsupervised algorithms are often used in an exploratory setting, where the goal is to help a data scientist understand the data, and their output informs human judgment.
- Unsupervised learning can also be a preprocessing step: the raw data is reshaped into a new representation, and the supervised algorithm learns from that representation.
- A new representation can sometimes improve the accuracy of supervised algorithms and can lead to reduced memory and time consumption, but neither outcome is guaranteed.
- Accuracy and resource savings are separate outcomes and should be measured separately, and the order and inputs of each pipeline step should be clear before results are compared.