Class Balance in Datasets
More generally, binary classification is any classification problem where the target variable has two classes.
Counting Before Modeling
Before any model can learn from movie reviews, you need to know what the data looks like. A useful first question is simple: how many examples belong to each class? The answer comes out as a pair of numbers, one per class, and that pair is the first thing to compare against what you expect. If the numbers are not what you expected, everything trained on that data is suspect.
This article follows one dataset check from start to finish. You will see what the check is counting, what the expected answer is, how to tell which split and which line produced a mismatch, and why the balance of the counts matters for the model that comes later.
Sentiment Analysis as Two-Way Labeling
Sentiment analysis, in the sense used here, is a binary classification task: each review is assigned the label positive or negative based on its text.
The task takes a review and assigns it one of two labels, using only the text content of that review. Every review must land in exactly one of the two categories. Because the target has exactly two possible values, this is a standard binary classification problem. More generally, binary classification is any classification problem where the target variable has two classes.
The reviews are the inputs, the two labels positive and negative are the possible outputs, and the classifier is the learned rule that maps one to the other. The classes are the two labels, so every count you read from the dataset is a count of reviews per label.
Reading Class Counts in Each Split
The dataset was collected so that the two classes are balanced: there are as many positive reviews as negative ones. In the training set and in the test set, each class has 12500 samples. A balanced split therefore shows 12500 for the positive class and 12500 for the negative class, and the gap between the two counts is zero.
The counts are produced with np.bincount, which returns how many times each integer value appears in an array. Labels stored as 0 and 1 make the output a two-element array, with one count for each class. The check is run separately on the training labels and on the test labels.
[12500 12500]
[12500 12500]np.bincount(y_train)counts the labels in the training splitnp.bincount(y_test)counts the labels in the test split, separately- Each printed pair lists one count per class, so both pairs should read 12500 12500
Tracing Where a Check Diverges
When a dataset check does not match the expected output, the useful move is to compare each split separately and then trace the mismatch back to the line that produced it. The diagram below follows the check in order. It first counts the training labels and compares them with the expected 12500 per class. Only if the training counts match does it move on to the test labels. The first comparison that fails is the step where the check diverges from what was expected.
Locating a Mismatch in the Training Split
The training check prints [12500 11800] instead of the expected [12500 12500]. Find the step where the check diverges.
- 1Identify the expected result
Each class should have 12500 training samples, so the expected output is 12500 12500.
- 2Compare the observed counts
The first class matches at 12500, but the second class reads 11800, a shortfall of 700 samples.
- 3Decide which split failed
The mismatch appears in the training counts, so the test split is not the place to look first. The check diverged at the training comparison.
- 4Trace back to the line
The observed numbers came from
np.bincount(y_train), so the fault lies in how the training labels were produced or loaded, not in the test labels.
The check diverges at the training count step. The training split is unbalanced, and the test split should not be checked as the source of the problem.
From Review Text to Numbers
A review starts as a string of characters, and a machine learning algorithm cannot work with raw strings directly. The text must first be converted into a numeric representation that the algorithms can process. Preprocessing is the set of steps that prepares the text for that conversion, and it begins with cleaning up the raw content.
The source shows one concrete preprocessing step. Each document in the test data is passed through a replacement that swaps the HTML line-break marker for a space. The replacement is applied to every document in the list, so the cleaning covers the whole collection, not a single review.
- b'<br />' is a bytes literal, written with the b prefix, that matches the HTML line-break marker
- b' ' replaces the marker with a single space
- The list comprehension applies the replacement to every document in the list
Why a Lopsided Dataset Misleads
Balance matters because a model trained on a lopsided dataset can look accurate simply by favoring the larger class. A model that learns to output the majority label most of the time will score well on a metric that only counts correct answers, even though it has learned nothing about the difference between positive and negative reviews.
What do you think happens?
A dataset has 900 positive reviews and 100 negative reviews. A model always predicts positive. What fraction of its predictions do you expect to be correct?
Reveal answer
90 percent
The model is right on all 900 positive reviews and wrong on all 100 negative ones, so 900 out of 1000 predictions are correct. A score this high says nothing about whether the model can tell the classes apart.
| Model behavior | Score on a 900 to 100 dataset | What it actually shows |
|---|---|---|
| Always predicts the majority class | 90 percent | Looks accurate but never identifies a negative review |
| Discriminates between the two classes | Depends on how well it separates the reviews | Shows real ability to tell positive from negative |
A high accuracy on a lopsided dataset can come from the majority class alone
Common Mistakes with Class Counts
Common mistake Checking only one split
What it looks likeRunning np.bincount on the training labels and declaring the whole dataset balanced
Why it failsA dataset can look balanced in one split and not in the other, so each split is verified on its own.
Do this insteadRun np.bincount on the training labels and on the test labels, and compare each result with the expected 12500 per class.
Common mistake Reading the counts without comparing them to the expected values
What it looks likeSeeing [11800 12500] printed and moving on because the output looks like two normal numbers
Why it failsThe check only means something when the observed counts are compared to what was expected.
Do this insteadRead the pair as one count per class and check whether the gap between them is zero.
Common mistake Feeding raw review strings directly to an algorithm
Why it failsA machine learning algorithm cannot work with raw strings, so the text must first be converted into a numeric representation.
Do this insteadRun the preprocessing steps, such as replacing the line-break marker, and then the numeric conversion before training.
Common mistake Assuming a high accuracy means the model works
Why it failsOn a lopsided dataset, a model that always predicts the larger class can score well without discriminating between classes.
Do this insteadVerify the class counts first, and judge the model by how it handles both classes rather than by overall accuracy alone.
Practice: Judging a Split
A colleague reports that the training split of a sentiment dataset is balanced. The printed output of np.bincount(y_train) is [12500 11900]. Decide whether the training split matches the expected counts, name the class and the size of the gap, and say which line of the check you would inspect first.
Hints
- The expected output for a balanced split is 12500 for each class.
- Compare the second number of the pair with 12500 and subtract to find the gap.
- The observed counts came from
np.bincount(y_train), so that is the line to trace.
Key Takeaways
- 1Sentiment analysis here is binary classification: each review gets one of two labels, positive or negative, based only on its text.
- 2A balanced dataset has 12500 samples per class in both the training and test splits, and
np.bincountis used to verify each split. - 3When a check diverges, compare each split separately and trace the mismatch back to the line that produced it.
- 4Review text must be preprocessed and converted to a numeric representation before a machine learning algorithm can use it.
- 5Verify class counts before trusting accuracy, because a lopsided dataset can make a model look accurate by favoring the larger class.
At a glance
- Sentiment analysis in this source is a binary classification task: each review is labeled positive or negative from its text.
- The dataset is balanced, with 12500 positive and 12500 negative samples in both the training and test splits, verified with
np.bincount. - When a check diverges, compare each split separately and trace the mismatch to the line that produced it.
- Review text must be converted into a numeric representation, after preprocessing, before learning algorithms can use it.
- Counts come first because a lopsided dataset can make a model look accurate by favoring the larger class.