Working with Binary Data
learn : GaussianNB , BernoulliNB , and MultinomialNB . GaussianNB can be applied to any continuous data, while BernoulliNB assumes binary data and MultinomialNB assumes count data (that is, that each feature represents an integer count of something, like how often a word appears in a sentence). BernoulliNB and MultinomialNB are mostly used in text data classification.
Start with the Feature Values
The important question is not simply whether your data is numerical. The important question is what each feature value represents. A feature might record whether something is present, how many times it occurs, or a continuous measurement. Naive Bayes provides different model variants for these different kinds of data: BernoulliNB assumes binary data, MultinomialNB assumes count data, and GaussianNB can be applied to continuous data.
Presence Versus Frequency
Binary data records a two-way condition. In text classification, a binary feature can record whether a word appears in a sentence: one value represents presence and the other represents absence. Count data preserves more detail. Instead of recording only whether a word appears, it records how often the word appears as an integer count. The same sentence can therefore produce two different feature representations.
One Sentence, Two Representations
Represent the sentence "cat dog cat" using the vocabulary [cat, dog]. First record presence or absence. Then record the number of appearances.
Check presence: The word cat appears, and the word dog appears. A presence-based representation records both features as present.
Count appearances: The word cat appears twice, while dog appears once. A count-based representation preserves those integer frequencies.
Using the vocabulary [cat, dog], the binary representation is [1, 1], while the count representation is [2, 1].
Selecting the Model
| Model | Assumed feature type | Text-data interpretation |
|---|---|---|
| BernoulliNB | Binary data | Whether a feature is present or absent |
| MultinomialNB | Count data | How many times a feature appears |
| GaussianNB | Continuous data | A continuous feature value |
Choose the variant according to what each feature value represents.
For text classification, BernoulliNB and MultinomialNB are the two variants most closely associated with the feature representations described here. Use BernoulliNB when the features express binary presence or absence. Use MultinomialNB when the features express integer counts, such as how often a word appears in a sentence. GaussianNB belongs with continuous data rather than these binary or count representations.
From Sentence to Classifier
A text-classification workflow begins with words in a sentence and ends with features that a model can use. First, decide what the features mean. If each feature answers whether a word appears, the resulting representation is binary and fits BernoulliNB. If each feature records an integer word count, the representation is count data and fits MultinomialNB. The model choice follows the representation; it is not determined merely by the fact that the original input was text.
Mistakes in Model Selection
Treating every numeric feature as continuous data
MultinomialNB assumes count data, while GaussianNB is the variant described for continuous data.
Fix:
Ask whether the number is an integer count of something before choosing the model.Using BernoulliNB when frequency information matters
A binary representation does not preserve the difference between one occurrence and multiple occurrences.
Fix:
Use a count representation and MultinomialNB when the features are intended to record how often words appear.Assuming text automatically requires one specific Naive Bayes variant
The source distinguishes BernoulliNB and MultinomialNB by the type of feature data, even though both are mostly used in text data classification.
Fix:
Select the variant after identifying whether the text features are binary or count-valued.
Check Your Choice
A text dataset represents each vocabulary word with the number of times it appears in a sentence. Which Naive Bayes variant matches this feature representation, and why?
Hints
- Identify whether the features record presence or integer frequency.
- Match the feature type to the model assumptions.
Practice Solution
A text dataset represents each vocabulary word with the number of times it appears in a sentence. Which Naive Bayes variant matches this feature representation?
Identify the data type: The features are integer counts because they record how often a word appears.
Match the assumption: MultinomialNB assumes count data.
MultinomialNB is the matching variant.
Key Takeaways
- BernoulliNB assumes binary features that represent two-way conditions such as presence or absence.
- MultinomialNB assumes count features, such as how often a word appears in a sentence.
- GaussianNB can be applied to continuous data.
- BernoulliNB and MultinomialNB are mostly used for text data classification.
- Choose the model from the meaning of the feature values, not merely from the fact that the input is text.
Key Takeaways
- Binary features record presence or absence.
- Count features record integer frequencies.
- Use BernoulliNB for binary data, MultinomialNB for count data, and GaussianNB for continuous data.
- In text classification, the representation chosen for the words determines which Naive Bayes variant fits.