Halfspace Hypotheses
The same dot product ⟨w, x⟩ can be interpreted probabilistically by logistic regression or deterministically by a halfspace.
One Score, Two Interpretations
Both logistic regression and a halfspace hypothesis begin with the same quantity: the dot product ⟨w, x⟩. The difference is what happens next. A halfspace uses the sign of this value to choose a label directly, while logistic regression passes the value through the sigmoid function to produce a probability.
From Dot Product to Probability
Logistic regression treats ⟨w, x⟩ as an input to the sigmoid function. The sigmoid converts that input into a probability between the two extremes of 0 and 1. When the dot product is very large, the resulting probability is near one extreme. When the dot product is very small, meaning very negative, the resulting probability is near the opposite extreme. When the dot product is close to zero, the logistic output is near 0.5.
In this comparison, a logistic prediction is probabilistic because its output communicates how close the model is to one probability extreme or the other. A value near 0.5 indicates uncertainty rather than a strong preference for either extreme.
Reading Extreme and Central Scores
Three dot-product situations
Interpret the logistic output when ⟨w, x⟩ is very positive, close to zero, or very negative.
Very positive: A very large dot product is sent through the sigmoid and produces a probability near one extreme, near 1.
Close to zero: A dot product close to zero produces an output near 0.5. The model is least certain because the result is near the middle rather than near either probability extreme.
Very negative: A very small dot product, meaning a very negative one, is sent through the sigmoid and produces a probability near the opposite extreme, near 0.
The magnitude and sign of ⟨w, x⟩ determine where the logistic probability lies: near 1 for very positive values, near 0.5 for values close to zero, and near 0 for very negative values.
The important pattern is not a particular numerical calculation. It is the interpretation of location. Large magnitude means the logistic output is near an extreme, while magnitude close to zero means the output is near the middle.
Logistic Versus Halfspace Predictions
| Method | Use of ⟨w, x⟩ | Type of output | Treatment of uncertainty |
|---|---|---|---|
| Logistic regression | Passes the dot product through the sigmoid | A probability | Can indicate uncertainty with an output near 0.5 |
| Halfspace hypothesis | Uses the sign of the dot product | A label of 1 or -1 | Does not express uncertainty |
Suppose the shared dot product is close to zero. Logistic regression preserves that lack of a strong direction by producing a probability near 0.5. The halfspace does something different: it still uses the sign and returns either 1 or -1. Therefore, the halfspace gives a definite label even in the region where logistic regression communicates the greatest uncertainty.
The Decision Boundary and Uncertainty
The value |⟨w, x⟩| measures how far the dot product is from zero in magnitude. When this magnitude is close to zero, the dot product is near the central transition between negative and positive values. Logistic regression responds with an output near 0.5, so it is least certain. A halfspace does not show this gradation: it uses the sign and returns 1 or -1.
Common Interpretation Mistakes
Treating the dot product itself as the final logistic probability.
Logistic regression first passes the dot product through the sigmoid function.
Fix:
Separate the shared score ⟨w, x⟩ from the sigmoid output, which is the probability.Assuming that a halfspace reports uncertainty.
A halfspace always returns 1 or -1 and does not express uncertainty.
Fix:
Use the logistic output near 0.5 to identify uncertainty; treat the halfspace output as a deterministic label.Thinking that an output near 0.5 means high certainty.
The source identifies an output near 0.5 as uncertainty when the magnitude of the dot product is close to zero.
Fix:
Recognize that probabilities near the middle signal less certainty than probabilities near the extremes.
Check Your Interpretation
For each situation, state whether logistic regression should be near a probability extreme or near 0.5, and state whether a halfspace would express uncertainty: a very positive dot product, a very negative dot product, and a dot product close to zero.
Hints
- Very positive and very negative dot products lead to opposite probability extremes.
- A dot product close to zero leads to a logistic output near 0.5.
- The halfspace returns 1 or -1 in every case.
- A correct response should identify very positive values with a probability near one extreme, very negative values with the opposite extreme, and values close to zero with a probability near 0.5. Only the logistic prediction communicates the uncertainty near the center; the halfspace still returns 1 or -1.
Key Takeaways
- Logistic regression and a halfspace hypothesis start with the same dot product ⟨w, x⟩.
- Logistic regression sends the dot product through the sigmoid function to produce a probability.
- Very large and very small dot products produce probabilities near opposite extremes, while a dot product close to zero produces an output near 0.5.
- A halfspace uses the sign of the dot product to return 1 or -1 and does not express uncertainty.
- Logistic regression is least certain when |⟨w, x⟩| is close to zero because its output is near the middle value, 0.5.