Weight Vectors and Bias Terms
A halfspace hypothesis assigns one of two labels to an input vector.
From Features to Two Labels
Suppose every instance in a feature space must receive one of two labels. A halfspace hypothesis provides the rule for making that binary decision. It separates the feature space into regions associated with the labels -1 and +1. The separation is controlled by a weight vector and a bias term.
The Separating Boundary
The weight vector and bias parameter together determine the halfspace hypothesis. In two dimensions, the boundary associated with that hypothesis is a hyperplane that appears as a line. This line separates the feature space into two regions. The weight vector is perpendicular to the line, so it provides the direction used to describe the relationship between the boundary and the feature space.
The word halfspace refers to the regions created by the separating hyperplane. The hyperplane itself is the boundary; the regions on its sides are the parts of feature space associated with the two possible classifications.
Parameter Roles
| Part of the hypothesis | Role described in the model |
|---|---|
| Weight vector | Helps determine the hypothesis and is perpendicular to the hyperplane in two dimensions. |
| Bias term | Works with the weight vector to determine the hypothesis. |
| Hyperplane | The separating boundary; in two dimensions, it is a line. |
| Regions on the sides | The areas associated with the positive and negative labels. |
Reading the Two Sides
Classifying Points Relative to a Line
Imagine a two-dimensional feature space divided by the line produced by a halfspace hypothesis. Determine the label for an instance described as above the line and for an instance described as below the line.
Locate the boundary: The hyperplane is the separating line in the two-dimensional feature space.
Locate the first instance: An instance described as above the hyperplane belongs to the region associated with the positive label.
Locate the second instance: An instance described as below the hyperplane belongs to the region associated with the negative label.
Interpret the geometry: The words above and below describe the geometric relationship with the hyperplane and the direction of the weight vector. They do not necessarily mean physical up and down in an application.
The first instance receives the positive label, and the second instance receives the negative label.
Common Interpretation Errors
Treating the hyperplane as one of the two regions.
The hyperplane is the boundary. The regions on its sides are the parts associated with the two classifications.
Fix:
Distinguish the boundary from the two regions it separates.Assuming above and below always mean physical up and down.
The terms describe the geometric relationship with the hyperplane and the direction of the weight vector.
Fix:
Use the model's geometry rather than a literal physical interpretation.Forgetting that the hypothesis has two possible labels.
A halfspace hypothesis assigns one of two labels to an input vector.
Fix:
Keep the binary outcome in view: the two labels are -1 and +1.Treating the weight vector or bias term as the entire hypothesis by itself.
The weight vector and bias parameter together determine the hypothesis.
Fix:
Describe both parameters as parts of the same halfspace hypothesis.
Check Your Understanding
A two-dimensional feature space contains a line that separates instances into two regions. Explain what the line represents, what determines the corresponding hypothesis, and how an instance on each side receives its label.
Hints
- Identify the geometric name for the separating line.
- Mention both the weight vector and the bias term.
- Use the positive and negative labels -1 and +1.
- A halfspace hypothesis assigns one of two labels to an input vector. Its weight vector and bias term jointly determine the hypothesis. In two dimensions, the associated hyperplane is a line perpendicular to the weight vector. The line divides feature space into two regions: instances on the positive side receive the positive label, while instances on the negative side receive the negative label.
Key Takeaways
- A halfspace hypothesis is a binary classification rule for an input vector.
- The weight vector and bias term together determine the hypothesis.
- In two-dimensional feature space, the hyperplane is a line perpendicular to the weight vector.
- The hyperplane separates feature space into regions associated with positive and negative labels.
- Above and below describe geometric relationships with the boundary and the weight-vector direction, not necessarily physical directions.