Defining and Using Classes
During training, the SVM learns how important each of the training data points is to represent the decision boundary between the two classes. Typically only a subset of the training points matter for defining the decision boundary: the ones that lie on the border between the classes. These are called support vectors and give the support vector machine its name.
The Boundary Question
When an SVM is trained to separate two classes, it does not treat every training data point as equally important for representing the decision boundary. Instead, it learns how important each point is to that boundary. The points that lie on the border between the classes are typically the most important. These points are called support vectors, which gives the support vector machine its name.
Training Data to Boundary
The learning process can be understood as a change in focus. The SVM begins with training data from the two classes. During training, it learns the importance of the individual points for representing the boundary. The points that lie on the border between the classes become the support vectors, and these support vectors help define the decision boundary.
Support Vectors on the Border
Support vectors are the subset of training data points that matter for representing the decision boundary, typically because they lie on the border between the two classes.
The important relationship is not simply that a support vector belongs to one class. Its importance comes from its position at the border between the classes. Support vectors from the two classes are associated with the boundary that separates those classes.
Relevant and Non-Relevant Points
| Training-point group | Position described in the source | Role in representing the boundary |
|---|---|---|
| Support vectors | Typically on the border between the classes | Matter for defining or representing the decision boundary |
| Other training points | Not identified as the border points | Typically do not matter as much for representing the decision boundary |
A training set can contain many points, but typically only a subset matters for representing the decision boundary. Calling a point a support vector does not mean that it is a separate class. It means that the point is one of the important training points associated with the class border.
A Small Boundary Example
Identifying the support vectors
Imagine training data containing points from class 0 and class 1. A few points lie on the border between the two classes, while the remaining points are farther from that border. Which points would typically be the support vectors?
Step 1: Identify the two classes: Separate the training data conceptually into class 0 and class 1, the two classes that the SVM must distinguish.
Step 2: Locate the class border: Focus on the points that lie on the border between the two classes. Their position makes them important for representing the boundary.
Step 3: Name the important points: The border points are the support vectors. The other training points are not typically the points that define the decision boundary.
The support vectors are the subset of training points lying on the border between class 0 and class 1.
The example does not require counting every training point. The central test is whether a point belongs to the subset that matters for representing the boundary, typically because it lies on the border between the classes.
Common Identification Errors
Treating every training point as equally important to the decision boundary.
The SVM learns how important each training point is, and typically only a subset matters for representing the boundary.
Fix:
Look first for the points on the border between the classes; these are the points typically called support vectors.Defining a support vector as a third class.
Support vector describes the role of a training point in representing the boundary, not a new class.
Fix:
Keep the two classes separate from the support-vector role. A support vector is a point from the training data associated with the class border.Ignoring the point’s position relative to the class border.
The source identifies the border between the classes as the typical location of the points that matter for defining the decision boundary.
Fix:
Use the point’s relationship to the border as the main clue when identifying support vectors.
Check Your Understanding
An SVM is trained with data from two classes. Some points lie on the border between the classes, and many other points do not. Explain which points are typically called support vectors and why they matter.
Hints
- Start with the relationship between a point and the border between the classes.
- Then connect that relationship to the decision boundary.
What do you think happens?
If a training point lies on the border between class 0 and class 1, is it typically more likely to be important for representing the decision boundary than a point that is not on the border?
Reveal answer
Answer: Yes
The points that typically matter for defining the decision boundary are the ones that lie on the border between the classes. These points are called support vectors.
Key Takeaways
- An SVM learns from training data belonging to two classes.
- During training, it learns how important each training point is for representing the decision boundary.
- Typically, the points on the border between the classes are the important points.
- These points are called support vectors.
- Support vectors give the support vector machine its name because they help define the boundary between the two classes.
Key Takeaways
- An SVM separates two classes by learning from labeled training data.
- Only a subset of training points typically matters for representing the decision boundary.
- Points on the border between the classes are called support vectors.
- Support vectors are important because they help define the boundary separating the two classes.