Support Vector Machine: Maximum Margin
Try it: Support Vector Machine: Maximum Margin
How hard-SVM picks, among all separating lines, the one whose closest training point is farthest away, which points become support vectors, and how soft-SVM trades margin for violations with λ.
How it works
- Hard-SVM: minimise ‖w‖ subject to y(⟨w,x⟩ + b) ≥ 1 for every point; the margin is 1/‖w‖.
- In 2-D the optimum is fixed by one point of each class (their perpendicular bisector) or by two points of one class and one of the other.
- Check every such candidate against all constraints and keep the feasible one with the largest margin; the points on the margin lines are the support vectors.
- Compare with the Perceptron's separator, which also separates but usually with a smaller margin.
- Soft-SVM: minimise λ‖(w, b)‖² + average hinge loss by SGD (step 1/(2λt)), outputting the average iterate.
Default run (3 steps): Hard-SVM: among all (w, b) with y(⟨w,x⟩ + b) ≥ 1 for every point, find the smallest ‖w‖ — equivalently the separator whose closest point is farthest away (margin 1/‖w‖). … Max margin: w = (0.435, 0.696), b = -1.783, ‖w‖ = 0.82; margin 1/‖w‖ = 1.219 (band width 2.438). Support vectors p_5, p_6, p_7 sit exactly on y(⟨w,x⟩ + b) = 1. Checked 64 candidate sets. The Perceptron's line also separates, but its closest point is only 0.271 away.
Simplified: 2-D, at most 12 points, exact candidate enumeration instead of a general QP solver. The soft-SVM SGD regularises the bias too (x is extended with a constant 1) and runs a fixed number of seeded steps.
Loading the simulation…