Skip to content
All apps

Support Vector Machine (SVM)

About this app

Maximum margin and support vectors, solved exactly with SMO: points without a ring do not move the boundary, soft margin C, slack ξ, linear, lift (paraboloid in 3D) and RBF kernels.

The idea of an SVM

A support vector machine separates two classes with the boundary that has the largest margin: the widest empty street between them. Formally it minimises ½‖w‖² + C·Σξᵢ subject to yᵢ(w·xᵢ + b) ≥ 1 − ξᵢ; the margin width is 2/‖w‖. Points on the edge of the street or inside it are support vectors with αᵢ > 0, and only they determine w = Σ αᵢyᵢxᵢ. The parameter C sets how costly slack ξ is. The kernel trick replaces dot products with kernel values such as k(x,x′) = exp(−γ‖x−x′‖²), giving curved boundaries.

What you see

A feature space with the boundary, dashed margin lines (f = ±1) and rings around support vectors; slack ξ can be shown. A 3D view shows the decision function f(x) as height or, with the “lift” kernel, the mapping φ(x) = (x₁, x₂, ‖x‖²) onto a paraboloid, where a plane separates the circle from the ring. Readouts give margin width, ‖w‖, number of support vectors, Σξ, training error and the number of SMO steps.

Things to try

Drag a point without a ring: the boundary stays put. Drag a support vector: the boundary follows. Lower C on “overlapping” and watch the street become wider and more tolerant. On “circle in ring”, switch from the linear kernel to “lift ‖x‖²” and then to RBF. With the RBF kernel, notice how a large γ wraps the boundary tightly around individual points.

Frequently asked questions

What are support vectors in an SVM?
Support vectors are the training points that lie on the edge of the margin or violate it. Only they have a dual weight α > 0 and therefore determine where the boundary lies. Any other point can be moved or removed without changing the solution, as long as it stays outside the margin.
What does the C parameter do in a soft-margin SVM?
C weights the penalty for points that intrude into the margin or sit on the wrong side. A small C allows many violations and gives a wide, tolerant street; a large C penalises them heavily and gives a narrow, strict street that is more prone to overfitting.
What is the kernel trick?
The kernel trick uses the fact that an SVM only needs the data through dot products. Replacing them with a kernel function k(x,x′) lets the SVM work implicitly in a higher-dimensional feature space without computing the mapping. This produces non-linear boundaries in the original space.
What can I do in this app?
You place and drag points, choose the “separable”, “overlapping” or “circle in ring” dataset, set C and γ, and switch between the linear kernel, the lift and RBF. The app solves the dual problem with SMO and shows support vectors, slack and the decision function in 3D.

Subject: Machine Learning

More from Machine Learning

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps