k-Nearest Neighbors (kNN)
About this app
A query point, its k neighbours with distance shape and majority vote: decision regions from k = 1 (jagged) to k = 40 (smooth), train and test accuracy over k, metric and feature scaling change the neighbourhood.
How kNN classifies
k-nearest neighbours learns no formula in advance. For a new point it finds the k closest training points and lets them vote; the most common class wins. With k = 1 the space splits into cells around each training point, much like a Voronoi diagram: every outlier gets its own island and the model overfits. A very large k smooths everything until the more frequent class wins almost everywhere. Because kNN relies only on distances, the result depends strongly on the distance function and on how features are scaled.
What you see
A feature space with two classes, shaded by the vote at every location: the stronger the colour, the clearer the decision. Around the query point a dashed shape encloses exactly the k neighbours, a circle for Euclidean distance and a diamond for Manhattan. A panel lists the neighbours with distance and vote, and a chart shows training and test accuracy across k and marks the best k. Without test data the chart uses leave-one-out.
Things to try
Drag the query point across a class boundary and watch the vote flip. Click k = 1 and a large k in the accuracy chart and compare the regions. Switch between Euclidean and Manhattan distance. Then stretch feature x₁: the neighbourhood becomes an ellipse and different points vote. Notice how “standardise (×1)” undoes the effect.
Frequently asked questions
- How does the k-nearest neighbors algorithm work?
- kNN stores the training data and classifies a new point by the majority of its k nearest neighbours. There is no real training phase; the cost comes at prediction time, when distances to all stored points are computed. For regression, the neighbours’ mean replaces the majority vote.
- How do you choose k in kNN?
- A small k follows every outlier and tends to overfit, while a large k smooths too much and blurs real boundaries. A common approach is to pick k with validation data or cross-validation and plot accuracy against k. With two classes, an odd k avoids ties.
- Why do features need to be scaled for kNN?
- kNN decides by distance. A feature with a large numeric range, for example one measured in a small unit, dominates the distance and drowns out the others. Standardising to mean 0 and variance 1 gives all features comparable weight.
- What can I do in this app?
- You place a query point, choose k, the distance (Euclidean or Manhattan) and the scaling of x₁, and add or move training points. The app shows the neighbours with their distance shape, the majority vote, the decision regions, and training and test accuracy across k.
Subject: Machine Learning
More from Machine Learning
- AdaBoost
- Anatomy of a prompt
- Data privacy traffic lights
- Decision Tree
- Ensemble: Bagging vs. Boosting
- Finding faces: masks versus learning from examples
- k-Means
- AI theatre
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps