How does KNN classification work?

KNN classification predicts a new observation’s class from the labels of its K nearest training examples. The key operation happens at prediction time: measure distances, select neighbors, and combine their votes.

Step-by-step workflow

1. Prepare labeled data. Each training record has numerical feature values and a known class.
2. Choose K and a distance metric. For example, use K = 3 and Euclidean distance.
3. Measure distances from a new point to the stored training points.
4. Select the K nearest points and predict the class with the most votes.

For two-dimensional features, Euclidean distance is:

d = sqrt((x1 - a1)^2 + (x2 - a2)^2)

where (x1, x2) is the new point and (a1, a2) is a training point.

Worked example: classify a new customer

Assume two customer features are measured on comparable scales. A new customer has features (2, 2). The labeled training examples are:

• (2, 3) → Renew; distance = 1.000.
• (3, 2) → Renew; distance = 1.000.
• (3, 3) → Leave; distance = sqrt(2) ≈ 1.414.
• (0, 0) → Leave; distance = sqrt(8) ≈ 2.828.
• (5, 5) → Leave; distance = sqrt(18) ≈ 4.243.

With K = 3, the closest labels are Renew, Renew, Leave:

• Renew receives 2 votes.
• Leave receives 1 vote.
• Predicted class: Renew.

An unweighted neighbor-vote fraction for Renew is 2/3 ≈ 66.7%; this is not automatically a calibrated probability of renewal.

Try it in Python

PYTHON
from sklearn.neighbors import KNeighborsClassifier

X = [[2, 3], [3, 2], [3, 3], [0, 0], [5, 5]]
y = ['Renew', 'Renew', 'Leave', 'Leave', 'Leave']

model = KNeighborsClassifier(n_neighbors=3)
model.fit(X, y)
print(model.predict([[2, 2]]))  # ['Renew']

Practical details that matter

• Scaling: when features use different units, fit a scaler on training data and apply it consistently; otherwise one feature may dominate distance.
• Distance weighting: weights='distance' makes nearer examples count more than farther ones; it may change the winning class.
• Ties: class-vote ties or equal-distance neighbors near the Kth position need a well-defined tie-handling strategy.
• Prediction cost: KNN must search stored examples or a neighbor index when a query arrives; it does not simply evaluate a fixed linear formula.

In short: KNN makes a local prediction by computing distances and voting among neighbors. The choice of K, distance metric, weighting, and preprocessing controls the result.

References

• scikit-learn: Nearest Neighbors—Classification
• scikit-learn: KNeighborsClassifier