What are support vectors?

Support vectors are the training examples that actively determine an SVM's decision boundary. In the trained SVM's dual representation, they are the examples with nonzero coefficients; examples with zero coefficients do not appear in its decision function.

Why are they called support vectors?

• In a hard-margin linear SVM, support vectors are the closest points from the two classes, lying on the margin boundaries.
• These points support the position and orientation of the maximum-margin separator.
• Points farther away usually do not determine the separator. However, changing or removing a support vector can change the fitted boundary.
• In a soft-margin SVM, support vectors can also lie inside the margin or be misclassified; they are not always exactly on the two margin lines.

Practical example: temperature alert

Imagine a simple classifier trained with one feature, sensor temperature in degrees Celsius:

• Normal: 15°C and 18°C (class 0).
• Overheated: 24°C and 28°C (class 1).

For this separable dataset, the maximum-margin decision boundary is 21°C, halfway between 18°C and 24°C:

• 18°C and 24°C are support vectors, the two nearest opposing training examples.
• 15°C and 28°C are not support vectors; they are farther from the boundary.
• Removing 15°C leaves the separator at 21°C in this example. Removing 18°C changes the closest opposing pair to 15°C and 24°C, moving the separator to 19.5°C.

That illustrates the key idea: not every training point contributes equally to the final SVM boundary.

What does the model calculate?

In the dual form of a binary SVM, its decision score can be written as:

f(x) = sum over support vectors i of
       alpha_i * y_i * K(x_i, x) + b

• x_i is a support vector and y_i is its class label, encoded as -1 or +1.
• alpha_i > 0 is its learned dual weight.
• K(x_i, x) measures similarity through the chosen kernel; with a linear kernel it is a dot product.
• Other training points have alpha_i = 0, so they contribute nothing directly to this decision score.

Find support vectors in Python

PYTHON
from sklearn.svm import SVC

X = [[15], [18], [24], [28]]
y = [0, 0, 1, 1]  # Normal=0, Overheated=1

clf = SVC(kernel='linear', C=10000).fit(X, y)

print(clf.support_)          # [1 2] (training-row indices)
print(clf.support_vectors_)  # [[18.] [24.]]
print(clf.n_support_)        # [1 1] (one per class)

Do not confuse SVM with KNN: KNN finds nearby examples separately for each prediction. An SVM learns its support-vector set during training and uses it for subsequent predictions.

In short: support vectors are the influential examples that anchor an SVM's separating boundary; in a soft-margin model they may be on or inside the margin.

References

• scikit-learn: Support Vector Machines
• scikit-learn: SVC attributes
• scikit-learn: Maximum-margin separating hyperplane example