What is an RBF kernel?

The Radial Basis Function (RBF) kernel, also called the Gaussian kernel, measures similarity based on the distance between two feature vectors. It lets an SVM learn curved, nonlinear decision boundaries when a straight line cannot separate the classes.

Formula

K(x, z) = exp(-gamma * ||x - z||^2)

• x and z are two feature vectors.
• ||x - z||^2 is their squared Euclidean distance.
• gamma (γ) is a positive hyperparameter that controls how quickly similarity falls with distance.
• exp denotes the exponential function.

Interpretation: identical points have kernel similarity 1. As two points move farther apart, their similarity approaches 0. This value is a kernel similarity, not a predicted class probability.

Worked numerical example

Take x = (1, 2) and set gamma = 0.2.

• Nearby point `a = (2, 2)`: squared distance = (1-2)^2 + (2-2)^2 = 1; similarity = exp(-0.2 * 1) ≈ 0.819.
• Farther point `b = (2, 4)`: squared distance = (1-2)^2 + (2-4)^2 = 5; similarity = exp(-0.2 * 5) ≈ 0.368.

The RBF kernel therefore gives the nearer point greater similarity without inventing additional manual features.

How does it help an SVM?

A linear SVM creates a straight separating boundary in the supplied feature space. An RBF SVM uses these distance-based kernel values in its learned decision function, effectively working with a much richer feature representation. This allows nonlinear boundaries in the original input space.

Example: Imagine a sensor where readings close to (0, 0) are Normal, while readings around that point are Alert. A single straight line cannot put the center on one side while keeping the surrounding points on the other; an RBF SVM can model such a pattern.

Python example

PYTHON
from sklearn.svm import SVC

# Class 0 at the center; class 1 surrounds it
X = [[0, 0], [1, 0], [-1, 0], [0, 1], [0, -1]]
y = [0, 1, 1, 1, 1]

model = SVC(kernel='rbf', gamma=2, C=100)
model.fit(X, y)
print(model.predict([[0, 0], [1, 0], [0, 1]]))
# [0 1 1]

What does gamma change?

• Smaller gamma: similarity decreases slowly with distance; each training point has a broader range of influence. The decision boundary often becomes smoother.
• Larger gamma: similarity decreases rapidly; points affect a more local neighborhood. The boundary can become intricate and may overfit.

Gamma is not the same as C. gamma controls the kernel's distance scale; C controls how strongly SVM penalizes margin violations. Both should be tuned using validation data.

When is RBF useful?

• When classes appear nonlinearly separable and a linear baseline is inadequate.
• When a moderate-size dataset supports kernel calculations.
• After scaling numerical features, since Euclidean distances are affected by feature units.

Limitations: RBF is not always better than linear models, and kernel SVM training can become expensive on very large datasets.

In short: RBF compares examples through distance-based Gaussian similarity, enabling SVMs to learn nonlinear boundaries. Its most important parameter is gamma, which determines how local that similarity is.

References

• scikit-learn: RBF kernel definition
• scikit-learn: RBF SVM parameters
• scikit-learn: SVC API