What is the softmax function?

Softmax is a function that converts a vector of raw scores into a probability distribution across multiple classes.

For a vector of scores z, softmax for class i is:

softmax(z_i) = exp(z_i) / Σ exp(z_j)

This guarantees that every output is between 0 and 1 and that all outputs sum to 1.

Example

Suppose a model produces three class scores:

[2.0, 1.0, 0.1]

After softmax, they become approximately:

[0.66, 0.24, 0.10]

These can be interpreted as the model's relative probabilities for the three mutually exclusive classes.

Where is softmax used?

Softmax is commonly used at the output of a multiclass classification model when exactly one class should be selected from several possible classes.

For example:

Cat   → 0.66
Dog   → 0.24
Bird  → 0.10

The class with the highest probability is typically chosen as the prediction.

Important distinction

Softmax compares classes against each other, so increasing the probability of one class affects the others.

For problems where multiple labels can be true at the same time, sigmoid is usually more appropriate. See the existing answer on “What is the sigmoid activation function?” for that case.

Numerical stability

In practice, implementations subtract the largest input score before exponentiation:

softmax(z_i) = exp(z_i - max(z)) / Σ exp(z_j - max(z))

This produces the same probabilities while reducing the risk of numerical overflow.

In short, softmax turns multiple class scores into a normalized probability distribution, making it especially useful for single-label multiclass classification.