How do you interpret a normalized confusion matrix?

A normalized confusion matrix converts raw classification counts into proportions, making error patterns easier to compare when classes have different numbers of examples. The meaning of each proportion depends on the denominator.

Three ways to normalize

• Row-wise (`normalize='true'`): divide each cell by its actual class row total. Rows sum to 1 when they have examples. This answers: “Of all actual Cats, what fraction did we predict as Dogs?” The diagonal is per-class recall.
• Column-wise (`normalize='pred'`): divide each cell by its predicted class column total. Columns with predictions sum to 1. This answers: “Of all predicted Dogs, what fraction are really Dogs?” The diagonal is per-class precision.
• Global (`normalize='all'`): divide every cell by the total number of samples. All cells together sum to 1 (provided there are samples). It shows each outcome's proportion of the entire dataset; diagonal values sum to accuracy.

Numerical example (no table)

Imagine 12 images classified as Cat, Dog, or Bird:

• Actual Cat (5): predicted Cat 4, Dog 1, Bird 0.
• Actual Dog (4): predicted Cat 1, Dog 2, Bird 1.
• Actual Bird (3): predicted Cat 0, Dog 1, Bird 2.

Focus on the cell actual Dog → predicted Bird, which contains 1:

• Row-wise: Dog row total = 4, so 1 / 4 = 25% of actual dogs were mistaken for birds.
• Column-wise: Bird prediction column total = 3 (0 Cats + 1 Dog + 2 Birds), so 1 / 3 ≈ 33.3% of *predicted birds* were actually dogs.
• Global: 12 total images, so 1 / 12 ≈ 8.3% of all images were dogs predicted as birds.

The raw count stays 1; only its denominator changes.

Python example

PYTHON
from sklearn.metrics import confusion_matrix

y_true = ['Cat']*5 + ['Dog']*4 + ['Bird']*3
y_pred = ['Cat']*4 + ['Dog'] + ['Cat', 'Dog', 'Dog', 'Bird'] + ['Dog', 'Bird', 'Bird']
labels = ['Cat', 'Dog', 'Bird']

for mode in ['true', 'pred', 'all']:
    cm = confusion_matrix(y_true, y_pred, labels=labels, normalize=mode)
    print(mode, cm.round(3))

Which should you use?

• Row-wise: when investigating missed actual classes or comparing recall across classes with unequal support.
• Column-wise: when investigating how trustworthy predictions of each class are.
• Global: when measuring how much each type of prediction or error contributes to overall results.

Limitations: A normalized cell hides its raw sample count: 100% from one example is far less reliable than 100% from 1,000. Always examine counts and class support. Rows or columns with zero examples/predictions cannot be interpreted as meaningful recall/precision.

Takeaway: Ask “out of what?” before interpreting any percentage in a normalized confusion matrix.

Reference: scikit-learn confusion_matrix documentation.