What is selective classification, and how does a reject option work?

Selective classification (classification with a *reject option*) lets a model either return its predicted class or abstain when the example is too uncertain. Instead of forcing a prediction for every input, the system can route rejected cases to human review or a safer fallback. Its central design problem is the trade-off between coverage and risk on the accepted cases.

Beginner-friendly numerical example: support-ticket routing

A model predicts the appropriate ticket queue. It supplies a predicted label and the confidence of that top prediction. Suppose we test it on eight labeled tickets, ordered from highest to lowest confidence:

• Ticket 1: 0.98, prediction correct.
• Ticket 2: 0.95, correct.
• Ticket 3: 0.91, correct.
• Ticket 4: 0.87, incorrect.
• Ticket 5: 0.83, correct.
• Ticket 6: 0.76, correct.
• Ticket 7: 0.62, incorrect.
• Ticket 8: 0.54, incorrect.

Step 1 — Classify all eight. With no reject option, 5/8 are correct. Overall error rate = 3/8 = 37.5%; coverage is 100%.

Step 2 — Set acceptance threshold τ = 0.80. Accept a prediction only when top-label confidence ≥ 0.80.

• Accepted: Tickets 1–5; five predictions.
• Abstained: Tickets 6–8; three predictions. Importantly, one rejected prediction (Ticket 6) would have been correct: abstention is not a perfect error detector.

Step 3 — Calculate coverage.

• Coverage = accepted / all = 5/8 = 62.5%.
• Abstention rate = rejected / all = 3/8 = 37.5%.

Step 4 — Calculate selective risk.

• Among accepted tickets 1–5, only Ticket 4 is wrong.
• With 0–1 loss, selective risk = wrong accepted predictions / accepted predictions = 1/5 = 20%.
• Selective accuracy = 4/5 = 80%. This is not accuracy across all eight tickets.

Step 5 — Try a stricter threshold τ = 0.90.

• Accept only Tickets 1–3, all correct.
• Coverage falls to 3/8 = 37.5%, and observed selective risk becomes 0/3 = 0%.
• A zero error on three examples is *not* a reliable guarantee of zero future risk.

How is this different from a normal classification threshold?

• A class decision threshold chooses *which* label to output (for example, fraud versus not fraud).
• A reject threshold chooses *whether to trust and output* the classifier's proposed label at all. Low-scoring inputs receive abstain/review, not automatically the negative class.
• In multiclass classification, a common acceptance score is the maximum predicted class probability; other uncertainty scores or learned selection functions are possible.

How should selective classifiers be evaluated?

• Coverage: fraction of inputs automatically handled.
• Selective risk: average loss only on accepted inputs; for 0–1 loss, this is the accepted error fraction.
• Risk–coverage curve: vary the acceptance threshold and plot selective risk against coverage to reveal the operational trade-off.
• Rejected-case workload: count how many examples require human review and measure resulting delay or cost.
• Full-system quality: evaluate what happens to abstained cases; lower model-only selective risk is not enough if the fallback process fails.

Python illustration

PYTHON
import numpy as np

confidence = np.array([.98,.95,.91,.87,.83,.76,.62,.54])
correct = np.array([1,1,1,0,1,1,0,0], dtype=bool)

for threshold in [0.0, 0.80, 0.90]:
    accepted = confidence >= threshold
    coverage = accepted.mean()
    risk = np.mean(~correct[accepted]) if accepted.any() else float("nan")
    print(threshold, round(coverage, 3), round(risk, 3))
# 0.0  1.0    0.375
# 0.8  0.625  0.2
# 0.9  0.375  0.0

Applications and limitations

• Applications: customer-support routing, document classification, quality inspection, and high-stakes decision support with a qualified human fallback.
• Confidence can be misleading: an overconfident wrong prediction may still be accepted. Check calibration and the ability of confidence to distinguish correct from incorrect predictions.
• Selection bias: the accepted subset becomes easier as coverage falls. Always report risk *with* coverage; comparing risk alone is unfair.
• Threshold selection: use a representative validation set to select the operating point. Then report held-out performance and uncertainty; avoid tuning on the final test set.
• Review capacity: a stricter threshold may reduce automated errors but generate an unsustainable manual queue.
• Fairness and shift: acceptance rates and errors can differ by subgroup or change after deployment; monitor both.
• Zero acceptance: when no cases are accepted, empirical selective risk has a zero denominator and is undefined.

Key takeaway: Selective classification allows “I am not confident enough to decide.” It trades automatic coverage for potentially lower error among the predictions it does return.

References

• El-Yaniv & Wiener (2010), Foundations of Selective Classification
• Geifman & El-Yaniv (2017), Selective Classification for Deep Neural Networks