Machine Learning

Machine learning is a way to build software by fitting models to examples instead of writing every rule by hand. A model sees data, optimizes an objective, and generalizes the learned pattern to new inputs.

What is mutual information for estimating epistemic uncertainty in classification?

Mutual information (MI) in Bayesian predictive classification estimates how much a predicted label tells us about uncertain model parameters . A common estimator measures the disagreement among predictions from a Bayesian model's parameter samples, a deep ensemble, or Monte Carlo dropout. The acquisition score is often called BALD ( Bayesian Active Learning by Disagreement ). Formula For M sampled predictive probability vectors p⁽¹⁾, …, p⁽ᴹ⁾ for one example: Mean predictive distribution: p̄ = (1/M) Σₘ p⁽ᵐ⁾. Predictive entropy: H(p̄), measuring total uncertainty in the mixture. Mean individual entropy: (1/M) Σₘ H(p⁽ᵐ⁾), measuring average uncertainty within sampled models. Mutual information: MI ≈ H(p̄) − (1/M) Σₘ H(p⁽ᵐ⁾) . Here H(p) = −Σ꜀ p꜀ log₂ p꜀, in bits . Always use a consistent log base. Worked example: two models, two classes Suppose two plausible classifiers evaluate the same picture as Cat versus Dog : Model A: [Cat 0.90, Dog 0.10]. Model B: [Cat 0.10, Dog 0.90]. Step 1 — Average their probabilities: p̄ = [(0.90+0.10)/2, (0.10+0.90)/2] = [0.50, 0.50] . Step 2 — Entropy of the averaged distribution: H(p̄) = −0.5 log₂ 0.5 − 0.5 log₂ 0.5 = 1.000 bit . Step 3 — Entropy within each model: H(Model A) = −0.9 log₂ 0.9 − 0.1 log₂ 0.1 ≈ 0.469 bits . Model B has the same entropy, 0.469 bits . Mean individual entropy = (0.469+0.469)/2 = 0.469 bits . Step 4 — Subtract: MI = 1.000 − 0.469 ≈ 0.531 bits. Interpretation: each model is relatively confident, but they disagree about the class . That disagreement contributes substantial model related uncertainty. Contrast: uncertain models that agree Suppose both models predict [0.50, 0.50] : Entropy of their average = 1 bit . Mean individual entropy = 1 bit . MI = 0 bits even though predictive entropy remains high. This distinguishes uncertainty within every model from uncertainty arising because models disagree . It is an illustrative contrast, not an automatic identification of the exact source of real world data noise. How is MI estimated in practice? Deep ensemble: train several models with different initialization or training randomness; collect every model's class probability vector for the same input. Monte Carlo dropout: with an appropriately trained dropout network, enable dropout for stochastic inference passes (without updating weights), collect many probability vectors, and use the formula above. Bayesian parameter samples: sample model parameters from an approximate or exact posterior where feasible. Average probabilities , not class IDs or raw logits. Calculate entropy of the mean , then subtract mean entropy —those operations cannot be swapped. Python example Applications and limitations Active learning: prioritize examples where plausible models disagree, because obtaining their labels may be informative for learning. Review and monitoring: flag disputed predictions for human inspection or additional training data, then validate whether MI actually predicts errors. Approximation quality: ensembles and dropout samples are proxies for posterior uncertainty, not necessarily calibrated Bayesian posterior samples. MI may miss uncertainty when models share biases. Finite samples: few stochastic predictions lead to noisy estimates. Test robustness to model count and random seeds. Entropy is not automatically aleatoric uncertainty: the mean per model entropy is sometimes used as an approximation to data uncertainty, but it may also reflect model misspecification or approximation issues. Comparability and safety: MI depends on class count and probability quality. Report class count and log base, evaluate on shifted data and relevant subgroups, and do not treat high MI as a guarantee that a new label will improve a model. Key takeaway: Predictive entropy measures uncertainty of the averaged prediction ; mutual information subtracts the uncertainty within individual model predictions to highlight model disagreement. References Gal, Islam & Ghahramani (2017), Deep Bayesian Active Learning with Image Data Gal & Ghahramani (2016), Dropout as a Bayesian Approximation

What is variation ratio in classification uncertainty estimation?

Variation ratio (VR) is a simple measure of disagreement among categorical predictions collected for the same input from repeated stochastic model passes or multiple models. It asks what fraction of the predicted labels do not match the most frequently predicted label (the mode). Formula and interpretation Let T be the number of predicted label samples for one input. Find the modal class : the label receiving the greatest number of votes. Let f mode be how many votes that class receives. VR = 1 − f mode/T. VR = 0 : all predictions vote for the same class. Larger VR: votes are more dispersed among labels. For C classes, the theoretical maximum is 1 − 1/C , achieved by equal class frequencies when feasible. For two classes, the maximum is 0.5 . For equal count ties, the modal class may not be unique, but f mode (the maximum vote count) still is. Step by step example: eight stochastic predictions A classifier examines the same animal photo eight times using an ensemble or Monte Carlo dropout. Its predicted labels are: Passes 1–8: Cat, Dog, Cat, Bird, Cat, Dog, Cat, Dog . Step 1 — Count votes: Cat = 4 , Dog = 3 , Bird = 1 . Step 2 — Find the mode: Cat received the most votes; f mode = 4 . Step 3 — Calculate: VR = 1 − 4/8 = 0.50 . Step 4 — Interpret: 50% of votes disagree with Cat , indicating substantial variation among the eight categorical predictions. This is not a claim that the probability of being wrong is 50%. A useful contrast If all eight passes vote Cat, VR = 1 − 8/8 = 0 , regardless of how close their underlying softmax probabilities are to a tie. If three classes receive votes 3, 3, 2 , then VR = 1 − 3/8 = 0.625 . The most common class need not have a majority. Variation ratio is based on the chosen categorical label samples , not on the full predicted probability vectors. Python example How do you obtain the votes? Deep ensembles: collect a predicted class from each independently trained model. Monte Carlo dropout: run multiple stochastic forward passes on the same input with dropout enabled but no training updates ; record each resulting categorical prediction. Important implementation choice: traditional Bayesian VR definitions sample a class label from each predicted softmax distribution . Other implementations take the argmax label of each pass . These two procedures can give different VR values, so explicitly state which was used. Variation ratio versus mutual information (Question 221) VR counts class votes: it ignores how confident a model was when casting each vote. It is easy to compute but coarse. Mutual information compares probability distributions: it subtracts mean per model entropy from entropy of their average to quantify model related disagreement. For example, models that all predict Cat with probabilities [0.51, 0.49] yield argmax based VR = 0 , even though their probabilities are nearly tied. MI also depends on variation across models , not merely individual ambiguity, so the metrics capture different properties. VR is a vote dispersion indicator , not a complete measure or guaranteed isolation of epistemic uncertainty. Applications and limitations Active learning: identify cases with unstable class labels for annotation, then test whether this improves learning. Human review: prioritize disputed predictions in document classification, image inspection, or quality control. Error/OOD detection: validate VR as a score against actual mistakes and unfamiliar inputs; consider AUROC and selective risk at matched coverage. Sample count sensitivity: with only a few passes, votes and VR are noisy. Study robustness to ensemble size or MC samples. Class count dependence: VR ranges differ with the number of classes; report C, T, and the vote generation rule. Shared blind spots: models can agree confidently on the wrong class, giving VR = 0. Lost probability detail: VR cannot distinguish a narrow win from an overwhelming win when both produce the same modal count. Key takeaway: Variation ratio measures how often stochastic predicted labels disagree with the modal label , rather than how spread out their full probability distributions are. References Yarin Gal, Uncertainty in Deep Learning, section on variation ratios Gal, Islam & Ghahramani (2017), Deep Bayesian Active Learning with Image Data

How do deep ensembles estimate uncertainty in classification?

Deep ensembles estimate predictive uncertainty by training several neural networks independently and combining their predicted class probabilities. The models may learn different plausible solutions, so their agreement—or disagreement—provides useful information beyond any one model's confidence. How a deep ensemble is built Train multiple models: use the same classification task and training dataset, but train each network separately, typically with different random initializations and shuffled minibatches. Architecture may be shared; bootstrap samples or alternative architectures are optional diversity mechanisms. Retain separate models: do not average network weights; each trained network independently produces a class probability vector. Combine at prediction time: for M members and class c, ensemble probability p̄(c|x) = (1/M) × Σₘ pₘ(c|x) . Equal weights are a common baseline; learned weighting requires careful validation. Inspect uncertainty: use how concentrated the ensemble's mean probabilities are to describe predictive ambiguity, and how much member predictions differ to investigate model related uncertainty. Beginner friendly numerical example: three image classifiers Three separately trained models classify one image into Cat, Dog, Bird , returning: Model A: [0.80, 0.15, 0.05] — favors Cat. Model B: [0.60, 0.30, 0.10] — favors Cat. Model C: [0.40, 0.50, 0.10] — favors Dog. Step 1 — Average the Cat probabilities: (0.80 + 0.60 + 0.40) / 3 = 0.6000 . Step 2 — Average Dog probabilities: (0.15 + 0.30 + 0.50) / 3 = 0.3167 . Step 3 — Average Bird probabilities: (0.05 + 0.10 + 0.10) / 3 = 0.0833 . Step 4 — Check the distribution: 0.6000 + 0.3167 + 0.0833 ≈ 1.0000 . The ensemble predicts Cat , with a mean estimated probability of 60% . Step 5 — Read the disagreement: Model C prefers Dog, unlike Models A and B. This is a practical sign that the classification may depend on which trained model is used. The averaged probability reflects this uncertainty better than blindly returning Model A's 80% Cat prediction—but averaging does not automatically guarantee a calibrated 60% risk estimate. Contrast: If all three models instead output [0.80, 0.15, 0.05] , the ensemble's mean is still [0.80, 0.15, 0.05], and there is no observed inter model disagreement . That does not prove the image is easy or that every model is correct. How does ensemble diversity help? Different random seeds, minibatch orders, and optional data resampling can lead models to different weight configurations and decision boundaries. When plausible models disagree, this may indicate epistemic uncertainty , related to limited training evidence or alternative model explanations. Diversity is a proxy , not a mathematically exact posterior uncertainty estimate. A set of nearly identical models provides little extra information. Conversely, arbitrary disagreement caused by poorly trained members is not automatically useful uncertainty. Compare probability vectors , not logits or predicted class IDs, when producing the ensemble mean. This preserves probability interpretation. Python example Evaluation: is the uncertainty useful? Accuracy and proper probabilistic scores: evaluate on an untouched test set using accuracy, negative log likelihood, and Brier score. Calibration: inspect reliability diagrams and an explicitly defined ECE variant; do not assume averaging creates calibrated probabilities. Error detection: test whether high disagreement or low ensemble confidence identifies actual mistakes, using AUROC or risk at matched coverage. Distribution shift and out of distribution examples: evaluate how uncertainty behaves on shifted, corrupted, or unfamiliar inputs. A model ensemble can still agree confidently on an unseen or systematically biased input. Compare baselines: evaluate a single model and an ensemble of various sizes on identical held out data, including multiple training seeds if feasible. Computational trade offs and limitations Training cost: M independently trained members can require roughly M times the model training work, although separate training jobs are easy to parallelize. Inference cost: all M members may need forward passes, increasing compute, latency, and memory unless parallelized or compressed. Storage/deployment: M sets of model weights must be maintained, and updates must preserve compatibility across preprocessing and class labels. Diminishing returns: adding more similar members can offer little gain. Check validation results before increasing ensemble size. Correlated errors: all models may share the same dataset bias, architecture weakness, or distribution shift; disagreement alone cannot certify reliability. Applications: medical image triage, quality inspection, image classification, and review queues where both prediction quality and awareness of uncertainty matter. Key takeaway: Deep ensembles use independently trained models, average their class probabilities, and inspect member disagreement . This is a practical route to predictive uncertainty, but it is not a guarantee of Bayesian correctness or protection from distribution shift. References Lakshminarayanan, Pritzel & Blundell (2017), Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Google Research, Can You Trust Your Model's Uncertainty?

How does Monte Carlo dropout estimate predictive uncertainty in classification?

Monte Carlo (MC) dropout estimates predictive uncertainty by running one trained neural network multiple times on the same input while dropout remains stochastic at inference . Each pass uses a different random dropout mask and can produce a different probability distribution. Averaging those distributions gives a predictive estimate; variation across passes provides a practical proxy for sensitivity to the network's approximate model uncertainty. How MC dropout works Train normally: build a classifier with dropout layers and train its weights using supervised examples. Dropout randomly masks some activations during training. Freeze the weights for inference: no new labels, optimizer steps, or gradient updates are needed. Activate only dropout randomness: make dropout layers stochastic during repeated evaluation passes while keeping other inference sensitive components, especially BatchNorm, in evaluation mode. Repeat T times: generate T softmax probability vectors for the same image or input. Aggregate: average probabilities per class . Inspect variability across passes; stronger variation can indicate that the output is sensitive to the sampled dropout mask. Beginner friendly numerical example: four passes on one image A network predicts Cat, Dog, Bird using four independently sampled dropout masks: Pass 1: [0.80, 0.15, 0.05]. Pass 2: [0.60, 0.30, 0.10]. Pass 3: [0.40, 0.50, 0.10]. Pass 4: [0.70, 0.20, 0.10]. Step 1 — Average each class's predicted probability Cat: (0.80 + 0.60 + 0.40 + 0.70) / 4 = 0.6250 . Dog: (0.15 + 0.30 + 0.50 + 0.20) / 4 = 0.2875 . Bird: (0.05 + 0.10 + 0.10 + 0.10) / 4 = 0.0875 . Check: 0.6250 + 0.2875 + 0.0875 = 1.0000 . The averaged predicted class is Cat , with estimated probability 62.5% . Step 2 — Measure how much a single class changes Cat probabilities across passes: [0.80, 0.60, 0.40, 0.70] . Deviations from 0.625: [+0.175, −0.025, −0.225, +0.075] . Mean squared deviation (using denominator T = 4) = (0.030625 + 0.000625 + 0.050625 + 0.005625) / 4 = 0.021875 . Standard deviation = √0.021875 ≈ 0.148 , or 14.8 percentage points . This spread is an illustrative per class variation statistic , not a calibrated probability of model error or a complete separation of uncertainty sources. Step 3 — Interpret disagreement Pass 3 favors Dog, whereas the other three favor Cat. The final Cat prediction is stable enough to win on average, but not across every stochastic pass. If all dropout passes agreed perfectly, the observed variability would be zero; that would not prove that the prediction is correct or that the model has no uncertainty. A practical PyTorch inference pattern For models using functional dropout (F.dropout) or custom stochastic layers, explicitly check that their dropout behavior is controlled as intended; the example toggles ordinary module based dropout only. Compared with deep ensembles (Question 223) MC dropout: one trained set of weights, many stochastic forward passes generated through dropout masks. Relatively low storage cost; repeated passes increase inference cost. Deep ensembles: several independently trained weight sets , each yielding predictions. Training and model storage are more expensive, but members can explore substantially different solutions. Neither is guaranteed to give an exact Bayesian posterior. MC dropout can underrepresent uncertainty when its dropout approximation is poor, while ensembles can share errors and biases. Averaging predictions across stochastic passes is the common operation, but the source of variation differs. Practical setup, evaluation, and limitations Choose pass count with validation: too few samples produce noisy estimates. Compare results across 10, 30, 50 or more passes and inspect stability versus latency. Evaluate actual classification quality: compare accuracy, negative log likelihood, and Brier score with a deterministic baseline. Check uncertainty usefulness: test whether uncertainty rankings identify mistakes or shifted/out of distribution inputs, using error detection AUROC or selective risk at matched coverage. Check calibration: reliability diagrams and proper scoring rules reveal whether averaged probabilities align with observed frequencies; dropout itself does not guarantee calibration. Protect inference semantics: calling model.train() for the whole network can update BatchNorm running statistics even under no grad(); enable dropout selectively instead. Cost: T passes cost roughly T forward computations unless batching or hardware parallelism mitigates latency; results also depend on dropout placement and rates. Limitations: high confidence on an unfamiliar input, correlated sampled predictions, and training data bias can all hide risk. A low standard deviation is not a safety guarantee. Applications: image classification, quality inspection, document triage, and human review systems where confidence driven automation needs scrutiny. Key takeaway: MC dropout reuses one frozen trained model with fresh dropout masks at inference . Average its probability vectors for prediction and examine cross pass variability as an approximate uncertainty signal. References Gal & Ghahramani (2016), Dropout as a Bayesian Approximation PyTorch Dropout documentation PyTorch BatchNorm2d documentation

How can uncertainty estimates detect out-of-distribution (OOD) inputs in classification?

Out of distribution (OOD) detection asks whether an input differs meaningfully from the data a classifier was designed and trained to handle. One practical baseline uses prediction uncertainty : unfamiliar inputs may receive lower maximum class probability (or a higher uncertainty score), so the system can flag them for review instead of treating their predicted class as trustworthy. OOD is not the same as an ordinary classification mistake In distribution (ID) error: A cat photo drawn from the model's familiar image domain is mistakenly labeled Dog. The input is still from the expected domain. OOD input: A veterinary animal classifier receives a satellite photo, a medical scan, or an unfamiliar category or visual style. It may still output Cat or Dog, even though its label set does not meaningfully cover the image. Important: Some shifted images are hard but still in scope; the exact ID/OOD definition must be specified for each deployment. Incorrect ID predictions and correctly rejected OOD inputs are different evaluation questions. Simple uncertainty based OOD score For a model that predicts probabilities p(Cat|x), p(Dog|x), and so on, take maximum softmax probability (MSP) = max class probability . Define an OOD suspicion score = 1 − MSP . Larger means more suspicious. Choose a suspicion threshold using representative validation data, and flag scores at or above it. It is a heuristic baseline, not a guarantee that every OOD input has low confidence. Beginner friendly numerical example: eight images Consider four familiar ID animal images and four OOD images: ID images: MSP scores 0.95, 0.90, 0.75, 0.65 . OOD images: MSP scores 0.80, 0.55, 0.35, 0.20 . Step 1 — Convert to uncertainty scores ID uncertainty = 0.05, 0.10, 0.25, 0.35 . OOD uncertainty = 0.20, 0.45, 0.65, 0.80 . Step 2 — Set OOD flagging threshold = 0.30 Flag when 1 − MSP ≥ 0.30 , equivalently when MSP ≤ 0.70 . Detected OOD: scores 0.45, 0.65, 0.80 → 3 of 4 , so OOD recall/TPR = 75% . Missed OOD: the OOD image with uncertainty 0.20 (MSP 0.80) looks deceptively confident → 1 of 4 . False OOD alarms: the familiar image with uncertainty 0.35 is flagged → 1 of 4 ID , so ID false positive rate = 25% . Correctly kept ID: three familiar images remain unflagged → 3 of 4 . Step 3 — Calculate a simple AUROC for this example Treat OOD as the positive class and larger uncertainty scores as more OOD like. Compare every OOD score to every ID score: 4 × 4 = 16 pairs . The OOD score 0.20 outranks 2 ID scores; scores 0.45, 0.65, 0.80 each outrank all 4 ID scores: 2 + 4 + 4 + 4 = 14 correct pairs . AUROC = 14/16 = 0.875 (no ties here). AUROC uses all possible thresholds ; the 75% TPR and 25% FPR above describe only the chosen 0.30 threshold. Evaluation metrics: what they mean AUROC (↑): probability that a randomly sampled OOD case receives a more OOD like score than a randomly sampled ID case (half credit for ties). Random ranking is approximately 0.5; perfect separation is 1.0. AUPR (↑): area under a precision–recall curve, often summarized using average precision . Report whether OOD or ID is considered positive, because the value changes with this convention and with the positive class prevalence. FPR at 95% TPR (↓): pick an operating threshold that detects 95% of OOD inputs (under the OOD positive convention) and report what fraction of ID inputs it falsely flags. Some papers instead define ID as positive , meaning that at 95% ID retention this metric reports the fraction of OOD examples wrongly accepted as ID. Always specify the positive class and score direction. ID classification accuracy and accepted workload: also measure how accurately the original classifier handles familiar inputs and how often usable inputs get rejected. Python: evaluate the illustrative OOD score Practical evaluation design Define your deployment scope: specify valid input types, conditions, and unknown categories. Use ID validation data representative of actual traffic. Build challenging OOD test sets: include near OOD examples (visually related unknown classes or small domain shifts) and far OOD examples (unrelated input types). Easy far OOD cases alone overstate readiness. Avoid test leakage: select the score/threshold on a distinct validation set; reserve separate ID and OOD test sets for final comparisons. Use multiple metrics: AUROC, AUPR, FPR@95TPR, ID accuracy, and operational review workload address different failure modes. Report sample sizes, uncertainty intervals, score convention, and dataset composition. Evaluate changed conditions: corruptions, new capture devices, and shifted populations can change OOD detection quality after deployment. Applications and limitations Applications: medical image triage, industrial defect detection, animal or plant classifiers, and support ticket routers with an unknown content fallback. Confidently wrong OOD inputs: softmax can assign high confidence to unfamiliar examples ; the OOD image with MSP 0.80 above demonstrates why uncertainty threshold detection is imperfect. Inaccurate uncertainty proxies: entropy, ensemble disagreement, or MC dropout variation may provide different detection rankings; none universally resolves OOD detection. Unknown is not always a new label: some changes affect image style or sensor noise but preserve the class meaning; decide whether your policy should flag, adapt, or accept them. Operational cost: flagging too many valid inputs causes unnecessary manual work; failing to flag harmful OOD inputs can produce unsafe automated decisions. Key takeaway: Use uncertainty as a screening signal for unfamiliar inputs , not proof that a confident prediction is in distribution. Evaluate ID/OOD separation and the downstream cost of false alarms and missed OOD cases. References Hendrycks & Gimpel (ICLR 2017), OOD detection baseline Selective Classification Under Distribution Shifts — evaluation discussion

What is the difference between out-of-distribution detection and open-set recognition?

Out of distribution (OOD) detection identifies inputs that do not fit a defined reference distribution. Open set recognition (OSR) extends classification to a world where some semantic classes were absent during training : it must recognize familiar classes and assign Unknown to unfamiliar classes when appropriate. The two tasks overlap, but the distinction matters for how datasets and products are designed. Closed set, OOD detection, and open set recognition Closed set classification: assume every test example belongs to one of the trained categories. An animal model trained on Cat, Dog, Bird must choose one of these labels, even for a Horse photo. OOD detection: ask whether an input appears outside the intended training/reference distribution. Examples can include a medical scan in an animal photo pipeline, or a night vision image from a capture domain absent in training. This is often framed as ID versus OOD , without specifying an unknown semantic label. Open set recognition: handle valid inputs from novel semantic categories , for example Horse in a photo dataset where only Cat, Dog, and Bird were labeled during training. The model should output Unknown , not misidentify Horse as Dog. Important overlap: an unknown class is often an OOD input relative to trained categories. But not every domain/covariate shift implies a new class: a blurry cat is still a cat. Research benchmarks sometimes use OOD and OSR terminology differently, so define the evaluation protocol explicitly. Beginner friendly numerical example: an animal photo app A classifier was trained on Cat, Dog, Bird . It receives ten images: six from known classes and four horses (unknown class). The system uses an illustrative known class acceptance score and accepts a known label when score is at least 0.70 ; otherwise it returns Unknown . Known class images: K1: score 0.95 , correct predicted class → accepted correctly . K2: score 0.91 , correct → accepted correctly . K3: score 0.88 , incorrect known label → accepted incorrectly . K4: score 0.77 , correct → accepted correctly . K5: score 0.72 , correct → accepted correctly . K6: score 0.55 , correct candidate label but low score → rejected as Unknown . Unknown class images (all horses): U1: score 0.82 → incorrectly accepted as a familiar class . U2: score 0.61 → correctly rejected as Unknown . U3: score 0.40 → correctly rejected as Unknown . U4: score 0.20 → correctly rejected as Unknown . Step 1 — Evaluate known class recognition Five of six known examples are accepted. Four of six are both accepted and correctly classified: known class correct classification rate (CCR) = 4/6 = 66.7% . Among accepted known class examples, accuracy is 4/5 = 80% ; reporting only this number would hide the rejected known case. Step 2 — Evaluate unknown rejection Three of four horses are rejected: unknown rejection recall = 3/4 = 75% . One of four horses is incorrectly accepted as known: false known acceptance rate = 1/4 = 25% . Step 3 — Interpret the trade off Raising the acceptance threshold may reject more horses but also discard useful Cat/Dog/Bird inputs. Lowering it may classify more known examples while wrongly assigning known labels to horses. In production, Unknown means “not safely assigned to a trained category,” not “the model has discovered that the class is Horse.” Naming and learning a new category requires additional labeled data or an open world learning workflow. Appropriate evaluation Known class accuracy and recall: assess how well the model recognizes familiar classes, including the penalty for wrongly rejecting known inputs. Unknown detection AUROC/AUPR: assess the separation between known and unknown examples with a clearly specified positive class and score orientation. These measures alone do not prove correct recognition of the known class. OSCR (Open Set Classification Rate) curve: plot the correct classification rate on known examples against false known acceptance rate on unknown examples while varying the acceptance threshold. In this example, the point is CCR = 66.7%, unknown false known acceptance = 25% . Open set confusion analysis: count correctly classified knowns, incorrectly classified knowns, mistakenly rejected knowns, and unknowns falsely accepted as known. Operational metrics: review queue volume, response time, false rejection cost, and harm from calling an unknown category a known one. Report the number and difficulty of unknown categories, class balance, test set composition, and threshold selection procedure. Practical applications Wildlife cameras: classify known species while tagging unseen species for expert review. Visual quality inspection: identify known defect classes while flagging unmodeled defect types. Document intake: route known document types and send genuinely new types to an Unknown queue. Face or identity systems: avoid falsely claiming an unseen person belongs to one of the enrolled identities; use appropriate privacy and authorization safeguards. Implementation and production trade offs Start with a closed set classifier plus reject rule as a baseline, then evaluate dedicated open set methods such as distance to prototype scoring, feature density techniques, or specialized open set training objectives. Use separate validation and test splits for known classes and held out unknown classes. Do not tune hyperparameters on the exact unknown test categories you later claim are unseen. Test both semantic shift and covariate shift: new class (Horse) and new image conditions (night time Cat) require different product decisions. Reject scores are not certainty guarantees: an unknown can receive a high known class score, as U1 does; a difficult known input can be rejected, as K6 is. Unknown class diversity is unbounded: success on four horse images does not guarantee success on every unseen species or unrelated input domain. Open world recognition goes further than OSR if the system subsequently discovers, labels, and incorporates new classes over time. Merely outputting Unknown does not perform continual learning. Assess latency, human review capacity, fairness across capture conditions, dataset shift, and monitoring before deployment. Key takeaway: OOD detection focuses on whether the input belongs to the reference distribution . OSR focuses on accurately recognizing known classes while rejecting novel semantic classes . Effective OSR must solve both sides of that trade off. References Scheirer et al., Toward Open Set Recognition (2013) Geng, Huang & Chen, Recent Advances in Open Set Recognition: A Survey (2021) Wang, Vaze & Han, Dissecting Out of Distribution Detection and Open Set Recognition (2025)

What is the difference between data drift and concept drift in machine learning?

Data drift means the distribution of the inputs changes over time: Pₜ(X) ≠ P₀(X) . Concept drift means the relationship between features and the target changes: Pₜ(Y|X) ≠ P₀(Y|X) . They are different questions: Are we receiving different kinds of examples? versus Does the same kind of example now imply a different outcome? Both can coexist, and neither automatically proves that the model has become inaccurate. Beginner friendly example: a loan default model Suppose a bank predicts whether a customer will default. For simplicity, the only model input is employment type X: Salaried or Self employed . Historical reference population (1,000 customers): 800 salaried (80%) , with default rate 10% → 80 defaults . 200 self employed (20%) , with default rate 30% → 60 defaults . Overall default rate = (80 + 60)/1,000 = 14% . Scenario A — Data drift (covariate shift), no concept drift: The following month has 200 salaried (20%) and 800 self employed (80%) . Assume each group's conditional default rate stays unchanged: 10% for salaried, 30% for self employed. Expected defaults = 200 × 0.10 + 800 × 0.30 = 20 + 240 = 260 out of 1,000, or 26% . What changed? P(X): the proportion self employed rose from 20% to 80% . What stayed unchanged? P(default|Salaried) = 10% , and P(default|Self employed) = 30% . Overall default prevalence changed from 14% to 26% , but that alone does not mean concept drift . A model that accurately predicts risk within both groups could still remain correct. Scenario B — Concept drift with unchanged input mix: Keep the old input mix: 800 salaried and 200 self employed . A severe economic shock changes default risk within each employment type to 30% and 50% , respectively. Expected defaults = 800 × 0.30 + 200 × 0.50 = 240 + 100 = 340 , or 34% . What stayed unchanged? P(X): still 80% salaried, 20% self employed . What changed? P(default|X): 10% → 30% for salaried and 30% → 50% for self employed. A model continuing to predict the old conditional rates would underpredict risk even though feature distribution monitoring alone might show no shift . Scenario C — Both at once: If the input mix changes and default rates change within comparable employment groups, both data drift and concept drift are present. Numerical caveat: These are hypothetical population rates or expected counts, not statistically significant estimates from 1,000 random observations. Covariate shift, label shift, and concept drift Covariate shift: P(X) changes while P(Y|X) remains the same. Scenario A is a deliberately clean example. Concept drift / conditional shift: P(Y|X) changes. Scenario B isolates this effect. Label shift / prior shift: P(Y) changes while P(X|Y) stays the same. This is a different assumption from covariate shift and can change P(Y|X) through Bayes' rule. Important: A change in class prevalence P(Y) by itself does not diagnose label shift or concept drift. In Scenario A, prevalence changed as a consequence of changing the input mix. Label shift is not simply a synonym for any class balance change. Real systems can violate several of these assumptions simultaneously; taxonomy is an aid to diagnosis, not a guaranteed partition of every production incident. How to monitor the problem Monitor feature distributions without labels: track categorical proportions, missing values, new categories and numerical quantiles, segment level statistics, and feature level PSI, Jensen–Shannon distance, Wasserstein distance or suitable distribution tests. Compare matched windows and sample sizes. Watch data quality and pipelines: schema changes, unit mismatches, missing values, failed joins, and training serving inconsistencies can mimic drift. Check telemetry before attributing everything to changing customer behavior. Monitor outcomes once labels arrive: track segment wise default rates P(Y|X), observed versus predicted probabilities, Brier/log loss, precision/recall or other task relevant performance. Labels may arrive months later, so account for reporting delays and censoring. Drift tests: for numerical features, a KS test or distance based metric can detect distribution differences. For categorical features, use chi squared tests or proportion/distance checks with adequate expected counts. Conditional change analysis: compare outcome rates for matched feature segments or fit/test a recent period indicator, with appropriate adjustment for confounders and sufficient labeled observations. Avoid overreaction: a statistically significant shift on a huge dataset may be operationally trivial; a shift in a high impact minority segment may be urgent. Track effect size, confidence, sample volume and actual business outcomes. Does drift always hurt performance? Not necessarily: a classifier may generalize across the changed input mix. Distribution changes are a reason to investigate , not automatic evidence of degraded model quality. Data drift can change aggregate accuracy even if each segment's accuracy is stable, because the segment weights change. Concept drift may break the learned mapping and produce degraded calibration, classification decisions or costs despite unchanged feature proportions. Both may remain invisible in overall averages if harmful changes to one group are masked by improvements in another; examine important segments. Practical production response 1. Diagnose: confirm data lineage, compare reference and recent distributions, identify affected features and segments, check for pipeline faults and obtain trustworthy outcomes. 2. Assess impact: measure recent model quality and business cost against pre agreed guardrails; consider the delay before true labels are known. 3. Choose an intervention: fix a broken pipeline; recalibrate probabilities or adjust decision thresholds if justified; under suitable covariate shift assumptions consider importance weighting; retrain with representative recent labeled data when the learned relationship or relevant coverage has changed. 4. Validate before deployment: use time ordered train/validation/test splits, guard against leakage, compare the candidate with the existing model, and use a shadow or canary rollout where practical. 5. Continue monitoring: track input shifts, outcome based performance, operational alerts, fairness across segments and the effects of each model update. When should you retrain? Retraining is not mandatory just because PSI or a statistical test triggers . Retrain when fresh representative labels and reliable evaluation show meaningful degradation or changed target relationships, and a candidate model demonstrably improves the relevant outcomes. If labels are unavailable, prioritize input/data quality investigation, targeted labeling and safe fallbacks; drift proxies alone do not prove that retraining will help. Beware of automated retraining from contaminated, biased or delayed labels; use reproducible validation, monitoring and rollback. Applications and limitations Applications: credit risk, fraud detection, demand forecasting, recommender systems and support classification. Limitations: high dimensional joint shifts may not appear in individual feature tests; small segments have noisy estimates; delayed and selective labels complicate concept drift diagnosis. Production trade off: frequent retraining can improve adaptation but increases compute, review load and instability. Threshold updates and manual fallbacks may be safer near term fixes in some cases. Key takeaway: Data drift changes who or what the model sees; concept drift changes what the features mean for the outcome. Diagnose them with both distribution monitoring and outcome aware validation before choosing retraining. References Evidently: Data drift in production Evidently: Concept drift in production DataEval: Distribution shift taxonomy Google Cloud: MLOps automation and retraining

How can Population Stability Index (PSI) detect feature distribution drift?

Population Stability Index (PSI) compares the share of observations in fixed bins between reference and current production data. It detects changes in a single feature's distribution; it does not establish model performance degradation. Formula For bin i, reference share r i and current share c i: PSI = sum [(c i r i) ln(c i / r i)] . Use proportions and natural logarithms. PSI is zero when the binned distributions match, nonnegative, and not bounded by 1 . Numerical example: customer age Two groups of 1,000 applicants: Under 30: reference 500/1000 = 0.50 ; current 300/1000 = 0.30 . Ages 30–49: reference 300/1000 = 0.30 ; current 400/1000 = 0.40 . Age 50+: reference 200/1000 = 0.20 ; current 300/1000 = 0.30 . Compute individual contributions: Under 30: (0.30 0.50) ln(0.30 / 0.50) = 0.102165 . Ages 30–49: (0.40 0.30) ln(0.40 / 0.30) = 0.028768 . Age 50+: (0.30 0.20) ln(0.30 / 0.20) = 0.040547 . Total PSI = 0.102165 + 0.028768 + 0.040547 = 0.171480 . The first age group contributes most. This flags a changing applicant mix, not a proven reduction in prediction quality. Interpreting PSI Common heuristics: below 0.10 indicates relatively little shift; 0.10–0.25 suggests investigation; 0.25 or above often prompts closer scrutiny. These cutoffs are not universal or statistical significance thresholds. 0.1715 warrants investigating this feature under these heuristics, not immediately retraining. Choose operational alert thresholds with historical variation, sample size, risk and model sensitivity in mind. Binning and zeros For numeric data, choose reference quantiles or meaningful business intervals and freeze bin boundaries for production comparisons. Changing bin edges over time prevents a consistent comparison. For categorical data, use stable categories and explicitly track missing , new and other values. Too many bins produce sparse, noisy counts; overly broad bins obscure important movement. PSI ignores distances between bins. Zero bin proportions make a log ratio undefined and can imply an infinite unsmoothed PSI. One practical remedy is additive pseudocount smoothing to both reference and current counts , followed by renormalization. State the smoothing value and test sensitivity. With K bins and smoothing value a, use adjusted share (count i + a) / (total + K a) . Python demonstration Production monitoring and decisions Compare equivalent time windows against a trusted reference; monitor counts, missingness, new categories and per bin changes. Evaluate sample sizes, seasonal patterns, data quality defects and feature importance before escalating. Track model output distribution and, when ground truth labels arrive, actual error, loss and calibration. Small shifts in critical features can matter more than large shifts in irrelevant features. PSI is univariate and may miss changes in feature relationships. It does not detect conditional target changes P(Y|X) on its own. Do not automatically retrain based only on PSI. Investigate first, fix data pipelines if necessary, and retrain only when representative recent labels and validation indicate it improves outcomes. Monitor stability after release; guard against sparse bins, unequal sample sizes, arbitrary boundaries, seasonal effects and delayed labels. Key takeaway: PSI quantifies a feature distribution change across fixed reference bins. Use it as an investigation trigger, not evidence by itself that a deployed model should be retrained. References Haas and Sibbald: measuring data drift Evidently: PSI drift method

How do the Kolmogorov–Smirnov (KS) test and Wasserstein distance detect numerical feature drift?

The two sample Kolmogorov–Smirnov (KS) test and 1D Wasserstein distance compare reference and current numerical feature samples without requiring PSI's fixed bins. They measure different properties: KS captures the largest gap between empirical cumulative distributions , whereas Wasserstein 1 measures the total horizontal displacement of probability mass . Core definitions An empirical cumulative distribution function (ECDF) at value x is the fraction of sample observations ≤ x . KS statistic: D = sup x |F ref(x) − F cur(x)| , the maximum absolute ECDF separation. Its value lies in [0,1] , is dimensionless, and is unchanged by a strictly increasing transformation of both samples. 1D Wasserstein 1 distance: W1 = integral |F ref(x) − F cur(x)| dx . It can also be calculated by optimally matching sorted equal weight observations. W1 has the same units as the original feature . Both are univariate distribution comparison tools, not direct tests of model accuracy. Beginner friendly numerical example: monthly income feature A model historically saw these incomes in ₹ thousands : Reference (n=5): [40, 50, 60, 70, 80] . Current production (n=5): [65, 75, 85, 95, 105] . Both samples are sorted. Each current value is 25 thousand rupees larger than its corresponding reference value. Step 1 — Compute the ECDFs at informative values At x=60 : reference has 3/5 observations ≤ 60, so F ref(60)= 0.60 ; current has 0/5, so F cur(60)= 0.00 . The gap is 0.60 . At x=70 : F ref= 4/5=0.80 ; F cur= 1/5=0.20 . The gap is 0.60 . At x=80 : F ref= 5/5=1.00 ; F cur= 2/5=0.40 . The gap is 0.60 . Check all other observed jump points: no absolute gap exceeds 0.60. KS D = 0.60 . Step 2 — Compute 1D Wasserstein from sorted samples For two samples of equal length n with uniform weights, W1 = (1/n) sum i |sorted ref[i] − sorted cur[i]| . Matched distances are |40−65|=25 , |50−75|=25 , |60−85|=25 , |70−95|=25 , and |80−105|=25 , all in ₹ thousands. W1 = (25+25+25+25+25)/5 = 25 thousand rupees = ₹25,000 . The result represents the average amount of distance needed to transport the empirical probability mass under optimal one dimensional matching; it is not automatically an estimate of an individual customer's salary change. Step 3 — Understand the KS p value The two sample KS null hypothesis is that independent observations come from the same continuous underlying distribution . With these tiny samples, SciPy's two sided exact calculation gives p ≈ 0.357 for D=0.60, n ref=n cur=5. At a preselected 5% significance level , p 0.05 , so this tiny experiment does not reject the equal distribution null. This does not establish that the distributions are equal. The ₹25,000 Wasserstein distance still conveys the practical magnitude of the observed shift; different questions need different evidence. Reproduce with Python When should each method be used? Use KS when you need a dimensionless measure of the maximum cumulative separation and, under suitable sampling assumptions, a p value for equality of continuous distributions. Use Wasserstein 1 when the feature's numeric scale matters —for example, how many ₹ thousands the income distribution moved—and you want a magnitude sensitive distance. Use both when you need statistical evidence and an interpretable shift size. KS can see localized CDF discrepancies; Wasserstein accumulates differences over the numerical axis. Unlike PSI (Question 228), neither method requires manually defined bins. The main choice is instead how to validate significance, feature scales, reference periods and operational tolerances. Important limitations Sample size matters: with very small samples, KS can have low power despite a substantial observed shift; on huge datasets, a tiny operationally unimportant shift can have a very small p value. KS p value assumptions: the familiar two sample test assumes independent samples and a continuous null distribution. Many repeated values, discretized features or strong time dependence can make standard p values unreliable; consider a suitable permutation/block resampling design or other methods for those settings. Wasserstein has units and scale sensitivity: W1=25 is interpreted differently for income in ₹ thousands versus temperature in °C. If comparing features, use justified units or reference based normalization and report what was done. Outliers and tails: Wasserstein may respond strongly to distant observations, while KS only uses the largest vertical ECDF difference and ignores the magnitude of distance along the x axis. Neither is multivariate: a correlation change can go undetected when each feature's marginal distribution stays the same. Neither is concept drift detection: shifts in P(Y|X) require outcome based monitoring; a feature distribution alert alone does not prove worse predictions. Production monitoring workflow Fix data quality, timestamps and feature units first; select representative, sufficiently large, seasonally comparable reference and production windows. Compute KS D, its carefully interpreted p value and W1 in the feature's units . Inspect ECDFs, quantiles, missingness, affected segments and possible outliers. Account for multiple testing across many monitored features; a flood of unadjusted p values creates false positives. Calibrate action thresholds with historical variation, business impact and known safe changes; do not mechanically apply one W1 threshold to every feature. Investigate drift alerts alongside predictions and, when available, delayed ground truth labels, model loss and business KPIs. Avoid automatic retraining solely because a KS test is significant or W1 is large. Diagnose the source and validate a candidate fix on recent representative outcomes. Key takeaway: KS measures the maximum vertical difference between ECDFs; Wasserstein 1 measures the area between ECDFs and retains the feature's units. Report both the strength of statistical evidence and the practical size of a shift. References SciPy: ks 2samp SciPy: wasserstein distance

How can the chi-square test detect categorical feature distribution drift?

The Pearson chi square test of homogeneity evaluates whether the distribution of a categorical feature differs between a reference dataset and a recent production sample. For drift monitoring, it tests whether categories such as payment method, region, or device type have the same underlying proportions across two periods. Unlike numerical drift methods, it works with category counts , not ordered distances. Hypothesis and formula Null hypothesis (H0): the categorical feature has the same population proportions in the reference and production periods. Alternative (H1): at least one category proportion differs. Make a 2 × K contingency table : rows for reference and production, columns for the K categories. For each cell: expected count E = (row total × column total) / grand total . Chi square statistic: χ² = sum over all cells of (observed − expected)² / expected . Degrees of freedom = (number of rows − 1) × (number of columns − 1) ; for two periods and K categories, df = K − 1. Step by step numerical example: payment method drift An ecommerce classifier monitors a customer payment method feature with three categories: Card, UPI, and Cash . Observed reference counts (100 orders): Card 50 , UPI 30 , Cash 20 . Observed current counts (100 orders): Card 30 , UPI 40 , Cash 30 . Step 1 — Calculate totals Reference row total 100 , production row total 100 , grand total 200 . Category totals: Card 80 , UPI 70 , Cash 50 . Step 2 — Expected frequencies assuming no distribution change Reference Card: 100 × 80 / 200 = 40 ; current Card also 40 . Reference UPI: 100 × 70 / 200 = 35 ; current UPI also 35 . Reference Cash: 100 × 50 / 200 = 25 ; current Cash also 25 . Each row's expected counts total 40 + 35 + 25 = 100 . Step 3 — Compute chi square contributions for all six cells Reference Card: (50 − 40)²/40 = 2.5000 . Current Card: (30 − 40)²/40 = 2.5000 . Reference UPI: (30 − 35)²/35 = 0.7143 . Current UPI: (40 − 35)²/35 = 0.7143 . Reference Cash: (20 − 25)²/25 = 1.0000 . Current Cash: (30 − 25)²/25 = 1.0000 . Step 4 — Add contributions χ² = 2.5 + 2.5 + 0.7143 + 0.7143 + 1 + 1 = 8.4286 . Step 5 — Degrees of freedom and p value df = (2 − 1) × (3 − 1) = 2 . The asymptotic chi square distribution gives p ≈ 0.0148 . At a predeclared significance level of 0.05 , reject H0: the data provide evidence that the payment method distribution has changed. This is not a 98.52% probability that drift occurred, and it does not establish that the classifier has become less accurate. Python verification How large is the change? Statistical significance and effect size are not the same. Large samples can make even tiny shifts significant. A commonly reported association effect size is Cramér's V = sqrt(χ² / [N × min(R−1, K−1)]) , where N is the grand total. Here V = sqrt(8.4286 / (200 × 1)) ≈ 0.205 . This quantifies association between time period and payment category in the observed data. Do not assume one universal meaning of "small" or "large" V; interpret against business tolerance, category makeup, and historical baselines. Assumptions and small or rare categories Observations should represent independent events under the sampling design. Repeated purchases by the same person, overlapping windows, or time dependence can invalidate naive p values. Categories must be mutually exclusive and identically defined in both periods; use consistent handling of missing and newly appearing values. The usual chi square reference distribution is an approximation , so it can be unreliable with small expected counts. The often used rule of thumb is that expected cell counts should generally be at least 5 ; exact constraints depend on context. For a 2 × 2 table with sparse counts , Fisher's exact test is a standard alternative. For larger sparse tables, investigate suitable exact, Monte Carlo, or permutation methods; do not blindly apply a 2 × 2 only recipe. If categories are very rare, combine them only when the grouping is substantively defensible and documented. Do not merge classes merely to obtain the desired p value. For clustered users or strongly dependent observations, consider user level analysis or appropriate cluster/block resampling. Applications and production monitoring Applications: payment methods, region, browser type, device model, signup channel, document category, and other discrete features. Compare aligned reference and recent windows , checking changes in category definitions, missingness, traffic volume and seasonality. Monitor contributions by category : here Card contributes 5.0 of the total 8.4286 , helping focus investigation. Use a stable significance level and correct or otherwise account for multiple comparisons if testing dozens of features each day. Pair statistical alerts with effect sizes, absolute proportion shifts, model performance, segment outcomes, and business cost . If drift results from a pipeline bug, repair the pipeline. If real behavior changed, validate whether the deployed model is actually degraded before recalibration or retraining. Limitations The test detects changes in marginal categorical frequencies , not joint feature shifts or changes in P(Y|X) (concept drift). Large sample sizes can yield very small p values for negligible effects; small samples can fail to detect important drift. It does not convey category ordering or a physical distance between categories. A significant result is a signal for diagnosis , not a mandate to retrain or evidence of a causal explanation. Key takeaway: The chi square homogeneity test compares observed category counts with counts expected under stable proportions. Report χ², degrees of freedom, p value, effect size, and the affected categories before deciding whether to act. References SciPy: chi2 contingency SciPy: fisher exact