How does the Page–Hinkley test detect concept drift in a streaming model?

Page–Hinkley (PH) is a sequential, cumulative-sum change detector for a shift in the mean of a monitored numeric stream, such as per-example classification error (0/1) or regression loss. It accumulates small deviations instead of testing feature-distribution histograms. When errors or losses rise persistently above their prior behavior, the statistic grows and can trigger an alarm.

Core mechanism: upward mean-change detector

• At time t observe x_t and update the running mean μ_t = (x₁+…+x_t)/t (equivalently μ_t = μ_(t−1)+(x_t−μ_(t−1))/t).
• Choose a nonnegative tolerance δ to discount small upward fluctuations.
• Update the cumulative deviation m_t = m_(t−1) + (x_t − μ_t − δ), starting with m₀=0.
• Track the historical minimum M_t = min(0, m₁, …, m_t).
• Compute PH_t = m_t − M_t. This is nonnegative: it measures how far cumulative deviation has rebounded above its previous minimum.
• If PH_t > λ, where λ is the alarm threshold, report a potential upward shift (after a configurable warm-up).
• This is a transparent *unweighted, upward-only* educational formulation. Libraries can differ in their running-mean, forgetting-factor, two-sided and reset conventions. It is not the same as interpreting λ as a p-value.

Beginner-friendly numerical example: classification-error stream

A ticket-routing model reports x=0 when its prediction is correct and x=1 when incorrect. Use the hypothetical stream [0,0,0,0,1,1,1,1]. Choose δ=0.05, λ=1.00 and an illustrative minimum of four observations (a practical system should warm up longer).

Steps 1–4: all correct predictions

• t=1: x=0, μ=0, increment=0−0−0.05=−0.05; m=−0.05, M=−0.05, PH=0.
• t=2: μ=0, m=−0.10, M=−0.10, PH=0.
• t=3: μ=0, m=−0.15, M=−0.15, PH=0.
• t=4: μ=0, m=−0.20, M=−0.20, PH=0.
• Thus the historical minimum M remains −0.20 until the new failures arrive.

Step 5: first error

• μ₅ = 1/5 = 0.20.
• Increment = 1−0.20−0.05=0.75.
• m₅ = −0.20+0.75=0.55; M₅=−0.20.
• PH₅ = 0.55−(−0.20)=0.75: below λ=1.00, so no alarm.

Step 6: another error

• μ₆ = 2/6 ≈ 0.3333.
• Increment = 1−0.3333−0.05≈0.6167.
• m₆ ≈ 0.55+0.6167=1.1667; M₆=−0.20.
• PH₆≈1.1667−(−0.20)=1.3667 > 1.00, so the detector alarms at t=6.
• The change is in the monitored error stream. It warrants investigation and is not definitive proof of concept drift.

What if we kept accumulating without resetting?

• t=7: μ₇=3/7≈0.4286, m₇≈1.6881, PH₇≈1.8881.
• t=8: μ₈=4/8=0.50, m₈≈2.1381, PH₈≈2.3381.
• Those t=7–8 numbers are counterfactual continuation of the same unreset statistic. A real detector may reset immediately after the first alarm, so they are not predictions of a particular library's post-alarm output.

Verify with minimal Python

PYTHON
observations = [0, 0, 0, 0, 1, 1, 1, 1]
delta, lam = 0.05, 1.0
running_sum, cumulative, minimum = 0.0, 0.0, 0.0
for t, x in enumerate(observations, start=1):
    running_sum += x
    mean = running_sum / t
    cumulative += x - mean - delta
    minimum = min(minimum, cumulative)
    ph = cumulative - minimum
    if t in (4, 5, 6):
        print(t, round(mean, 4), round(cumulative, 4),
              round(minimum, 4), round(ph, 4))
    if t >= 4 and ph > lam:
        print("First alarm at", t)  # 6
        break

Tuning lambda, delta and direction

• λ (lambda): raising the alarm threshold demands more cumulative evidence, generally reducing false alarms but increasing detection delay; lowering λ catches smaller/shorter departures at the cost of extra alerts. Its numerical scale depends on the monitored values.
• δ (delta): subtracts a tolerance at each step. Larger δ discounts modest mean increases and may miss slow, small drifts; smaller δ reacts more readily to noise.
• Minimum observations: choose enough clean post-reset data to stabilize the mean; a warm-up of 4 is only for teaching. For example, scikit-multiflow documents min_instances=30 by default.
• Direction: the formula above catches increases in a loss/error signal. To detect a falling accuracy or other mean reduction, monitor an appropriately negated series or use a supported two-sided variant.
• Warning vs drift: standard River/scikit-multiflow Page–Hinkley detectors report drift only; unlike DDM/EDDM, there is no built-in warning state in those implementations. A custom early-warning threshold is possible but must be explicitly designed and evaluated.

Production design

• Which signal? Use binary per-example error for classification, or bounded/clipped appropriately scaled numeric loss for regression/forecasting. Strongly unbounded losses or sporadic outliers can dominate a cumulative statistic.
• Delayed labels: log prediction ID, event timestamp, model version and predicted output. Join finalized labels as they arrive, handle missing/censored labels, and decide deliberately whether to process by event time or label-arrival time. Do not treat pending ground truths as correct.
• Avoid misleading alarms: sudden label-policy changes, pipeline defects, class mix changes, threshold updates, class imbalance and correlated repeated observations can change error means without proving P(Y|X) drift.
• Sensitivity to noise: isolated errors may be absorbed by δ and λ, but serially correlated bursts can trigger false alarms. Calibrate under representative stable sequences and realistic change scenarios.
• Intervention: on alarm, inspect root causes and slice-level loss, collect representative new labels, evaluate a candidate on time-ordered holdouts, and deploy with rollout/rollback controls if performance improves. Reset or rebase detector state after confirmed adaptation; avoid mixing versions.

Comparison with nearby lessons

• DDM (Question 234): watches cumulative binary error probability with a binomial standard deviation and separate warning/drift rules.
• EDDM (Question 235): watches distances *between mistakes* and ratios relative to a historical best.
• ADWIN (Question 236): compares older and newer subwindow means under a confidence bound and automatically adapts its window length.
• Page–Hinkley: accumulates tolerated deviations of a numeric stream from its running mean; it is lightweight (constant-sized statistic in the basic version) and suitable for detecting sustained mean increases.
• Trade-off: PH is simple and memory-efficient, but parameters depend on scale, old-history effects can delay subtle changes, and a mean-change alarm does not identify its cause.

Key takeaway: Page–Hinkley tracks m_t − min(previous m). A persistent upward shift makes this statistic grow beyond λ. Use the alert as a reason to investigate observed performance, not an automatic retraining command.

References

• River: Page–Hinkley implementation
• scikit-multiflow: PageHinkley
• Change detection and adaptation in multi-target regression on data streams