Model Monitoring
Why monitor a model at all?
Training accuracy is a snapshot. The moment a model is deployed, it begins to age. The code never changes, the weights never change — yet performance can quietly erode because the world the model sees in production drifts away from the world it was trained on. A fraud model trained before a new scam appears, a demand forecaster trained before a supply shock, a recommender trained before a viral trend: each degrades silently, and nothing in the logs screams about it.
Model monitoring is the practice of continuously watching a deployed model and its data so that this silent decay becomes visible. The hard part is that the obvious signal — accuracy — usually is not available in real time, because the true labels arrive late or never. So good monitoring leans heavily on signals you can measure immediately: the distribution of the inputs, the distribution of the predictions, data-quality checks, and operational metrics.
The core tension: the thing you most want to track (was the prediction correct?) is the thing you can least often observe in time. Monitoring is the art of detecting trouble without waiting for ground truth.
Data drift vs. concept drift
A model learns a joint distribution over inputs x and targets y, which factors as P(x, y) = P(x) · P(y | x). “Drift” just means one of these factors changed after deployment. Which factor changed determines what you can detect and how you should respond.
Data drift (covariate shift)
P(x) changes but P(y | x) stays the same. The inputs move into regions the model rarely saw — new user segments, new sensor ranges — even though the underlying rule relating x to y is unchanged. Detectable from inputs alone, no labels required.
Concept drift
P(y | x) changes — the relationship itself moved because the world changed. The same input now implies a different answer (e.g. spending patterns that once meant “fraud” are now normal). Hard to detect without labels, because P(x) can look completely unchanged.
Label / prior shift
The class base rate P(y) changes — e.g. the fraction of fraudulent transactions doubles during a holiday — while P(x | y) stays fixed. Shows up in the prediction distribution and calibration even if individual features look stable.
Why accuracy falls with the code unchanged: the model encodes a fixed estimate of P(y | x). If the inputs drift into sparsely-trained regions (data drift) the estimate is unreliable there; if the true P(y | x) moves (concept drift) the estimate is simply wrong. Either way the decision boundary is now in the wrong place for the data arriving today — no bug required.
Detecting drift without labels
Because ground-truth labels are usually delayed or absent, the first line of defense is unsupervised: compare the live input distribution against a saved training reference distribution. If the two distributions diverge, something has changed — even before you know whether predictions were right. Three standard tools:
Population Stability Index (PSI)
Bin the feature, then compare the fraction of mass in each bin for reference (p_ref) vs. live (p_live):
PSI = Σᵢ (p_liveᵢ − p_refᵢ) · ln(p_liveᵢ / p_refᵢ)
Rule of thumb: PSI < 0.1 = no real shift, 0.1–0.2 = moderate (investigate), > 0.2 = major drift (act). PSI is symmetric-ish and the de-facto standard in credit and risk monitoring.
KL divergence
Measures the extra surprise of modeling live data with the reference distribution:
D_KL(live ‖ ref) = Σᵢ p_liveᵢ · ln(p_liveᵢ / p_refᵢ)
Always ≥ 0, and 0 only when the distributions match. It is asymmetric (live‖ref ≠ ref‖live), so pick the direction deliberately. Measured in nats with the natural log.
Kolmogorov–Smirnov (KS) test
For continuous features, the KS statistic is the maximum gap between the two cumulative distribution functions, D = supₓ |F_live(x) − F_ref(x)|. It needs no binning and yields a p-value, but with millions of rows even tiny, harmless shifts become “significant” — so monitor effect size, not just the p-value.
The crucial caveat: these unsupervised tests detect data drift in P(x). They are blind to concept drift when P(x) is unchanged — the inputs can look identical while P(y | x) silently moves. Catching that genuinely requires labels (or a proxy for them). Input-distribution monitoring is necessary but not sufficient.
Interactive: a drift-detection monitor
The gray histogram is the model's frozen training reference for one feature. Use the sliders to shift, spread, or skew the live distribution — simulating the world changing over time. PSI and KL are recomputed live from the two histograms (with zero-bin guarding). When PSI crosses 0.2 the monitor turns red and fires an alert.
Simulate drift
The gray bars are the fixed training reference. The outlined bars are the live feature distribution. Notice that no ground-truth labels are used anywhere here — PSI and KL compare only the input distributions, so drift is caught even when you do not yet know whether predictions were right.
Try this: a pure mean shift moves the mass sideways and PSI climbs fast. A small spread with no shift still raises PSI — the tails now land in bins the reference barely covered. This is exactly how a monitor flags trouble using inputs alone, with no labels in sight.
Interactive: drift accumulates, accuracy decays, retrain
Drift is rarely a single jump — it accumulates. Here the live distribution drifts a little further each “day.” PSI (orange) rises while real accuracy (green) falls. The dashed red line is the PSI alert threshold; where it is first crossed becomes the retraining trigger (purple). Crank up the drift speed and watch the trigger fire earlier.
How fast the world moves away from the training distribution.
What else to watch
Input drift is only one channel. A production monitoring stack layers several signals so a problem invisible in one shows up in another.
| Signal | What it catches | Needs labels? |
|---|---|---|
| Input / feature distribution | Data drift / covariate shift via PSI, KL, KS per feature | No |
| Prediction distribution | Shift in the rate of predicted classes / output values — a clue to label shift or concept drift | No |
| Confidence / score calibration | Rising low-confidence rate, or scores that no longer match realized frequencies | Partial (labels confirm) |
| Data quality / schema | Nulls, type mismatches, out-of-range values, new categories, a broken upstream pipeline | No |
| Business / operational | Latency, throughput, error rate, and downstream KPIs (clicks, approvals, revenue) — the metrics that actually matter | No |
| Accuracy / loss vs. ground truth | The real thing — concept drift included — but only once labels arrive | Yes |
Data quality first. Before blaming the model, confirm the inputs are sane. A huge fraction of “model degradation” incidents are really a renamed column, a unit change, a feature pipeline that started emitting nulls, or a join that silently dropped rows. Schema and range checks catch these instantly and cheaply.
The delayed-label problem
The reason monitoring leans so hard on unsupervised signals is the delayed-label problem: the ground truth needed to measure accuracy often arrives long after the prediction, or never.
- Loan default: you may not learn whether a loan defaults for months or years after approval.
- Churn / lifetime value: the “answer” unfolds over the customer's whole future.
- Medical outcomes: a diagnosis may only be confirmed after extended follow-up.
- No feedback at all: for many ranking or pricing decisions you never observe the counterfactual outcome.
Until labels land, you are flying on drift metrics, prediction distributions, and data-quality checks. When labels finally arrive, they feed a delayed accuracy curve that confirms (or contradicts) the earlier unsupervised warnings — and feeds the retraining set.
Alerting, retraining triggers, and the feedback loop
Monitoring is only useful if it drives action. The loop closes when metrics crossing thresholds trigger an alert, a human investigation, and ultimately a retrain on fresher data.
Setting thresholds
- Static rules of thumb (PSI > 0.2) are a fine starting point.
- Better: thresholds calibrated on historical noise so you do not drown in false alarms.
- Require a metric to stay breached for several windows before paging — single-window spikes are often noise.
- Tune the alert/no-alert tradeoff like any detector: too tight and you miss real drift, too loose and people ignore the pager.
Retraining triggers
- Scheduled: retrain every N days regardless — simple, but wasteful or too slow.
- Triggered: retrain when drift or a degraded metric crosses threshold — responsive, the goal of monitoring.
- Performance-based: retrain when realized accuracy on freshly-labeled data drops below a floor.
- Always validate the new model offline before it replaces the old one — a retrain can make things worse.
The feedback loop: serve → log inputs, predictions, and (eventually) labels → monitor drift and performance → alert on breach → retrain and validate → redeploy. A healthy production ML system runs this loop continuously, not as a one-off. Beware feedback contamination: when a model's own decisions shape the data it later trains on (you only see repayment for loans you approved), the loop can entrench its own biases.
Common pitfalls
✗ Mistakes
- • Monitoring only accuracy — and waiting forever for labels
- • Assuming stable inputs mean a healthy model (concept drift hides)
- • Forgetting to floor zero bins, so PSI/KL explode to ∞
- • Trusting KS p-values at huge sample sizes (everything is “significant”)
- • Alert fatigue from thresholds that are too sensitive
- • Skipping data-quality checks and blaming the model
✓ Good practice
- • Layer signals: inputs, predictions, quality, business KPIs
- • Save a fixed training reference to compare against
- • Track per-feature drift, not just an aggregate
- • Watch effect size and trends, not just p-values
- • Require sustained breaches before paging
- • Automate the serve → monitor → retrain → validate loop
Key Takeaways
- Deployed models degrade silently: the code is unchanged but the world drifts away from the training distribution, so the fixed decision boundary lands in the wrong place.
- Data drift changes P(x) (covariate shift); concept drift changes P(y | x) (the world's rule moved); label shift changes the base rate P(y).
- Because labels are delayed or absent, drift detection is usually unsupervised: compare live vs. reference input distributions with PSI, KL divergence, or the KS test.
- Unsupervised input monitoring catches data drift but is blind to concept drift when P(x) is unchanged — that needs labels.
- PSI = Σ (p_live − p_ref)·ln(p_live/p_ref); alert around PSI > 0.2. Floor zero bins at a small ε so the log never blows up.
- Drift metrics warn before accuracy fully collapses, giving you time to retrain. Close the loop: serve → monitor → alert → retrain → validate → redeploy.