SPC for ML

SPC for ML borrows a discipline manufacturing has used for a century — statistical process control — and points it at machine-learning metrics: accuracy, latency, eval scores, drift statistics. Instead of eyeballing a dashboard, you put the metric on a control chart with an explicit rule — a Shewhart-style static threshold, an EWMA chart, a CUSUM — and the rule, not a tired human, decides when the process has left its normal state. That step alone is a real upgrade over vibes.

The honest catch is calibration. Classical SPC rules are priced for an idealized world — independent points, known mean and sigma, a chart read at a controlled cadence — and ML metrics live in the opposite one: autocorrelated, heavy-tailed, watched continuously. The false-alarm promises printed in the textbooks degrade accordingly, and the drift-detector benchmark measures exactly how far, on labeled synthetic streams. Classical relatives you will meet in the SPC literature — Shewhart charts, CUSUM, EWMA, Page–Hinkley — all share this model-based calibration; ValidAnytime does not ship a Page–Hinkley detector, and its closest shipped relative is CUSUM, Page's own test.

The modern resolution is two tiers rather than a winner: run the SPC charts as fast, unbudgeted early warnings — they are genuinely quick on abrupt shifts — and put the false-alarm budget on an anytime-valid tier whose guarantee is built to survive continuous checking. You keep SPC's speed and discipline without pretending its calibration is a guarantee.