False discovery rate

The false discovery rate is the fraction of fired alarms that were wrong: of everything that fired, how many were false. When you watch one metric, you worry about one false alarm; when you watch hundreds, that fraction becomes the right question. Controlling it is what separates 'we get lots of alerts' from 'when it fires, we look' — because you have a bound on how often the alarm cries wolf across the whole system.

This matters precisely because alert fatigue is a fleet-level problem. A per-metric error rate that seems tiny becomes a daily flood once you multiply it by every stream and every look. An FDR guarantee spends a single budget across all of them, so the noise stays bounded no matter how many monitors you add.

ValidAnytime controls FDR online — across streams and across time — using multiple-testing procedures adapted to anytime-valid e-values. It is the piece most dashboards lack: they alert per-metric, with no global promise about how many of those alarms are false.