ValidAnytime
Documentation menu

Documentation

Overview

ValidAnytime turns any stream of numbers into alarms you can trust — using anytime-valid statistics, so looking as often as you like never inflates your false-alarm rate.

Reading with an AI assistant? Grab llms.txt — a plain-text map of the product and API you can paste into context.

The problem

Fixed-threshold dashboards make the same statistical mistake over and over: they test the same hypothesis on every new data point, then act surprised when noise crosses the line. Watch a live metric long enough and a false alarm becomes a near-certainty — so teams mute the tool and miss the failure that matters. This is the peeking problem.

The approach

ValidAnytime is built on anytime-valid sequential statistics — e-processes, confidence sequences, and ARL-calibrated e-detectors — where the error guarantee holds under continuous monitoring and optional stopping. Across many streams, online false-discovery control bounds false alarms fleet-wide. Every monitor can run two alarm tiers: the classical control charts (CUSUM, EWMA, static thresholds, rolling bands) as a sensitive warning tier whose calibration is model-based and therefore unbudgeted, and the e-process page tier, where alarms arrive inside a stated false-alarm budget.

  • Catch real change, ignore noise. Evidence accumulates; an alarm fires only when it crosses a calibrated threshold.
  • Look anytime. Peek once or continuously — the false-alarm budget is preserved.
  • Every alarm is explainable. Each one ships a guarantee_tag and theorem_ref — including an honest heuristic_adaptive tag on warning-tier alarms that carry no anytime-valid guarantee.

What you get

  • A simple ingest API and a Python SDK for any metric you produce.
  • A zero-touch backtest gate that proves a config on your own history before it goes live.
  • Trustworthy alarms with a guaranteed false-alarm budget.

Next steps

Quickstart