Documentation menu
Configure
Use-case matrix
Pick the template that matches your metric — or paste a sample and let auto-config pick. Every row is a distinct, gate-verified config: they share one budgeted page tier and differ only in how 'normal' is defined and which direction the warning tier watches.
The shared page tier
Every use-case runs the same budgeted, anytime-valid page tier: an ARL-calibrated e_detector on the conformal miss stream plus a coverage_e_process (false-alarm probability ≤ 1%, however often you look). That guarantee is metric-agnostic — so what a use-case tunes is the calibrator (how strict “normal” is) and the direction its sensitive, unbudgeted warning tier watches. Adaptive calibrators are deliberately not used for detection: they re-cover a drifting stream and would hide the very change you want caught.
| Template | Signal | Watches | Calibrator | Warn tier | Why |
|---|---|---|---|---|---|
General metric default | value | either direction | split-window (64) | two-sided CUSUM | Defend the learned level; page on a sustained shift either way. |
ML model drift ml_drift | prediction error / accuracy proxy | either direction | split-window (128) | two-sided CUSUM | A stiffer, lower-variance reference catches genuine drift without chasing routine noise. |
LLM / agent quality llm_quality | judge score / pass rate (higher is better) | downward | split-window (64) | downward EWMA | Don't adapt (that hides a slide); the warn tier watches only downward slides. |
Latency / cost spikes latency | latency / cost (lower is better) | upward | split-window (64) | upward CUSUM | The up-only warn detector accumulates sustained increases; improvements don't page. |
Business KPI kpi | KPI value (either direction matters) | either direction | split-window (128) | two-sided EWMA | A long window rides routine daily/weekly noise; a two-sided EWMA flags a sustained departure from the defended level either way. |
Pass a template id to POST /v1/monitors as config: {"template": "llm_quality"}, or fetch the resolved configs as JSON from GET /v1/use-cases (no auth). Directionality shown here lives in the warning tier; the budgeted page tier is two-sided everywhere (a true one-sided budgeted certificate is coming with bounded-risk monitors).
Don’t want to choose?
Paste a sample of your metric and the deterministic auto-config reads its shape — level, scale, direction, drift — picks and tunes a row, then ratifies it on your history with the backtest gate (“caught it on day X”). Via the API: POST /v1/onboarding/suggest (optionally with a use_case hint). Or point an agent at it — see the agent guide and the API reference.
On the roadmap
These capabilities are already supported by the underlying engine and are next on our build list. If one fits your use case, tell us — we prioritize by real demand and will email you the moment it ships.
Scheduled connectors
Auto-pull your metric on a schedule — no code, no webhook.
Point ValidAnytime at PostHog, a SQL warehouse, or another source and we fetch new values for you on a cadence. Today you push data in (CSV, batch API, or an inbound webhook URL); scheduled pull is what's next.
Multi-signal monitors
A/B & champion–challenger, error budgets, forecast calibration — all anytime-valid.
Beyond a single metric: a valid win certificate when one model beats another, a one-sided page when an SLO error budget breaks, or a catch when a probabilistic forecaster drifts out of calibration. The e-processes already live in the engine; wiring the extra signals is next.