ValidAnytime
  • Try it free
  • Observatory
  • Pricing
  • Docs
Sign inStart free
ValidAnytime

A guaranteed false-alarm budget across your whole fleet, valid no matter how often you look.

Made by Compiled Intelligence — a frontier AI lab working on quantitative finance from first principles; ValidAnytime is the monitoring we built for our own model fleets, productized.

Product

  • How it works
  • Pricing
  • Observatory
  • Try it free
  • Dashboard

Solutions

  • Who it's for
  • For LLM engineers
  • For MLOps engineers
  • For SRE & platform
  • LLM eval regression
  • Model drift detection
  • Alert fatigue

Developers

  • Docs
  • Quickstart
  • Python SDK
  • API reference

Learn

  • Benchmark
  • Detector guides
  • Glossary
  • Compare
  • Why dashboards lie
  • Blog
  • Anytime-valid 101
  • Winter Storm Uri case study
  • Incident ledger
  • Observatory methodology

Company

  • Contact
  • Privacy
  • Terms

Observatory alarms & product updates, by email

Rare emails only. We store your address and the tag “footer” — nothing else. Unsubscribe anytime.

© 2026 ValidAnytime. All rights reserved.

Every alarm ships with its guarantee_tag and theorem_ref.

Observatory
CI

Has California (CAISO)'s demand forecast drifted?

CISO · updated Jul 6, 22:08 UTC

No regime change detected

California (CAISO) — no forecast-error regime change detected

The day-ahead forecast's error has stayed within its calibrated band. Not a certificate that nothing happened.

quiet streak 5.5 yearsbudget ≈1 false page per BA-yearwatching since Jan 2021
5.68%
mean abs. forecast error

of demand (MAPE)

25,283 MW
mean demand
902 MW
typical error

median |error|

7,931 MW
worst-hour error

99th pctile |error|

The evidence, last two weeks

Because the baseline is the operator’s own forecast, seasonality is subtracted for free — what’s left is the miss. The detector pages only when that miss enters a new regime and clears the page line.

0 = the operator’s forecast is exactly rightforecast error — actual demand minus day-ahead forecast (MW)page line (≈1 false page per grid-year)accumulated evidence (anytime-valid log-statistic)Jun 22, 2026Jul 6, 2026
Real EIA-930 data for California (CAISO) — the last 336 hourly points. The forecast is the grid operator’s own day-ahead demand forecast (not ours); the error is actual − forecast, peaking at +10,665 MW in this window. The evidence statistic stayed under the page line: no regime change detected. That is not proof nothing happened. Back-in-band is a display heuristic, not a certificate of absence.

Forecast-error regime changes0 in the monitored history

Each is a dated page inside the stated budget. Labels attach only where a window overlaps a documented grid-stress event — never invented. We detect that the forecast’s error changed regime, not that the grid failed or why.

No forecast-error regime change has cleared this grid’s page budget in the monitored history. Absence of entries is not proof nothing happened — it means nothing crossed the page line.

How this verdict is formed

The hourly error runs through a conformal band that learns California (CAISO)’s own normal forecast-error rate, into a Shiryaev–Roberts e-detector on a bounded rolling window. An alarm fires only when accumulated evidence crosses the page line — and because the statistic is anytime-valid, checking as often as you like never inflates the false-alarm rate. Full recipe, and exactly what we do not claim, on the methodology page.

Embed this badge

A live status pill for California (CAISO), refreshed every publish. Drop it in a README, status page, or dashboard — it always shows the current verdict and links back here. It reads no regime change while the forecast holds, and flips to forecast drift detected when the e-detector pages.

California (CAISO) demand-forecast monitor · ValidAnytimelive preview
Markdown
[![California (CAISO) demand-forecast monitor · ValidAnytime](https://validanytime.com/observatory/badge/CISO)](https://validanytime.com/observatory/CISO)
HTML
<a href="https://validanytime.com/observatory/CISO"><img src="https://validanytime.com/observatory/badge/CISO" alt="California (CAISO) demand-forecast monitor · ValidAnytime"></a>
Monitor your metricsBacktest your data