{ "cadence": "weekly — Tue 04:15 UTC (.github/workflows/accuracy_backtest.yml)", "checks": [ { "artefact_summary": { "fleet_annual_r": 0.844, "r_annual_median": 0.609, "r_monthly_metered_median": 0.877, "r_monthly_metered_p10": 0.487, "r_monthly_metered_p90": 0.971, "ratio_iec2_median": 0.8212, "ratio_tt_2012-2015": 0.8553, "ratio_tt_2016-2018": 0.924, "ratio_tt_median": 0.824, "ratio_tt_pre-2012": 0.7428 }, "caveats": [ "metered CF includes grid curtailment and real availability — the metered ÷ modeled ratio is an UPPER bound on resource-model error", "frame filter (≥ 50 MW, in service before 2019) is read from the artefact's meta.filter — carried as sample.min_cap_mw / sample.in_service_before and flagged in metrics_from_summary, not recomputed; the rows are consistent with it (smallest plant 50 MW, latest op_year 2018)", "turbine-true leg: 559 of 583 plants carry a tt_ratio; the artefact records 19 excluded for USWTDB coverage outside range (summary.tt_coverage_excluded, carried as sample.n_tt_coverage_excluded — copied, not recomputed: the rows cannot separate the two exclusion reasons), the remaining 5 carry no usable turbine-true statistics (no tt record, or too few tt months) — all 24 are excluded from the tt statistics", "fleet_annual_r / worst_year_* are recomputed from the artefact's summary per-year fleet series (n = 6 years), not from per-plant rows — the per-plant monthly r is the sturdier claim", "Measured CF includes grid curtailment (not modeled; LBNL reports mid-to-high single-digit wind curtailment in SPP/ERCOT/MISO in recent years) and real availability/degradation (modeled as a fixed net-farm loss): the ratio is an UPPER bound on resource-model error, and part of the residual varies with fleet AGE (see by_vintage_tt) — older cohorts under-perform the model more than younger ones.", "The turbine-true leg converts each turbine group's single containing ERA5 cell (no bilinear cell blend), so part of the baseline-vs-turbine-true difference is chain construction, not siting. The auto layer's class map is fleet-calibrated (2026-07-29, USWTDB specific powers; IEC_VAVE_BINS derivation in weather-store/derive_iec_bins.py): the auto leg now tracks the turbine-true fleet CF within ~0.3% — before calibration the nominal IEC Vave bins put ~78% of fleet capacity on the low-wind Class III curve and auto read +7.4% above the validated level.", "Fleet-aggregate annual r has n=6 years and is dominated by the 2023 low; the per-plant annual r distribution and the worst-year agreement are the sturdier inter-annual claims.", "Fleet filter: US onshore >=50 MW plants fully in service before 2019 (2023 EIA-860 vintage); measured basis is gross-of-storage plant wind net generation." ], "forecast": "cfe8760.fetch_cf_year product chain (ERA5-rooted, meso × terrain × density; iec2 baseline + turbine-true leg) at each plant, monthly CF", "frame": "US onshore ≥ 50 MW plants fully in service before 2019; monthly-shape stats on the M-frame (Respondent Frequency M/AM only — annual respondents' months are EIA allocations, not meters)", "group": "resource_record", "id": "eia923_wind_us", "label": "US wind — EIA-923 metered generation", "metrics": { "fleet_annual_n_years": 6, "fleet_annual_r": 0.844, "r_annual_median": 0.609, "r_monthly_metered_median": 0.877, "r_monthly_metered_p10": 0.487, "r_monthly_metered_p90": 0.971, "ratio_iec2_median": 0.8212, "ratio_iec2_p10": 0.5129, "ratio_iec2_p90": 1.0744, "ratio_tt_by_vintage": { "2012-2015": { "cap_gw": 24.7, "n": 159, "ratio_median": 0.8553 }, "2016-2018": { "cap_gw": 20.1, "n": 113, "ratio_median": 0.924 }, "pre-2012": { "cap_gw": 38.6, "n": 287, "ratio_median": 0.7428 } }, "ratio_tt_median": 0.824, "ratio_tt_p10": 0.5684, "ratio_tt_p90": 0.9969, "worst_year_measured": "2023", "worst_year_modeled": "2023" }, "metrics_from_summary": [ "fleet_annual_r", "fleet_annual_n_years", "worst_year_measured", "worst_year_modeled", "sample.years", "sample.min_cap_mw", "sample.in_service_before", "sample.n_tt_coverage_excluded" ], "realised": "EIA-923 monthly net wind generation ÷ EIA-860 nameplate (metered basis, gross of storage)", "recomputed_from_records": true, "sample": { "cap_gw": 87.2, "in_service_before": 2019, "min_cap_mw": 50.0, "n_plants": 583, "n_tt_coverage_excluded": 19, "n_turbine_true": 559, "n_turbine_true_excluded": 24, "n_with_metered_monthly": 570, "years": [ 2019, 2024 ] }, "self_check": { "basis": "recomputed value rounded to the artefact's stated precision", "discrepancies": [], "median_convention": "statistics.median — the two middle values are averaged for an even n", "n_compared": 10, "not_compared": [], "tolerance": 0.002 }, "source": "/data/eia923_validation.json", "source_built": "2026-07-30T14:26:22Z", "source_version": 2, "status": "measured" }, { "artefact_summary": { "all_r_monthly_median_upper": 0.887, "all_ratio_median_upper": 0.6743, "offshore_curtailment_share_capw": 0.0378, "offshore_r_monthly_median_upper": 0.939, "offshore_ratio_median_upper": 0.776, "offshore_ratio_pre_curtailment_median_upper": 0.778, "onshore_curtailment_share_capw": 0.1553, "onshore_r_monthly_median_upper": 0.859, "onshore_ratio_median_upper": 0.58, "onshore_ratio_pre_curtailment_median_upper": 0.701 }, "caveats": [ "metered CF includes curtailment (decomposed per BMU from Elexon FPN − accepted level) and real availability — ratio_median is delivered ÷ modeled; ratio_pre_curtailment_median adds the curtailed energy back", "metered window Feb 2019–Dec 2024 (71 months) is read from the artefact's summary method text — carried as sample.months_span and flagged in metrics_from_summary, not recomputed (a site carries only its n_months); the rows are consistent with it (longest site record 69 months, median 69)", "median convention: the artefact's summary takes the upper middle value (sorted[n // 2]) for an even n — for all 62 sites its ratio_median / r_monthly_median are 0.6743 / 0.887 (carried as *_median_upper); the published medians average the two middle values (0.6677 / 0.882); the per-type groups have an odd n, so both conventions agree there", "Elexon B1610 settlement-metered monthly CF on per-month REPORTING capacity (BMUs past their commissioning ramp and reporting >=2% of their median; site-months below 60% of steady reporting capacity dropped) vs the iec2 chain, Feb 2019 - Dec 2024; December fully repaired (the original sweep window clipped 31 Dec)", "Elexon FPN minus bid-offer-accepted level, time-weighted per settlement period, per BMU (the standard constraint-volume method); pre-curtailment ratio = delivered ratio x (1 + curtailed/metered). Residual pre-curtailment gap = model bias + wakes/availability beyond the fixed farm losses.", "v2: per-month REPORTING-capacity denominators (commissioning ramps, B1610 gaps and part-retirements excluded) + full Decembers. Per-site monthly r tests temporal shape at a fixed cell; CROSS-SITE r tests site ranking — near zero here; treat fleet ratios as fleet context, never a per-site correction." ], "forecast": "iec2 chain monthly CF at each BMU", "frame": "GB wind BMUs past commissioning ramp, Feb 2019–Dec 2024; curtailment = FPN − accepted level (Elexon), per BMU", "group": "resource_record", "id": "gb_b1610_wind", "label": "GB wind — Elexon B1610 settlement-metered generation", "metrics": { "all": { "cap_gw": 13.0, "n": 62, "r_monthly_median": 0.882, "r_monthly_median_upper": 0.887, "ratio_median": 0.6677, "ratio_median_upper": 0.6743 }, "offshore": { "cap_gw": 9.6, "curtailment_share_capw": 0.0378, "curtailment_share_median": 0.0033, "n": 25, "r_monthly_median": 0.939, "r_monthly_median_upper": 0.939, "ratio_median": 0.7761, "ratio_median_upper": 0.7761, "ratio_pre_curtailment_median": 0.7781 }, "onshore": { "cap_gw": 3.4, "curtailment_share_capw": 0.1553, "curtailment_share_median": 0.1413, "n": 37, "r_monthly_median": 0.859, "r_monthly_median_upper": 0.859, "ratio_median": 0.5799, "ratio_median_upper": 0.5799, "ratio_pre_curtailment_median": 0.701 } }, "metrics_from_summary": [ "sample.months_span" ], "realised": "Elexon B1610 settlement-metered monthly CF on per-month reporting capacity", "recomputed_from_records": true, "sample": { "by_type": { "offshore": 25, "onshore": 37 }, "cap_gw": 13.0, "months_span": [ "2019-02", "2024-12" ], "n_months_median": 69, "n_sites": 62 }, "self_check": { "basis": "recomputed value rounded to the artefact's stated precision", "discrepancies": [], "median_convention": "the artefact's summary takes the upper middle value (sorted[n // 2]) for an even n; compared like-for-like via *_median_upper, recomputed from the rows in that convention; the published *_median averages the two middle values (statistics.median)", "n_compared": 10, "not_compared": [], "tolerance": 0.002 }, "source": "/data/gb_wind_validation.json", "source_built": "2026-08-02T04:42:37Z", "status": "measured" }, { "artefact_summary": { "cf_bias_mean": 0.1244, "cf_r_max": 0.953, "cf_r_median_upper": 0.878, "cf_r_min": 0.637, "speed_bias_pct_mean": 28.4, "speed_r_max": 0.984, "speed_r_median_upper": 0.89, "speed_r_min": 0.538, "worst_year_agree": 28, "worst_year_n": 36 }, "caveats": [ "MERRA-2 is an independent reanalysis, not a meter: agreement bounds the reanalysis-choice error, not the model-vs-meter error", "sign convention (etl/merra2_crosscheck.py): speed_bias_pct_mean = mean over sites of 100 · (MERRA-2 − ours) / ours, where ours is our raw ERA5-store 100 m annual-mean speed and MERRA-2 is the shear-lifted WS50M; cf_bias_mean = mean over sites of (MERRA-2 CF − our CF); positive = MERRA-2 higher; both are plain means of the per-site values; the artefact's summary rounds speed_bias_pct to 1 dp", "track spans: speed_n_years / cf_n_years are the sites' own; the calendar spans (sample.speed_track_years / sample.cf_track_years) are read from the artefact's summary method text, not from the rows, and are flagged in metrics_from_summary", "median convention: the artefact's summary takes the upper middle value (sorted[n // 2]) for its 36 sites — speed_r_median_upper 0.89 and cf_r_median_upper 0.878 are that figure (the one the methodology page quotes); the published speed_r_median 0.889 and cf_r_median 0.866 average the two middle values", "NASA POWER (MERRA-2), chain-symmetric: both sides use RAW reanalysis 100 m speeds — ours native from the ERA5 store cell (no meso/terrain/density corrections), MERRA-2's WS50M lifted with a per-site empirical power-law alpha from its own WS10M/WS50M monthly means (clamped (0.05, 0.45), fallback 0.14). Speed track 1981-2024: annual means, r + bias + drought agreement (our worst year within MERRA-2's worst 3 of 44). CF track 2001-2024 (POWER hourly floor): hourly speeds through the identical bare cf_model.wind_cf_series both sides — reanalysis difference in product units; fill hours skipped, years <95% valid dropped. Product numbers additionally carry meso/terrain/density corrections, validated separately (PVGIS, EIA-923 planned).", "36 globally stratified windy-region on-land sites; excludes offshore, low-wind and equatorial-calm cells — headline stats are for this sample, not map-wide", "chance worst-year agreement: ~7% (3 of 44 years)" ], "forecast": "our raw ERA5-store 100 m annual-mean speed (no corrections) and bare cf_model.wind_cf_series CF", "frame": "36 globally-stratified windy on-land sites; speed track 1981–2024 (n = 44 yr), CF track 2001–2024 (n = 24 yr)", "group": "resource_record", "id": "merra2_wind_reanalysis", "label": "Wind resource — MERRA-2 reanalysis cross-check", "metrics": { "cf_bias_mean": 0.124, "cf_r_max": 0.953, "cf_r_median": 0.866, "cf_r_median_upper": 0.878, "cf_r_min": 0.637, "speed_bias_pct_mean": 28.397, "speed_r_max": 0.984, "speed_r_median": 0.889, "speed_r_median_upper": 0.89, "speed_r_min": 0.538, "worst_year_agreement": { "agree": 28, "n": 36 } }, "metrics_from_summary": [ "sample.speed_track_years", "sample.cf_track_years" ], "realised": "NASA POWER MERRA-2 WS50M lifted to 100 m with a per-site empirical shear (independent assimilation system, not a meter)", "recomputed_from_records": true, "sample": { "cf_n_years": 24, "cf_track_years": [ 2001, 2024 ], "n_alpha_empirical": 36, "n_sites": 36, "speed_n_years": 44, "speed_track_years": [ 1981, 2024 ] }, "self_check": { "basis": "recomputed value rounded to the artefact's stated precision", "discrepancies": [], "median_convention": "the artefact's summary takes the upper middle value (sorted[n // 2]) for an even n; compared like-for-like via *_median_upper, recomputed from the rows in that convention; the published *_median averages the two middle values (statistics.median)", "n_compared": 10, "not_compared": [], "tolerance": 0.002 }, "source": "/data/merra2_crosscheck.json", "source_built": "2026-07-28T06:50:19Z", "status": "measured" }, { "artefact_summary": { "fixed_bias_ac_mean": 0.058, "fixed_bias_dc_mean": 0.011, "fixed_corr_mean": 0.952, "fixed_nmae_pct_mean": 40.57, "ghi_corr_mean": 0.969, "tracker_bias_ac_mean": 0.058, "tracker_bias_dc_mean": 0.006, "tracker_corr_mean": 0.902, "tracker_nmae_pct_mean": 46.83 }, "caveats": [ "PVGIS is a model (SARAH3 / ERA5 irradiance through the JRC PV chain), not a meter — agreement bounds model-vs-model spread, not model-vs-meter error", "per-site values in the artefact are stored to 3–4 dp, so means recomputed from them can differ from the artefact's own aggregate in the last digit", "PVGIS seriescalc (EU JRC) — 1 kWp crystSi PV, loss=8%: fixed at cf_model's optimal tilt (equatorward) and single-horizontal-axis tracker (trackingtype=1); P/1000 -> CF. Horizontal G(i) for the GHI-input check; WS10m for the wind cross-check.", "Solar compares cf_model v2 NET AC CF against PVGIS modelled PV output for the MATCHING array (tracker->trackingtype=1, fixed->optimal tilt). Hourly correlation validates the irradiance->POA->PV physics; the annual-energy bias is a design/normalisation difference (cf_model's ILR 1.30 + AC clip vs PVGIS's 1:1 kWp), not a site-selection error. The residual NMAE is inflated by normalising an intermittent quantity (half the hours are night-time zeros) by a small mean. GHI corr isolates the weather-input agreement. wind_cf is a GROSS-curve cross-consistency check vs PVGIS WS10m shear-lifted to 100 m — both ERA5-rooted, magnitude shear-dependent; honest correlation only, not validation. net_loss_factor documents the net-vs-gross farm loss stack empirically. S-hemisphere tracker comparisons (Sydney) read ~4-5 pp high vs PVGIS trackingtype=1 with corr ~0.80 while the SAME site's fixed array validates cleanly (corr 0.969, DC-basis +0.013) — a PVGIS tracker-request convention artifact at southern latitudes, not a model error (our tracker geometry is hemisphere-symmetric and parity-tested against CompoundVision).", "PVGIS SARAH3/ERA5 hourly stamps are hour-beginning (:10 past); our labels are hour-ending. The index shift was chosen ONCE by max hourly correlation on the first site's tracker CF and applied to every PVGIS series at every site. offset 1 = PVGIS shifted forward one hour to meet our hour-ending labels.", "artefact carries no build stamp (no build date is recorded in it); provenance is the model version it names (carried here as source_version) and the repo commit that added it" ], "forecast": "cf_model v2 net AC CF, fixed at optimal tilt and single-axis tracker, hourly", "frame": "six globally-stratified sites, calendar 2020, 8,783 h per site", "group": "resource_record", "id": "pvgis_solar", "label": "Solar — PVGIS hourly cross-validation", "metrics": { "fixed": { "bias_ac_mean": 0.058, "bias_dc_mean": 0.011, "corr_mean": 0.952, "corr_min": 0.905, "mae_mean": 0.07, "mean_ours": 0.235, "mean_truth": 0.177, "n_sites": 6, "nmae_pct_mean": 40.567 }, "ghi_corr_mean": 0.969, "n_hours_per_site": 8783, "tracker": { "bias_ac_mean": 0.058, "bias_dc_mean": 0.006, "corr_mean": 0.902, "corr_min": 0.8, "mae_mean": 0.091, "mean_ours": 0.262, "mean_truth": 0.204, "n_sites": 6, "nmae_pct_mean": 46.833 } }, "metrics_from_summary": [], "realised": "PVGIS seriescalc (EU JRC, SARAH3/ERA5) hourly PV output for the matching array, 1 kWp DC reference (a model, not a meter)", "recomputed_from_records": true, "sample": { "n_hours_per_site": 8783, "n_sites": 6, "pvgis_db": [ "PVGIS-ERA5", "PVGIS-SARAH3" ], "sites": [ "Berlin, DE", "Delhi, India", "Drax, UK", "Phoenix, US", "Seville, Spain", "Sydney, AU" ], "year": 2020 }, "self_check": { "basis": "recomputed value rounded to the artefact's stated precision", "discrepancies": [], "median_convention": "statistics.median — the two middle values are averaged for an even n", "n_compared": 9, "not_compared": [], "tolerance": 0.002 }, "source": "/data/cfe8760_validation.json", "source_built": null, "source_version": "2.0.0", "status": "measured" }, { "caveats": [ "Out-of-sample RMSE/MAE of forecast farm power (MW) against Elexon B1610 metered generation, IEC reference curve (before) vs site-calibrated curve (after). Curve-isolation: density pinned 1.225, single deterministic open-meteo wind member, no MOS/ensemble. Curves re-fit on a day-block holdout (every 3rd calendar day held out) and scored on those held-out days — both sides span the full wind distribution, test hours excluded from the fit, so no in-sample leakage.", "Out-of-sample RMSE/MAE of forecast farm power (MW) against ENTSO-E A73 per-unit metered generation, IEC reference curve (before) vs site-calibrated curve (after). Curve-isolation: density pinned 1.225, single deterministic open-meteo wind member, no MOS/ensemble. Curves re-fit on a day-block holdout (every 3rd calendar day held out) and scored on those held-out days — both sides span the full wind distribution, test hours excluded from the fit, so no in-sample leakage. Hours whose bidding-zone day-ahead price cleared negative are excluded from fit and scoring (suspect marks consumed from the calibration_pairs evidence store) — a negative price is a market decision to curtail, not a weather miss. Hours without price evidence are kept: only evidenced-negative hours are dropped, and each farm's curtailment_evidence_coverage reports how much of its scored window the evidence store actually covered.", "Wind farms only — solar units in the ENTSO-E unit map are excluded (power-curve calibration is a wind-speed->power fit). Hours whose bidding-zone day-ahead price cleared negative are excluded from fit and scoring (suspect='negative_da_price' marks in the calibration_pairs evidence store); hours without price evidence are kept, and each farm's curtailment_evidence_coverage reports how much of its scored window carried price evidence.", "NOT a full operational backtest: a curve-isolation proof (air density pinned, a single deterministic wind member, no MOS / ensemble) — it bounds the gain from calibrating each farm's power curve on metered data, not the operational forecast's error; the operational wind forecast is scored on the CompoundVision wind card", "7 of 74 listed GB farms are suppressed by the source and carry no before / after RMSE (source reasons: low_wind_output_corr); the fleet block and the top-farms table cover the scored farms only", "imported, not recomputed: CompoundVision's own published backtest as served at https://compoundvision.compoundingenergy.com/v1/scorecard/gb and https://compoundvision.compoundingenergy.com/v1/scorecard/eu (source stamp 2026-09-24T21:55:19.017768+00:00) — CEAtlas carries its numbers with provenance; nothing is recomputed from them, and the only figures made here are counts of the payload's own rows" ], "forecast": "CompoundVision forecast farm power (MW) from the IEC reference power curve (before) and the site-calibrated curve (after) — a curve-isolation run: air density pinned, one deterministic wind member, no MOS / ensemble", "frame": "67 GB farms scored (7 suppressed), 10,890 MW, 60-day window, day-block holdout; EU sub-block 3 farms, 1,363 MW", "group": "resource_record", "id": "cv_gb_b1610_calibration", "label": "GB wind power curves — CompoundVision calibration vs Elexon B1610", "metrics": { "eu": { "farm_count_scored": 3, "farm_count_suppressed": 0, "fleet": { "capacity_mw_scored": 1363.0, "capacity_weighted_improvement_pct": 58.21, "farms_improved": 3, "farms_regressed": 0, "mean_improvement_pct": 58.31, "median_improvement_pct": 59.39, "total_rmse_mw_reduction": 229.459 }, "window_days": 60 }, "farm_count_scored": 67, "farm_count_suppressed": 7, "fleet": { "capacity_mw_scored": 10889.7, "capacity_weighted_improvement_pct": 46.52, "farms_improved": 66, "farms_regressed": 1, "mean_improvement_pct": 43.51, "median_improvement_pct": 46.04, "total_rmse_mw_reduction": 1279.71 }, "top_farms_by_capacity": [ { "capacity_mw": 1320.0, "improvement_pct": 48.89, "name": "Hornsea 2 - Optimus and Breesea", "rmse_after_mw": 162.8643, "rmse_before_mw": 318.6518, "test_pairs": 241 }, { "capacity_mw": 1218.0, "improvement_pct": 47.24, "name": "Hornsea 1 - Heron & Njord", "rmse_after_mw": 173.4413, "rmse_before_mw": 328.7593, "test_pairs": 244 }, { "capacity_mw": 950.0, "improvement_pct": 60.98, "name": "Moray East", "rmse_after_mw": 101.9955, "rmse_before_mw": 261.4167, "test_pairs": 130 }, { "capacity_mw": 630.0, "improvement_pct": 39.22, "name": "London Array Phase 1", "rmse_after_mw": 80.3501, "rmse_before_mw": 132.2015, "test_pairs": 244 }, { "capacity_mw": 588.0, "improvement_pct": 48.99, "name": "Beatrice", "rmse_after_mw": 87.3296, "rmse_before_mw": 171.2033, "test_pairs": 238 } ], "window_days": 60 }, "realised": "Elexon B1610 settlement-metered farm generation (GB); ENTSO-E A73 per-unit metered generation (the EU sub-block)", "recomputed_from_records": false, "sample": { "actuals_source": "elexon_b1610", "capacity_mw_scored": 10889.7, "computed_at": "2026-09-24T21:55:19.017768+00:00", "curtailment_filter": "elexon_windfor_boalf", "eu": { "actuals_source": "entsoe_unit", "capacity_mw_scored": 1363.0, "computed_at": "2026-09-27T03:49:55.733745+00:00", "curtailment_filter": "negative_da_price", "farm_count_scored": 3, "farm_count_suppressed": 0, "farms_listed": 3, "method": "day_block_holdout_refit", "region": "EU", "window_days": 60 }, "farm_count_scored": 67, "farm_count_suppressed": 7, "farms_listed": 74, "method": "day_block_holdout_refit", "region": "GB", "window_days": 60 }, "source": { "fetched_at": "2026-09-29T04:32:04Z", "generated_at": "2026-09-24T21:55:19.017768+00:00", "product": "CompoundVision", "schema": 1, "urls": [ "https://compoundvision.compoundingenergy.com/v1/scorecard/gb", "https://compoundvision.compoundingenergy.com/v1/scorecard/eu" ], "window": "60 days, day-block holdout" }, "status": "imported" }, { "forecast": "CENovaSage day-ahead hub/zone LMP (the day_ahead lead, published ~12:00 UTC of D−1)", "frame": "the CONUS scorecard's own record (etl/scorecard.py, public/data/scorecard/scorecard.json), rolling 30 d, $/MWh", "group": "operational_forecast", "headline": { "as_of": "2026-09-27", "basis": "da", "fields_present": [ "before_close_share", "coverage", "end", "era", "mae_median_series", "n_artefact_bus_h", "n_pre_contract_days", "skill_p1d", "start", "system" ], "lead": "day_ahead", "lead_metric": "skill_p1d", "n_markets": 7, "n_scored": 7, "n_skill_scored": 7, "no_market": [ "frcc", "serc" ], "regions": { "ercot": { "before_close_share": 1.0, "coverage": 1.0, "end": "2026-09-27", "era": "runner", "mae_median_series": 12.603, "mae_usd_mwh": 12.463, "n_artefact_bus_h": 0, "n_days": 5, "n_pre_contract_days": 0, "n_price_days": 5, "skill_p1d": -0.948, "smape_pct": 28.555, "start": "2026-08-29", "system": "runner|hifld-ercot:71d2b7f7315d|pockets-spp-ercot+offer-f12+gapfill" }, "isone": { "before_close_share": 1.0, "coverage": 1.0, "end": "2026-09-27", "era": "runner", "mae_median_series": 8.105, "mae_usd_mwh": 8.041, "n_artefact_bus_h": 0, "n_days": 8, "n_pre_contract_days": 0, "n_price_days": 8, "skill_p1d": -0.453, "smape_pct": 20.358, "start": "2026-08-29", "system": "runner|hifld-isone:f6f4dbd01ac9|homing-isone+avail-isone+gapfill+ordc-prod" }, "miso": { "before_close_share": 1.0, "coverage": 1.0, "end": "2026-09-27", "era": "runner", "mae_median_series": 8.47, "mae_usd_mwh": 8.719, "n_artefact_bus_h": 0, "n_days": 2, "n_pre_contract_days": 0, "n_price_days": 2, "skill_p1d": -0.859, "smape_pct": 25.82, "start": "2026-08-29", "system": "runner|hifld-miso:abfae69482c8|tiesplit-miso+offer-miso-ta45+dell+gapfill+demand930+mom-level+hydro-xba+ba-sets-aeci+tiesplit-ti" }, "nyiso": { "before_close_share": 1.0, "coverage": 1.0, "end": "2026-09-27", "era": "runner", "mae_median_series": 5.786, "mae_usd_mwh": 5.424, "n_artefact_bus_h": 0, "n_days": 5, "n_pre_contract_days": 0, "n_price_days": 5, "skill_p1d": -0.961, "smape_pct": 14.67, "start": "2026-08-29", "system": "runner|hifld-nyiso:6dc173a1a6da|yards-spp-frcc-serc-nyiso+gapfill+ordc-prod" }, "pjm": { "before_close_share": 1.0, "coverage": 1.0, "end": "2026-09-27", "era": "runner", "mae_median_series": 7.821, "mae_usd_mwh": 8.935, "n_artefact_bus_h": 0, "n_days": 3, "n_pre_contract_days": 0, "n_price_days": 3, "skill_p1d": -0.618, "smape_pct": 30.845, "start": "2026-08-29", "system": "runner|hifld-pjm:edfea1d90d7d|pjm-isolf+offer-nla+gapfill+demand930+ti-fill+avail-ov+ordc-g2+offer-oc+offer-lag+ordc-shadow" }, "spp": { "before_close_share": 1.0, "coverage": 1.0, "end": "2026-09-27", "era": "runner", "mae_median_series": 11.438, "mae_usd_mwh": 11.438, "n_artefact_bus_h": 0, "n_days": 7, "n_pre_contract_days": 0, "n_price_days": 7, "skill_p1d": -0.241, "smape_pct": 36.846, "start": "2026-08-29", "system": "runner|hifld-spp:3080f1f59310|pockets-spp-ercot+seam-spp-v2c-gv2+yards-spp-frcc-serc-nyiso+gapfill+ordc-prod" }, "wecc": { "before_close_share": 1.0, "coverage": 1.0, "end": "2026-09-27", "era": "runner", "mae_median_series": 6.754, "mae_usd_mwh": 6.681, "n_artefact_bus_h": 0, "n_days": 5, "n_pre_contract_days": 0, "n_price_days": 5, "skill_p1d": -0.662, "smape_pct": 18.686, "start": "2026-08-29", "system": "runner|hifld-wecc:3de69bf5a409|wecc-combined+cf-frame" } }, "scorecard_generated_at": "2026-09-28T11:51:43Z", "skill_p1d_status": "present", "window_days": 30, "window_days_from_record": true }, "href": "/scorecard", "id": "conus_da_prices", "label": "CONUS day-ahead prices — CENovaSage vs market", "note": "Separate record, separate page; $/MWh; carried here as a pointer only — the card leads with skill vs yesterday's day-ahead price once the scorecard record carries it (a bare MAE on a few-day window flatters), and the review caveats are the scorecard session's dated findings, not recomputed here.", "realised": "the market's published day-ahead price", "review_caveats": { "as_reviewed_on": "2026-09-10", "frame_qualification": { "authored_by": "etl/run_accuracy_backtest.py", "basis": "the cited review's §1 Data basis", "clause": 1, "regions_with_runner_era_pairs": [ "ISONE", "NYISO", "MISO", "PJM", "WECC" ], "regions_without_runner_era_pairs": [ "ERCOT", "SPP" ] }, "note": "The scorecard session's dated review findings, carried verbatim — not recomputed by this job. The first clause's 'every region' means the regions its cited review had runner-era pairs for (ISONE, NYISO, MISO, PJM, WECC); ERCOT and SPP had none — this framing is added by this job from the cited review's §1 Data basis, not the reviewer's words. Once the scorecard record carries per-market skill vs yesterday's day-ahead price, that value supersedes the first clause.", "recomputed": false, "source": "docs/AWS_SOLVER_CHECKLIST.md — CONUS RUN REVIEW, step 2", "text": [ "Raw model skill vs yesterday's day-ahead price is negative in every region on the 10 runner-era days reviewed 2026-09-10 (−0.55 … −1.90)", "the local evening was not scored before 2026-09-11", "only 4 September days of the current engine are in the committed record", "day-ahead rows published after a market's gate are flagged per day on /scorecard." ] }, "source": "/data/scorecard/scorecard.json", "source_built": "2026-09-28T11:51:43Z", "status": "measured_elsewhere" }, { "caveats": [ "day-ahead-lead (D+0) forecast vintages, frozen before settlement, vs the settled Elexon Market Index (MID) outturn, per 30-minute settlement period — never re-solved history", "MID is the volume-weighted price of short-term products (half-hour to four-hour) traded within 8 hours of the settlement period's submission deadline — the reference Elexon uses in the imbalance calculation. It is not the day-ahead auction price. Because those trades happen close to delivery they price in late wind revisions, plant trips and interconnector changes that a day-ahead forecast cannot know, so this is a harder benchmark than the day-ahead auction, not an easier one. Periods whose traded volume falls below Elexon's liquidity threshold are excluded rather than graded as zero.", "Graded exclusively on forecast vintages persisted before settlement — never on re-solved history. Re-based 2026-08-28; the previous methodology graded stored rows that later re-solves could overwrite and read ~7% more favourable (MAE £16.77 vs £18.00 at day-ahead lead on the switch date).", "The window bounds the SOLVE date, matching the internal by-lead surface and the empirical confidence bands.", "Lead buckets floor ELAPSED hours (horizon_h // 24), not calendar days: a 23:00 solve forecasting 02:00 the next morning is D+0.", "Per-4h-block r² + tail recall on the same graded rows — a flat forecast can hide in headline MAE/r²; it cannot hide here.", "Model side: D+0 forecast vintages frozen before settlement. Benchmark side: the settled outturn, which no re-solve can rewrite.", "Auto-computed from the frozen forecast-vintage archive — the same rows the public attestation chain hashes at solve time. No curation, no cherry-picked windows, and no re-solved history.", "Full-physics solves run 4x daily on the 00/06/12/18 UTC weather cycles. The 06:05 UTC solve carries the overnight model run and publishes by ~08:30-08:55 UK - ahead of the N2EX GB day-ahead auction close at 09:50 UK - so the morning forecast is in hand before bids are due.", "imported, not recomputed: CEGridSight's own published backtest as served at https://gridsight.compoundingenergy.com/api/public/track-record and https://gridsight.compoundingenergy.com/api/accuracy/price-metrics?days=90&zone_code=GB-Z16 and https://gridsight.compoundingenergy.com/api/accuracy/per-zone?days=90 (source stamp 2026-09-28) — CEAtlas carries its numbers with provenance; nothing is recomputed from them, and the only figures made here are counts of the payload's own rows" ], "forecast": "CEGridSight GB national day-ahead price: the served price of each forecast vintage, frozen before settlement (D+0 is the day-ahead lead; D+1 onward further out)", "frame": "30-day headline: 5,720 settlement periods at D+0; by-lead D+0…D+13; 90-day price metrics on GB-Z16 (2026-06-30–2026-09-28, n 4,359); per-zone view over 27 zones; streak on a 90-day window (89 days available)", "group": "operational_forecast", "id": "cgs_gb_da_prices", "label": "GB day-ahead prices — CEGridSight track record vs Elexon MID", "metrics": { "baselines": { "previous_day": { "mae": 43.52, "model_mae_on_same_periods": 24.6, "n_periods": 5708, "skill_pct": 43.5 }, "seasonal_naive": { "mae": 52.41, "model_mae_on_same_periods": 24.48, "n_periods": 5708, "skill_pct": 53.3 } }, "by_lead": [ { "bias": 4.99, "lead": "D+0", "lead_day": 0, "mae": 24.57, "n_pairs": 5720, "spike_recall": 0.773 }, { "bias": 16.94, "lead": "D+1", "lead_day": 1, "mae": 34.88, "n_pairs": 5528, "spike_recall": 0.714 }, { "bias": 20.75, "lead": "D+2", "lead_day": 2, "mae": 39.55, "n_pairs": 5336, "spike_recall": 0.687 }, { "bias": 19.91, "lead": "D+3", "lead_day": 3, "mae": 41.1, "n_pairs": 5144, "spike_recall": 0.647 }, { "bias": 19.72, "lead": "D+4", "lead_day": 4, "mae": 41.27, "n_pairs": 4952, "spike_recall": 0.619 }, { "bias": 19.32, "lead": "D+5", "lead_day": 5, "mae": 42.84, "n_pairs": 4760, "spike_recall": 0.592 }, { "bias": 17.98, "lead": "D+6", "lead_day": 6, "mae": 40.43, "n_pairs": 4567, "spike_recall": 0.638 }, { "bias": 21.91, "lead": "D+7", "lead_day": 7, "mae": 43.57, "n_pairs": 4376, "spike_recall": 0.678 }, { "bias": 20.88, "lead": "D+8", "lead_day": 8, "mae": 42.2, "n_pairs": 4184, "spike_recall": 0.697 }, { "bias": 22.53, "lead": "D+9", "lead_day": 9, "mae": 43.97, "n_pairs": 3992, "spike_recall": 0.694 }, { "bias": 23.72, "lead": "D+10", "lead_day": 10, "mae": 48.4, "n_pairs": 3797, "spike_recall": 0.665 }, { "bias": 22.91, "lead": "D+11", "lead_day": 11, "mae": 47.97, "n_pairs": 3607, "spike_recall": 0.676 }, { "bias": 23.98, "lead": "D+12", "lead_day": 12, "mae": 49.25, "n_pairs": 3416, "spike_recall": 0.66 }, { "bias": 26.48, "lead": "D+13", "lead_day": 13, "mae": 48.68, "n_pairs": 1856, "spike_recall": 0.739 } ], "headline_30d": { "bias_gbp_mwh": 4.99, "mae_gbp_mwh": 24.57, "n_days": 30, "n_periods": 5720, "r2": 0.72, "sigma_ratio": 0.976, "window_days": 30 }, "metrics_90d": { "actual_std": 48.44141961500424, "bias": 0.1337188632303674, "days": 90, "forecast_std": 48.27283726962228, "low_threshold": 40.0, "mae": 20.695922066525064, "n": 4359, "neg_threshold": 0.0, "pearson_r": 0.8324468375688965, "pearson_r2": 0.6929677373784567, "per_hour_summary": { "n_hours": 24, "n_hours_r2_gt_0_1": 24, "n_hours_r2_gt_0_4": 24, "n_hours_with_r2": 24, "r2_max": 0.6800065481471594, "r2_mean": 0.6237505203481862, "r2_median": 0.6342985041259104, "r2_min": 0.4611286943370701 }, "r2_skill": 0.6660401568409873, "rmse": 27.99393734325246, "spike_threshold": 150.0, "std_ratio": 0.9965198719046263, "tails": { "low": { "actual_count": 408, "event_bias": 27.32746710466757, "event_mae": 35.7856930982056, "forecast_count": 279, "hit_rate": 0.5833333333333334, "hits": 238, "precision": 0.8530465949820788, "threshold": 40.0 }, "negative": { "actual_count": 155, "event_bias": 19.61631398336764, "event_mae": 33.38660549173724, "forecast_count": 98, "hit_rate": 0.32903225806451614, "hits": 51, "precision": 0.5204081632653061, "threshold": 0.0 }, "spike": { "actual_count": 1142, "event_bias": -4.610007973781028, "event_mae": 20.704214821619612, "forecast_count": 1205, "hit_rate": 0.7338003502626971, "hits": 838, "precision": 0.6954356846473029, "threshold": 150.0 }, "spike_175": { "actual_count": 396, "event_bias": -0.7198017897960883, "event_mae": 23.032259731175788, "forecast_count": 464, "hit_rate": 0.6515151515151515, "hits": 258, "precision": 0.5560344827586207, "threshold": 175.0 }, "spike_200": { "actual_count": 97, "event_bias": -2.456803849380705, "event_mae": 26.701703543736574, "forecast_count": 170, "hit_rate": 0.5979381443298969, "hits": 58, "precision": 0.3411764705882353, "threshold": 200.0 } } }, "per_zone_90d": { "days": 90, "gb": null, "gb_z16": { "bias": 0.13, "days_covered": 91, "first_date": "2026-06-30", "last_date": "2026-09-28", "mae_price": 20.7, "n_pairs": 4359, "r_squared": 0.693, "sigma_ratio": 0.997, "zone": "GB-Z16" }, "n_zones": 27, "n_zones_with_data": 27 }, "skill_detail": { "blocks_r2": [ { "hours": "00-04", "n": 943, "r2": 0.734 }, { "hours": "04-08", "n": 947, "r2": 0.681 }, { "hours": "08-12", "n": 952, "r2": 0.692 }, { "hours": "12-16", "n": 959, "r2": 0.601 }, { "hours": "16-20", "n": 963, "r2": 0.57 }, { "hours": "20-24", "n": 956, "r2": 0.621 } ], "n_negative_periods": 278, "n_spikes_gt150": 2725, "negative_recall": 0.36, "spike_recall_gt150": 0.77 }, "streak": { "best_90d": 6, "current": 4, "days_beaten_90d": 46, "definition": "Consecutive complete days whose D+0 forecast vintages — frozen before settlement, passthrough periods excluded — beat BOTH the seasonal-naive (same period 7 days earlier) and previous-day benchmarks on that day's mean absolute error, all three scored on the same settlement periods against the settled outturn. A complete day with fewer than 40 of 48 comparable periods ends the streak as unmeasured rather than being skipped over.", "last_day": "2026-09-28", "min_periods_per_day": 40, "n_insufficient": 0, "n_pending": 0, "window_days": 90, "window_days_available": 89 } }, "realised": "the settled Elexon Market Index (MID) outturn per settlement period — not the day-ahead auction price (the source calls MID the harder benchmark)", "recomputed_from_records": false, "sample": { "days_covered_90d": 91, "definitions": { "previous_day": "Actual price at the same settlement period yesterday.", "seasonal_naive": "Actual price at the same settlement period 7 days earlier.", "streak": "Consecutive complete days whose D+0 forecast vintages — frozen before settlement, passthrough periods excluded — beat BOTH the seasonal-naive (same period 7 days earlier) and previous-day benchmarks on that day's mean absolute error, all three scored on the same settlement periods against the settled outturn. A complete day with fewer than 40 of 48 comparable periods ends the streak as unmeasured rather than being skipped over." }, "n_daily_rows": 30, "n_days_30d": 30, "n_leads": 14, "n_pairs_90d": 4359, "n_periods_30d": 5720, "n_zones_90d": 27, "shadow_column": "served_price", "streak_last_day": "2026-09-28", "window_90d": { "end": "2026-09-28", "start": "2026-06-30" }, "zone": "GB national (single-price market)", "zone_code": "GB-Z16" }, "source": { "fetched_at": "2026-09-29T04:32:04Z", "generated_at": "2026-09-28", "generated_at_basis": "the track record's last graded day — the track record carries no build stamp; all three payload dates agree", "grading": "vintage", "headline_lead": "D+0", "product": "CEGridSight", "schema": null, "stamps": { "price_metrics_window_end": "2026-09-28", "streak_last_day": "2026-09-28", "track_record_last_day": "2026-09-28" }, "urls": [ "https://gridsight.compoundingenergy.com/api/public/track-record", "https://gridsight.compoundingenergy.com/api/accuracy/price-metrics?days=90&zone_code=GB-Z16", "https://gridsight.compoundingenergy.com/api/accuracy/per-zone?days=90" ], "window": "30 d headline (track record) · 90 d price metrics · 90 d per-zone · 90 d streak" }, "status": "imported" }, { "caveats": [ "skill_score is vs same-horizon persistence; >0 beats the naive forecast.", "nmae = median capacity-normalised MAE across scored regions (fraction).", "Scored on the OPERATING fleet — the same basis /v1/forecast serves by default.", "p10–p90 band coverage is the source's top-level coverage figure — the share of actuals inside its p10–p90 band per the CompoundVision contract (the payload states no definition or target); a well-calibrated p10–p90 band covers 80% of actuals, and that target is carried beside the value so the page never types it", "1 of 35 regions with a wind block (FR) is marked by the source as a fleet-basis mismatch — its modelled fleet and the actuals feed cover different fleets — so their error figures are withheld here on the source's own verdict and they are not scored; the source's reason is carried on each region's row", "2 of 35 regions with a wind block carry no skill score this run — the source's own reason for each is carried on the region's row", "imported, not recomputed: CompoundVision's own published backtest as served at https://compoundvision.compoundingenergy.com/v1/accuracy (source stamp 2026-09-29T04:31:50.833363+00:00) — CEAtlas carries its numbers with provenance; nothing is recomputed from them, and the only figures made here are counts of the payload's own rows and the fleet totals (farms, MW) summed over them" ], "forecast": "CompoundVision per-region wind generation forecast (hourly, out to the source's maximum horizon), scored on the OPERATING fleet — the basis its /v1/forecast serves by default — against same-horizon persistence", "frame": "35 of 38 regions carry a wind block (26 EU, 9 US); 33 scored, 24 beating persistence; rolling 14 d, up to 48 h ahead; operating fleet — 12,231 farms, 461.3 GW", "group": "operational_forecast", "id": "cv_wind_forecast", "label": "Wind generation forecast — CompoundVision rolling backtest", "metrics": { "band_coverage_frac": 0.7611, "band_coverage_target_frac": 0.8, "gb": { "basis_suppressed": false, "beats_persistence": true, "bias_frac": 0.0274, "capacity_mw": 30991.400000000023, "computed_at": "2026-09-29T03:37:32.316180+00:00", "farms": 569, "n_samples": 212, "nmae_frac": 0.0463, "rmse_frac": 0.0606, "skill_reason": null, "skill_score": 0.5989 }, "lookback_days": 14, "max_hours_ahead": 48, "median_skill_score": 0.4083, "nmae_frac": 0.0658, "regions": [ { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1399, "region": "AT", "skill_reason": null, "skill_score": 0.4102 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.037, "region": "BE", "skill_reason": null, "skill_score": 0.6259 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 204, "nmae_frac": 0.1107, "region": "BG", "skill_reason": null, "skill_score": 0.3062 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 334, "nmae_frac": 0.3496, "region": "CAISO", "skill_reason": null, "skill_score": -0.4567 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1783, "region": "CH", "skill_reason": null, "skill_score": -0.4632 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.4048, "region": "CZ", "skill_reason": null, "skill_score": -0.4239 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0658, "region": "DE", "skill_reason": null, "skill_score": 0.3159 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.074, "region": "DK", "skill_reason": null, "skill_score": 0.6714 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1117, "region": "EE", "skill_reason": null, "skill_score": 0.2217 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 333, "nmae_frac": 0.0451, "region": "ERCOT", "skill_reason": null, "skill_score": 0.5223 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0297, "region": "ES", "skill_reason": null, "skill_score": 0.5236 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 182, "nmae_frac": 0.05, "region": "FI", "skill_reason": null, "skill_score": 0.6403 }, { "basis_suppressed": true, "beats_persistence": null, "coverage_ratio": 4.522, "n_samples": 211, "nmae_frac": null, "region": "FR", "skill_reason": "fleet_basis_mismatch: modelled 47,056 MW vs actuals peak 10,407 MW (ratio 4.5) — the actuals feed under-reports this fleet", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 212, "nmae_frac": 0.0463, "region": "GB", "skill_reason": null, "skill_score": 0.5989 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 167, "nmae_frac": 0.1196, "region": "GR", "skill_reason": null, "skill_score": -0.2864 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.2888, "region": "HR", "skill_reason": null, "skill_score": -0.5086 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0594, "region": "HU", "skill_reason": null, "skill_score": 0.5427 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 193, "nmae_frac": 0.0848, "region": "IE", "skill_reason": null, "skill_score": 0.2858 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 307, "nmae_frac": 0.4787, "region": "ISONE", "skill_reason": null, "skill_score": -0.6601 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0915, "region": "IT", "skill_reason": null, "skill_score": 0.1606 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1497, "region": "LT", "skill_reason": null, "skill_score": 0.2807 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1261, "region": "LU", "skill_reason": null, "skill_score": -0.0126 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.3276, "region": "LV", "skill_reason": null, "skill_score": 0.2717 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 333, "nmae_frac": 0.0482, "region": "MISO", "skill_reason": null, "skill_score": 0.4508 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1329, "region": "NL", "skill_reason": null, "skill_score": -2.04 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0601, "region": "NO", "skill_reason": null, "skill_score": 0.4891 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 333, "nmae_frac": 0.041, "region": "NYISO", "skill_reason": null, "skill_score": 0.5339 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 307, "nmae_frac": 0.047, "region": "PJM", "skill_reason": null, "skill_score": 0.615 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0483, "region": "PL", "skill_reason": null, "skill_score": 0.6816 }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1102, "region": "PT", "skill_reason": null, "skill_score": -0.1032 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0413, "region": "RO", "skill_reason": null, "skill_score": 0.4825 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0336, "region": "SE", "skill_reason": null, "skill_score": 0.7361 }, { "basis_suppressed": false, "beats_persistence": null, "coverage_ratio": null, "n_samples": 308, "nmae_frac": 0.0037, "region": "SERC", "skill_reason": "persistence_rmse_below_threshold", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 333, "nmae_frac": 0.0501, "region": "SPP", "skill_reason": null, "skill_score": 0.5035 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 305, "nmae_frac": 0.0343, "region": "WECC", "skill_reason": null, "skill_score": 0.4083 } ], "regions_beating_persistence": 24, "regions_scored": 33 }, "realised": "each region's actual wind generation as CompoundVision's own backtest holds it (the scorecard payload does not name the actuals feed per region); the baseline is same-horizon persistence of those actuals", "recomputed_from_records": false, "sample": { "capacity_mw_total": 461300.974, "families": { "eu": 26, "us": 9 }, "farms_total": 12231, "regions_basis_suppressed": 1, "regions_beating_persistence": 24, "regions_scored": 33, "regions_skill_suppressed": 2, "regions_total": 38, "regions_with_block": 35, "registry_spec_regions": [ "DE", "DK", "GB", "SE" ], "skill_suppressed_reasons": { "FR": "fleet_basis_mismatch: modelled 47,056 MW vs actuals peak 10,407 MW (ratio 4.5) — the actuals feed under-reports this fleet", "SERC": "persistence_rmse_below_threshold" } }, "source": { "fetched_at": "2026-09-29T04:32:04Z", "generated_at": "2026-09-29T04:31:50.833363+00:00", "product": "CompoundVision", "schema": "scorecard-v1", "urls": [ "https://compoundvision.compoundingenergy.com/v1/accuracy" ], "window": "rolling 14 days" }, "status": "imported" }, { "caveats": [ "skill_score is vs same-horizon persistence; >0 beats the naive forecast.", "nmae = median capacity-normalised MAE across scored regions (fraction).", "Scored on the OPERATING fleet — the same basis /v1/forecast serves by default.", "p10–p90 band coverage is the source's top-level coverage figure — the share of actuals inside its p10–p90 band per the CompoundVision contract (the payload states no definition or target); a well-calibrated p10–p90 band covers 80% of actuals, and that target is carried beside the value so the page never types it", "6 of 38 regions with a solar block (AT, BE, CH, LU, NL, SI) are marked by the source as a fleet-basis mismatch — its modelled fleet and the actuals feed cover different fleets — so their error figures are withheld here on the source's own verdict and they are not scored; the source's reason is carried on each region's row", "8 of 38 regions with a solar block carry no skill score this run — the source's own reason for each is carried on the region's row", "imported, not recomputed: CompoundVision's own published backtest as served at https://compoundvision.compoundingenergy.com/v1/accuracy (source stamp 2026-09-29T04:31:50.833363+00:00) — CEAtlas carries its numbers with provenance; nothing is recomputed from them, and the only figures made here are counts of the payload's own rows and the fleet totals (farms, MW) summed over them" ], "forecast": "CompoundVision per-region solar generation forecast (hourly, out to the source's maximum horizon), scored on the OPERATING fleet — the basis its /v1/forecast serves by default — against same-horizon persistence", "frame": "38 of 38 regions carry a solar block (28 EU, 10 US); 30 scored, 29 beating persistence; rolling 14 d, up to 48 h ahead; operating fleet — 10,551 farms, 300.7 GW", "group": "operational_forecast", "id": "cv_solar_forecast", "label": "Solar generation forecast — CompoundVision rolling backtest", "metrics": { "band_coverage_frac": 0.4112, "band_coverage_target_frac": 0.8, "gb": { "basis_suppressed": false, "beats_persistence": true, "bias_frac": -0.033, "capacity_mw": 12738.000000000013, "computed_at": "2026-09-29T03:37:32.316180+00:00", "farms": 954, "n_samples": 212, "nmae_frac": 0.0415, "rmse_frac": 0.0821, "skill_reason": null, "skill_score": 0.7716 }, "lookback_days": 14, "max_hours_ahead": 48, "median_skill_score": 0.6616, "nmae_frac": 0.119, "regions": [ { "basis_suppressed": true, "beats_persistence": null, "coverage_ratio": 0.14, "n_samples": 211, "nmae_frac": null, "region": "AT", "skill_reason": "fleet_basis_mismatch: modelled 610 MW vs actuals peak 4,362 MW (ratio 0.14) — the actuals cover a much larger fleet than modelled", "skill_score": null }, { "basis_suppressed": true, "beats_persistence": null, "coverage_ratio": 0.076, "n_samples": 211, "nmae_frac": null, "region": "BE", "skill_reason": "fleet_basis_mismatch: modelled 578 MW vs actuals peak 7,613 MW (ratio 0.08) — the actuals cover a much larger fleet than modelled", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 204, "nmae_frac": 0.1738, "region": "BG", "skill_reason": null, "skill_score": 0.6441 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 334, "nmae_frac": 0.0505, "region": "CAISO", "skill_reason": null, "skill_score": 0.8585 }, { "basis_suppressed": true, "beats_persistence": null, "coverage_ratio": 0.004, "n_samples": 211, "nmae_frac": null, "region": "CH", "skill_reason": "fleet_basis_mismatch: modelled 14 MW vs actuals peak 3,298 MW (ratio 0.00) — the actuals cover a much larger fleet than modelled", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.2356, "region": "CZ", "skill_reason": null, "skill_score": 0.5752 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.3661, "region": "DE", "skill_reason": null, "skill_score": 0.4352 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0724, "region": "DK", "skill_reason": null, "skill_score": 0.71 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0938, "region": "EE", "skill_reason": null, "skill_score": 0.1251 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 333, "nmae_frac": 0.0351, "region": "ERCOT", "skill_reason": null, "skill_score": 0.8844 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0831, "region": "ES", "skill_reason": null, "skill_score": 0.6791 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.2027, "region": "FI", "skill_reason": null, "skill_score": 0.5381 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0711, "region": "FR", "skill_reason": null, "skill_score": 0.7553 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 303, "nmae_frac": 0.0566, "region": "FRCC", "skill_reason": null, "skill_score": 0.7363 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 212, "nmae_frac": 0.0415, "region": "GB", "skill_reason": null, "skill_score": 0.7716 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 167, "nmae_frac": 0.0481, "region": "GR", "skill_reason": null, "skill_score": 0.8123 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.318, "region": "HR", "skill_reason": null, "skill_score": 0.5944 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1421, "region": "HU", "skill_reason": null, "skill_score": 0.7051 }, { "basis_suppressed": false, "beats_persistence": null, "coverage_ratio": null, "n_samples": 203, "nmae_frac": 0.1923, "region": "IE", "skill_reason": "persistence_rmse_below_threshold", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 307, "nmae_frac": 0.0845, "region": "ISONE", "skill_reason": null, "skill_score": 0.4705 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.4719, "region": "IT", "skill_reason": null, "skill_score": 0.4993 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.22, "region": "LT", "skill_reason": null, "skill_score": 0.4775 }, { "basis_suppressed": true, "beats_persistence": null, "coverage_ratio": 0.207, "n_samples": 211, "nmae_frac": null, "region": "LU", "skill_reason": "fleet_basis_mismatch: modelled 66 MW vs actuals peak 320 MW (ratio 0.21) — the actuals cover a much larger fleet than modelled", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.1043, "region": "LV", "skill_reason": null, "skill_score": 0.6859 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 333, "nmae_frac": 0.0268, "region": "MISO", "skill_reason": null, "skill_score": 0.8559 }, { "basis_suppressed": true, "beats_persistence": null, "coverage_ratio": 29.331, "n_samples": 211, "nmae_frac": null, "region": "NL", "skill_reason": "fleet_basis_mismatch: modelled 6,442 MW vs actuals peak 220 MW (ratio 29.3) — the actuals feed under-reports this fleet", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": false, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0914, "region": "NO", "skill_reason": null, "skill_score": -0.5178 }, { "basis_suppressed": false, "beats_persistence": null, "coverage_ratio": null, "n_samples": 307, "nmae_frac": 0.2387, "region": "NYISO", "skill_reason": "persistence_rmse_below_threshold", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 307, "nmae_frac": 0.0266, "region": "PJM", "skill_reason": null, "skill_score": 0.8593 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.2547, "region": "PL", "skill_reason": null, "skill_score": 0.544 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0665, "region": "PT", "skill_reason": null, "skill_score": 0.7122 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.0569, "region": "RO", "skill_reason": null, "skill_score": 0.7768 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 211, "nmae_frac": 0.4232, "region": "SE", "skill_reason": null, "skill_score": 0.4005 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 308, "nmae_frac": 0.0982, "region": "SERC", "skill_reason": null, "skill_score": 0.5146 }, { "basis_suppressed": true, "beats_persistence": null, "coverage_ratio": 0.008, "n_samples": 211, "nmae_frac": null, "region": "SI", "skill_reason": "fleet_basis_mismatch: modelled 8 MW vs actuals peak 990 MW (ratio 0.01) — the actuals cover a much larger fleet than modelled", "skill_score": null }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 209, "nmae_frac": 0.0583, "region": "SK", "skill_reason": null, "skill_score": 0.805 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 333, "nmae_frac": 0.1336, "region": "SPP", "skill_reason": null, "skill_score": 0.5797 }, { "basis_suppressed": false, "beats_persistence": true, "coverage_ratio": null, "n_samples": 299, "nmae_frac": 0.0794, "region": "WECC", "skill_reason": null, "skill_score": 0.6332 } ], "regions_beating_persistence": 29, "regions_scored": 30 }, "realised": "each region's actual solar generation as CompoundVision's own backtest holds it (the scorecard payload does not name the actuals feed per region); the baseline is same-horizon persistence of those actuals", "recomputed_from_records": false, "sample": { "capacity_mw_total": 300739.95, "families": { "eu": 28, "us": 10 }, "farms_total": 10551, "regions_basis_suppressed": 6, "regions_beating_persistence": 29, "regions_scored": 30, "regions_skill_suppressed": 8, "regions_total": 38, "regions_with_block": 38, "registry_spec_regions": [ "DE", "DK", "GB", "SE" ], "skill_suppressed_reasons": { "AT": "fleet_basis_mismatch: modelled 610 MW vs actuals peak 4,362 MW (ratio 0.14) — the actuals cover a much larger fleet than modelled", "BE": "fleet_basis_mismatch: modelled 578 MW vs actuals peak 7,613 MW (ratio 0.08) — the actuals cover a much larger fleet than modelled", "CH": "fleet_basis_mismatch: modelled 14 MW vs actuals peak 3,298 MW (ratio 0.00) — the actuals cover a much larger fleet than modelled", "IE": "persistence_rmse_below_threshold", "LU": "fleet_basis_mismatch: modelled 66 MW vs actuals peak 320 MW (ratio 0.21) — the actuals cover a much larger fleet than modelled", "NL": "fleet_basis_mismatch: modelled 6,442 MW vs actuals peak 220 MW (ratio 29.3) — the actuals feed under-reports this fleet", "NYISO": "persistence_rmse_below_threshold", "SI": "fleet_basis_mismatch: modelled 8 MW vs actuals peak 990 MW (ratio 0.01) — the actuals cover a much larger fleet than modelled" } }, "source": { "fetched_at": "2026-09-29T04:32:04Z", "generated_at": "2026-09-29T04:31:50.833363+00:00", "product": "CompoundVision", "schema": "scorecard-v1", "urls": [ "https://compoundvision.compoundingenergy.com/v1/accuracy" ], "window": "rolling 14 days" }, "status": "imported" } ], "generated_at": "2026-09-29T04:31:50Z", "imports_run": { "fetched_at": "2026-09-29T04:32:04Z", "mode": "live", "source_fetched_at": { "cgs_per_zone": "2026-09-29T04:32:04Z", "cgs_price_metrics": "2026-09-29T04:31:54Z", "cgs_track_record": "2026-09-29T04:31:51Z", "cv_accuracy": "2026-09-29T04:31:50Z", "cv_scorecard_eu": "2026-09-29T04:31:50Z", "cv_scorecard_gb": "2026-09-29T04:31:50Z" }, "sources": { "cgs_per_zone": "ok", "cgs_price_metrics": "ok", "cgs_track_record": "ok", "cv_accuracy": "ok", "cv_scorecard_eu": "ok", "cv_scorecard_gb": "ok" } }, "notes": [ "Every number under a measured check was recomputed this run from the per-record rows (plants[] / sites[]) of the committed validation artefacts under public/data/ with stdlib statistics, except the values each check names in metrics_from_summary; nothing is typed in. Recomputed from records: eia923_wind_us, gb_b1610_wind, merra2_wind_reanalysis, pvgis_solar.", "Values taken from an artefact's SUMMARY / meta rather than recomputed from its per-record rows (flagged per check as metrics_from_summary; a sample. prefix names a sample count, span or frame threshold rather than a metric — the rows verify a threshold where they can, and the check's caveat says whether they agree): eia923_wind_us: fleet_annual_r, fleet_annual_n_years, worst_year_measured, worst_year_modeled, sample.years, sample.min_cap_mw, sample.in_service_before, sample.n_tt_coverage_excluded; gb_b1610_wind: sample.months_span; merra2_wind_reanalysis: sample.speed_track_years, sample.cf_track_years.", "window is assembled from the recomputed samples (the EIA-923 metered years, the MERRA-2 speed-track span, the PVGIS year, the CONUS headline's window_days); a check not measured this run reads 'not measured' in its segment.", "Rounding (the figures this job computes — the measured checks and the CONUS headline): ratio_* and *_share* 4 dp, cap_gw 1 dp, everything else 3 dp; an imported check's figures are carried as the source serialised them, never re-rounded; sorted keys; the same inputs and --as-of reproduce the file byte for byte.", "artefact_summary carries the artefact's own headline value beside each recomputed metric (copied verbatim, for a self-check); self_check lists any pair that disagrees by more than 0.002, compared in the artefact's own median convention (self_check.median_convention).", "Medians: a published *_median averages the two middle values for an even n (statistics.median); the MERRA-2 and GB artefacts' summaries take the upper middle value (sorted[n // 2]) instead, so those checks also publish *_median_upper — the artefact's own figure, and the one the methodology page and BENCHMARKS.md quote — and self-check against it like-for-like.", "A not_measured check carries no metrics — its reason says why the repo cannot compute it and its needs says what would make it computable.", "conus_da_prices.review_caveats carries the scorecard session's dated review findings (as reviewed on 2026-09-10; docs/AWS_SOLVER_CHECKLIST.md — CONUS RUN REVIEW, step 2) verbatim — its numbers are NOT recomputed by this job and its sentence is not re-worded; review_caveats.frame_qualification (authored_by etl/run_accuracy_backtest.py, from the cited review's §1 Data basis) names the regions the first clause's 'every region' spans (ISONE, NYISO, MISO, PJM, WECC; ERCOT and SPP had no runner-era pairs); the CONUS headline leads with skill_p1d (1 − MAE_model / MAE_persistence, read from scorecard.json), which supersedes the first clause once the record carries it — skill_p1d_status this run: present; optional window fields the record carries: before_close_share, coverage, end, era, mae_median_series, n_artefact_bus_h, n_pre_contract_days, skill_p1d, start, system.", "The imported checks (cv_gb_b1610_calibration, cgs_gb_da_prices, cv_wind_forecast, cv_solar_forecast) carry each source product's OWN published backtest with provenance — the product, the URLs read, the source's own stamp and the time each payload was fetched — and are never recomputed by this job; their metrics keep the source's units, each key named with its unit (percent or fraction), every value carried exactly as the source serialised it (never re-rounded here — the page rounds once, for display), and the only figures made here are counts of the payload's own rows and, on the two CompoundVision forecast cards, the fleet totals (farms, MW) summed over its region rows — that MW sum being the one imported-card figure rounded. This run: fetched live, payloads fetched by 2026-09-29T04:32:04Z; imported: cv_gb_b1610_calibration, cgs_gb_da_prices, cv_wind_forecast, cv_solar_forecast. The /api/accuracy handler overlays the server's newer hourly snapshot per check when it has one.", "The /api/accuracy handler appends a staleness line when generated_at is more than 30 days old." ], "producer": "etl/run_accuracy_backtest.py", "schema_version": 2, "updated": "2026-09-29", "version": "accuracy-backtest v2", "window": "metered 2019–2024 · reanalysis 1981–2024 · PVGIS 2020 · CONUS prices rolling 30 d" }