Backtest — 2026-08-21 to 2026-10-15
Forecast input: Open-Meteo single-runs archive — HRRR and NBM runs initialised 18Z (13:00 CDT) the afternoon before (about 8 h older than what the 21:30 run sees, so this is conservative), ECMWF and ICON runs initialised 12Z the day before; the live scorer runs unchanged on them (HRRR visibility and cloud levels included). Observed: IEM ASOS visibility at GYY, IKK, JOT, LOT, PNT, RZL. Event = min visibility 04:00–09:00 local ≤ 1.0 mi. Rain in the prior 48 h from the same ASOS records.
- station-mornings: 329, fog events: 34 on 13 distinct mornings (10% base rate; stations on one morning are correlated, so the effective sample is nearer the morning count)
- model coverage (share of cases each model scored): gfs_hrrr 100%, ncep_nbm_conus 100%, ecmwf_ifs025 100%, icon_seamless 100%
- AUC (ranking skill, 0.5 = none): 0.761 (in-sample)
- Brier score, in-sample: raw blend 0.258, calibrated 0.084, climatology baseline 0.093 (lower is better)
- Held-out-station Brier (fit on the other stations, score the held-out one), pooled: 0.085 — this is the honest number
- Platt calibration: p = sigmoid(5.51·raw -5.38)
Contingency (calibrated probability thresholds, in-sample)
| threshold | hits | misses | false alarms | correct neg | POD | FAR | CSI |
|---|
| 0.3 | 6 | 28 | 5 | 290 | 0.18 | 0.45 | 0.15 |
| 0.4 | 1 | 33 | 0 | 295 | 0.03 | 0.0 | 0.03 |
| 0.5 | 0 | 34 | 0 | 295 | 0.0 | 0.0 | 0.0 |
| 0.6 | 0 | 34 | 0 | 295 | 0.0 | 0.0 | 0.0 |
Reliability (calibrated, in-sample)
| forecast bin | n | observed fog rate |
|---|
| 0.0–0.1 | 191 | 0.04 |
| 0.1–0.2 | 81 | 0.16 |
| 0.2–0.3 | 46 | 0.15 |
| 0.3–0.4 | 10 | 0.5 |
| 0.4–0.5 | 1 | 1.0 |
Leave-one-station-out
| held-out station | n | events | Brier | AUC |
|---|
| GYY | 49 | 5 | 0.08 | 0.7818181818181819 |
| IKK | 56 | 6 | 0.083 | 0.81 |
| JOT | 56 | 10 | 0.14 | 0.7945652173913044 |
| LOT | 56 | 5 | 0.085 | 0.5509803921568628 |
| PNT | 56 | 3 | 0.058 | 0.7358490566037735 |
| RZL | 56 | 5 | 0.066 | 0.8960784313725491 |
Every fog morning (observed ≤ 1 mi) and what the score said
| date | station | obs min vis (mi) | raw | calibrated p | HRRR dpd_min | HRRR wind kt | HRRR vis min km | type called |
|---|
| 2026-08-21 | IKK | 0.25 | 0.75 | 0.23 | 2.5 | 1.8 | 15.2 | shallow ground fog |
| 2026-08-21 | JOT | 0.5 | 0.71 | 0.19 | 2.9 | 1.8 | 17.2 | shallow ground fog |
| 2026-08-21 | PNT | 0.25 | 0.83 | 0.31 | 1.5 | 3.6 | 13.3 | shallow ground fog |
| 2026-08-21 | RZL | 0.25 | 0.83 | 0.30 | 1.1 | 3.6 | 12.2 | shallow ground fog |
| 2026-08-24 | JOT | 0.75 | 0.69 | 0.17 | 2.5 | 2.2 | 15.3 | shallow ground fog |
| 2026-08-25 | IKK | 0.75 | 0.72 | 0.20 | 1.8 | 2.2 | 13.6 | shallow ground fog |
| 2026-08-25 | JOT | 1.0 | 0.53 | 0.08 | 4.4 | 1.9 | 20.5 | shallow ground fog |
| 2026-09-10 | JOT | 0.25 | 0.63 | 0.13 | 0.9 | 2.0 | 12.2 | shallow ground fog |
| 2026-09-10 | LOT | 0.25 | 0.69 | 0.17 | 0.3 | 1.7 | 0.0 | deep fog bank |
| 2026-09-10 | RZL | 0.25 | 0.74 | 0.21 | 1.0 | 4.4 | 12.2 | patchy fog, deepening in low ground |
| 2026-09-14 | JOT | 1.0 | 0.65 | 0.14 | 2.6 | 3.9 | 16.1 | shallow ground fog |
| 2026-09-17 | GYY | 0.25 | 0.44 | 0.05 | 0.5 | 2.3 | 12.2 | deep fog bank |
| 2026-09-17 | IKK | 1.0 | 0.52 | 0.07 | 0.7 | 3.2 | 12.2 | deep fog bank |
| 2026-09-17 | JOT | 0.75 | 0.44 | 0.05 | 0.5 | 3.0 | 6.1 | patchy fog, deepening in low ground |
| 2026-09-17 | LOT | 0.5 | 0.42 | 0.04 | 0.8 | 3.1 | 12.3 | deep fog bank |
| 2026-09-17 | PNT | 0.5 | 0.58 | 0.10 | 0.8 | 3.9 | 12.2 | patchy fog, deepening in low ground |
| 2026-09-18 | RZL | 0.75 | 0.68 | 0.16 | 1.2 | 6.4 | 12.7 | shallow ground fog |
| 2026-09-27 | GYY | 0.25 | 0.72 | 0.20 | 1.9 | 2.4 | 12.2 | shallow ground fog |
| 2026-09-27 | IKK | 0.25 | 0.76 | 0.23 | 1.9 | 3.4 | 13.9 | shallow ground fog |
| 2026-09-27 | JOT | 0.25 | 0.76 | 0.23 | 1.2 | 4.0 | 12.2 | shallow ground fog |
| 2026-09-28 | GYY | 0.75 | 0.67 | 0.15 | 1.9 | 4.9 | 12.7 | shallow ground fog |
| 2026-09-28 | IKK | 0.25 | 0.72 | 0.20 | 1.4 | 3.3 | 12.2 | shallow ground fog |
| 2026-09-28 | JOT | 0.25 | 0.68 | 0.16 | 1.7 | 4.2 | 13.2 | shallow ground fog |
| 2026-09-28 | PNT | 0.25 | 0.67 | 0.15 | 2.3 | 3.4 | 14.9 | shallow ground fog |
| 2026-09-28 | RZL | 0.25 | 0.79 | 0.26 | 1.2 | 4.8 | 12.2 | shallow ground fog |
| 2026-09-29 | GYY | 0.25 | 0.87 | 0.35 | 0.0 | 1.7 | 1.1 | deep fog bank |
| 2026-09-29 | IKK | 0.25 | 0.85 | 0.33 | 1.1 | 3.2 | 12.2 | shallow ground fog |
| 2026-09-29 | JOT | 0.25 | 0.82 | 0.30 | 2.1 | 2.7 | 13.9 | shallow ground fog |
| 2026-09-29 | LOT | 0.25 | 0.85 | 0.33 | 1.2 | 1.8 | 12.2 | shallow ground fog |
| 2026-09-29 | RZL | 0.25 | 0.97 | 0.49 | 0.6 | 2.4 | 0.1 | shallow ground fog |
| 2026-09-30 | LOT | 1.0 | 0.09 | 0.01 | 6.6 | 5.0 | 27.4 | shallow ground fog |
| 2026-10-03 | JOT | 0.5 | 0.75 | 0.22 | 1.6 | 3.5 | 13.7 | shallow ground fog |
| 2026-10-14 | GYY | 0.25 | 0.51 | 0.07 | 0.9 | 8.1 | 12.7 | shallow ground fog |
| 2026-10-14 | LOT | 1.0 | 0.51 | 0.07 | 1.0 | 6.9 | 12.4 | shallow ground fog |
Worst false alarms (calibrated p ≥ 0.5, observed > 1 mi)
| date | station | obs min vis (mi) | calibrated p | HRRR dpd_min | HRRR wind kt |
|---|
Skill by lead time (global models only: ECMWF, ICON, GFS 12Z runs; no HRRR, no calibration)
| lead (days before the morning) | station-mornings | fog events | AUC | fog rate when set-up ≥ 0.75 | 0.60–0.75 | 0.45–0.60 | < 0.45 |
|---|
| 1 | 313 | 32 | 0.746 | 26% of 38 | 16% of 70 | 11% of 54 | 3% of 151 |
| 2 | 314 | 32 | 0.726 | 17% of 30 | 19% of 63 | 14% of 70 | 3% of 151 |
| 3 | 314 | 32 | 0.738 | 19% of 36 | 21% of 66 | 10% of 71 | 3% of 141 |
| 5 | 313 | 32 | 0.627 | 0% of 21 | 20% of 59 | 12% of 76 | 7% of 157 |
dashboard