Skip to content

Add diagnostic windgust (support for multi-datasets) - #178

Open
radiradev wants to merge 10 commits into
mainfrom
feat/diagnostic-decoders
Open

radiradev wants to merge 10 commits into
mainfrom
feat/diagnostic-decoders

Conversation

@radiradev

@radiradev radiradev commented Jun 9, 2026 •

Copy link
Copy Markdown

Support running the multi-decoder with a VMAX_10M on REAL-CH1:

had to pin anemoi-inference (radiradev/anemoi-inference@fix/empty-input-propagate-date) with two fixes. (still unsure if the padding one is needed).

Also unsure what is the best way here to deal with the templates and their patches #129

image

radiradev added 2 commits June 9, 2026 13:39
Support running the multi-output "realv2" anemoi architecture (ICON-CH1 cutout
forecaster with a diagnostic VMAX_10M stream on the REAL-CH1 / ICON-CH1 1km grid)
on top of current main:

- config/windgust.yaml: showcase + experiment config. extra_requirements pins an
  anemoi-inference fork branch (radiradev/anemoi-inference@fix/empty-input-propagate-date)
  carrying the two fixes this checkpoint needs (EmptyInput date propagation +
  format_dataset_name output-path padding), so no runtime source patching is needed.
  The showcase section drives the VMAX_10M animation over the Alpine (icon-ch /
  switzerland) domains via the configurable-showcase mechanism.
- resources/inference: realv2 inference config, metadata patch, VMAX_10M GRIB
  template + templates index, and a VMAX_10M template generation step.
- src/plotting: VMAX_10M colormap; redirect VMAX_10M loads to the sibling
  realv2-*.grib output file.
- workflow/rules/plot.smk: treat VMAX_10M (period maximum) like accumulations and
  skip lead time 0.
- docs/realv2_vmax10m.md: notes on the realv2 showcase.

Verified: `evalml showcase` and `evalml experiment` both build a complete DAG
against this config (--dry-run).
@radiradev radiradev changed the title Add realv2 (diagnostic VMAX_10M) multi-output forecaster showcase Add diagnostic windgust showcase Jun 9, 2026
@radiradev radiradev changed the title Add diagnostic windgust showcase Add diagnostic windgust (support for multi-datasets) Jun 10, 2026
@radiradev
radiradev marked this pull request as ready for review June 10, 2026 15:49
- assign_mask: # fill local/global overlapping points with nan
mask: "global/cutout_mask"
# Dianostic decoder on ICON grid only
realv2:

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

realv2 should be called something else like secondary_decoder

# Dianostic decoder on ICON grid only
realv2:
grib:
path: grib/realv2-{date}{time:04}_{step:03}.grib

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here too

radiradev and others added 5 commits September 22, 2026 17:57
Implement CAPE_MU diagnostic decoder following the VMAX_10M (windgust)
pattern: inference config, metadata patch, GRIB template (paramId=500151,
typeOfLevel=atmMU), and experiment config for the bounded CAPE checkpoint.

Data loader improvements:
- Auto-discover diagnostic GRIB stream prefixes instead of hardcoding
- Support atmMU level type in variable_name_profile

Inference rule fixes:
- Truncate Python version to major.minor (uv lacks some patch releases)
- Tolerate crash-on-exit if expected GRIB output was produced
- Verify expected step count before marking inference as successful
…ence robustness)

- compat.py: replace removed REALV2_PARAMS with diagnostic stream
  auto-discovery; skip global IFS merge for params without IFS mapping
- common.smk: handle aggregated param names (e.g. TOT_PREC1) in
  PARAMS_WITHOUT_STEP_ZERO_VALUE; add CAPE_MU to step-zero exclusions
- colormap_defaults.py: add CAPE_MU colormap for showcase animations
- plot.smk: increase plot_scoremaps runtime to 30m
…case

- Decompose aggregated params (e.g. TOT_PREC6 → TOT_PREC + accu=6)
  so GRIB loading uses the base param while output filenames keep the
  original name for Snakemake
- Convert precipitation from meters (IFS convention) to mm in
  preprocess_field
- Update disaggregation comment to clarify anemoi-inference writes
  period accumulations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants