Skip to content

Add STL short-horizon forecast regression test - #13

Merged
StatMixedML merged 1 commit into
StatMixedML:mainfrom
janrth:feat/regression_test
Jul 20, 2026
Merged

Add STL short-horizon forecast regression test#13
StatMixedML merged 1 commit into
StatMixedML:mainfrom
janrth:feat/regression_test

Conversation

@janrth

@janrth janrth commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an end-to-end regression test for short-horizon HyperTreeSTL forecasts with fcst_h=3 and fcst_h=1.

The test trains a small real STL model and verifies that forecasting returns the expected number of rows with finite forecast values. This preserves coverage for the short-horizon crash path while leaving the implementation itself to the conformal branch rework that already fixed the underlying STL behavior.

@StatMixedML

Copy link
Copy Markdown
Owner

@claude Please review this PR

@claude

claude Bot commented Jul 20, 2026

Copy link
Copy Markdown

Claude finished @StatMixedML's task in 57s —— View job


PR Review: Add STL short-horizon forecast regression test

  • Gather context and read changed files
  • Review code quality and correctness
  • Provide feedback

Verdict: Approve — Clean, focused regression test. A few minor observations below but nothing blocking.

What it does

Adds a parametrized test (fcst_h=3 and fcst_h=1) to TestSTLForecastContinuation that trains a small HyperTreeSTL model on 48 observations and asserts:

  1. The forecast DataFrame has exactly fcst_h rows.
  2. All forecast values are finite (no NaN/Inf from the short-horizon code path).

Strengths

  • Focused scope — exercises the exact crash path (short horizon < period) without over-fitting to internal implementation details.
  • Good parameterizationfcst_h=1 is the tightest edge case; fcst_h=3 covers the small-but-not-minimal case.
  • Reuses existing helperself._series(n=48, fcst_h=fcst_h, seed=fcst_h) keeps the test self-contained and deterministic.
  • Lightweight modelnum_iterations=5, num_leaves=4 keeps CI fast.

Minor observations (non-blocking)

  1. seed=fcst_h — Using the forecast horizon as the random seed is a bit unusual. It works (produces different but deterministic data per parametrize case), but a fixed seed like seed=42 with an explicit comment would be slightly more readable. This is purely stylistic.

  2. No assertion on column presence — The test checks len(fcst) and fcst["fcst"], which implicitly validates the column exists. If you ever want to guard against schema drift you could add assert "fcst" in fcst.columns, but this is very optional given the existing assertions already fail on a missing column.

  3. Potential future extension — If the conformal branch rework lands and type="paper" diverges further from type="default", consider parametrizing over both variants here as well (similar to test_conformal_intervals below). Not needed now.

Overall this is a well-written, minimal regression test that directly covers the reported short-horizon crash. LGTM.


@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.17%. Comparing base (cb29603) to head (7001341).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #13   +/-   ##
=======================================
  Coverage   95.17%   95.17%           
=======================================
  Files          15       15           
  Lines        3462     3462           
  Branches      623      623           
=======================================
  Hits         3295     3295           
  Misses         78       78           
  Partials       89       89           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@StatMixedML
StatMixedML merged commit 2dc5eb7 into StatMixedML:main Jul 20, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants