Skip to content

feat: regression default metrics — RMSE first, MAPE only for positive targets - #66

Merged
rxavier merged 1 commit into
mainfrom
feat/preprocessing-p0-regression-metrics
Aug 3, 2026
Merged

rxavier merged 1 commit into
mainfrom
feat/preprocessing-p0-regression-metrics

Conversation

@rxavier

@rxavier rxavier commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

Implements P0-3 from the ML defaults improvements plan (docs/ml-defaults-improvements.md).

Changes

  • Default regression metrics become ["neg_root_mean_squared_error", "neg_mean_absolute_error", "r2"].
    • RMSE is now the primary metric (first column), used as the ranking/ensemble sort key.
    • neg_mean_squared_error was outlier-dominated; neg_median_absolute_error dropped.
  • neg_mean_absolute_percentage_error is only included when the target is strictly positive. MAPE epsilon-floors the denominator in sklearn, silently producing astronomical values when y contains zeros.
  • get_target_info() now reports whether a regression target is positive (positive key).
  • Bumped scikit-learn floor to >=1.4.0 (required for neg_root_mean_squared_error).

Tests

  • Defaults: RMSE first, no MAPE on a mixed-sign target.
  • Strictly positive target includes MAPE.
  • Zero-containing target scores cleanly (no MAPE column, no NaNs).
  • Updated the two shape tests that hardcoded the old 4-metric count.

All 244 tests pass; ruff + format clean.

… targets

- Default metrics become [neg_root_mean_squared_error, neg_mean_absolute_error, r2]
  (RMSE is the primary sort key; neg_mean_squared_error was outlier-dominated)
- neg_mean_absolute_percentage_error is included only when the target is
  strictly positive, avoiding astronomical values on zero-containing targets
  (MAPE epsilon-floors the denominator)
- get_target_info() now reports whether a regression target is positive
- Bump scikit-learn floor to >=1.4.0 for neg_root_mean_squared_error

Closes P0-3 of the ML defaults improvements plan.
@rxavier
rxavier merged commit c3101a9 into main Aug 3, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant