Skip to content

fix(train): drop selection_metric from --model-config before model construction - #326

Merged
jayhesselberth merged 1 commit into
mainfrom
fix/324-selection-metric-model-config
Sep 15, 2026
Merged

jayhesselberth merged 1 commit into
mainfrom
fix/324-selection-metric-model-config

Conversation

@jayhesselberth

Copy link
Copy Markdown
Member

Summary

  • leech model optimize writes selection_metric into best_params.json for provenance, but handle_train had no consumer for that key — it fell through extra_kwargs into get_model()'s constructor kwargs and raised TypeError: model got unexpected keyword argument(s): selection_metric, breaking the documented optimize -> train --model-config best_params.json workflow.
  • Add "selection_metric" to the _explicit_keys set in src/leech/commands/train.py::handle_train, matching the existing checkpoint_metric precedent — a metric-selection value that is pure bookkeeping/provenance, not an architecture param.
  • best_params.json still records selection_metric after this change; only --model-config consumption strips it before get_model() is called.

Closes #324

Test plan

  • uv run pytest tests/test_gridsearch.py -v — 35 passed, including two new regression tests (TestBestParamsJsonRoundTrip):
    • test_best_params_json_records_selection_metric — confirms best_params.json still contains selection_metric (provenance not regressed)
    • test_best_params_json_does_not_crash_train — confirms handle_train(..., model_config=best_params.json) no longer raises TypeError
  • uv run pytest tests/test_config_propagation.py -v — 31 passed (existing _explicit_keys/handle_train coverage, no regression)
  • uv run ruff format --check src/leech/commands/train.py tests/test_gridsearch.py — clean
  • uv run ruff check src/leech/commands/train.py tests/test_gridsearch.py — clean
  • uv run ty check src/leech/ — clean

🤖 Generated with Claude Code

…nstruction

leech model optimize writes selection_metric into best_params.json for
provenance, but handle_train had no consumer for that key -- it fell
through extra_kwargs into get_model()'s constructor kwargs and raised
TypeError, breaking the documented optimize -> train --model-config
workflow. Add it to _explicit_keys alongside checkpoint_metric, the
existing precedent for exactly this kind of bookkeeping-only key.

Refs #324
@jayhesselberth
jayhesselberth merged commit a96079d into main Sep 15, 2026
3 checks passed
@jayhesselberth
jayhesselberth deleted the fix/324-selection-metric-model-config branch September 15, 2026 13:41
@jayhesselberth jayhesselberth mentioned this pull request Sep 15, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: optimize's best_params.json includes selection_metric, which train --model-config rejects

1 participant