Skip to content

Add precision, recall and AUROC scorers; fix nondeterministic weight fitting - #71

Merged
gAldeia merged 3 commits into
masterfrom
feature/classification_metrics
Sep 22, 2026
Merged

gAldeia merged 3 commits into
masterfrom
feature/classification_metrics

Conversation

@gAldeia

@gAldeia gAldeia commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

New classification scorers

  • precision, recall and roc_auc for binary and multiclass problems. For multiclass, precision and recall are macro averages, and AUROC is a macro one-vs-rest average.
  • average_precision_score now also works for multiclass (macro one-vs-rest).
  • The new scorers only drive selection, survival and the archive. Weights are still fit with log loss, and splits with gini impurity.
  • Class weights are applied as sample weights, matching sklearn. Values agree with sklearn to within 1e-4.
  • The scorer can be changed between partial_fit calls. partial_fit now also updates parameters_, so the new scorer is used by final model selection.

Fix: Floor/Ceil broke weight optimization

These nodes were flagged as weighted but left out of get_weights(). During fitting, evaluation still took a weight for them. That shifted every later weight to the wrong node and read past the end of the parameter array. As a result, fits depended on uninitialized memory, and runs with the same seed could differ. The fix is that only weighable node types take a weight during evaluation. test_fixed_seed_produces_identical_brush_runs now passes consistently.

Docs and tests

  • C++ tests for the new metrics, plus a regression test for the weight bug.
  • Python tests comparing brush's metrics with sklearn's, and covering scorer changes in partial_fit.

@gAldeia

gAldeia commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator Author

Looks good. Thanks

@gAldeia gAldeia closed this Sep 22, 2026
@gAldeia gAldeia reopened this Sep 22, 2026
@gAldeia
gAldeia merged commit 45f17d0 into master Sep 22, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant