Repository navigation
feat: classification default metrics — add log-loss and PR AUC - #67
Merged
Merged
Conversation
- Add neg_log_loss to all default classification metric lists and average_precision to the binary list, keeping roc_auc first so ranking behavior is unchanged - Gate proba-requiring metrics (neg_log_loss, average_precision) on the configured estimators supporting predict_proba: custom estimators without predict_proba (e.g. LinearSVC) keep a working default metric set instead of hard-failing during cross-validation - Factor estimator resolution out of _build_pipelines into _build_estimators_dict() so metrics can inspect the configured estimators - Update the multilabel shape test to derive the column count from the metrics Closes P0-4 of the ML defaults improvements plan.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements P0-4 from the ML defaults improvements plan (
docs/ml-defaults-improvements.md).Changes
neg_log_lossadded to all three default classification metric lists (binary, multiclass, multilabel-indicator).average_precisionadded to the binary list (PR AUC is informative under class imbalance).roc_auc/roc_auc_ovrstays first so ranking/ensemble behavior is unchanged.predict_proba. Custom estimators without it (e.g.LinearSVC) keep a working default metric set instead of hard-failing in cross-validation (which the existing error-analysis test relied on)._build_pipelinesinto_build_estimators_dict()so_build_metricscan inspect the configured estimators.Tests
neg_log_lossandaverage_precision; multiclass/multilabel includeneg_log_lossonly.neg_log_loss).All 248 tests pass; coverage 89.4% (gate 85%); ruff + format clean.