Skip to content

fix(infra): rank checkpoint names without crashing on stepless or .pt files - #1007

Open
AUTHENSOR wants to merge 1 commit into
goodfire-ai:mainfrom
AUTHENSOR:fix/checkpoint-name-parsing
Open

AUTHENSOR wants to merge 1 commit into
goodfire-ai:mainfrom
AUTHENSOR:fix/checkpoint-name-parsing

Conversation

@AUTHENSOR

Copy link
Copy Markdown

Summary

fetch_latest_checkpoint_name now ranks checkpoint filenames instead of crashing on them. The old sort key was int(x.split(".pth")[0].split("_")[-1]). It raised ValueError: invalid literal for int() with base 10: 'model' on the docstring's own second supported format, a bare <name>.pth sharing the list with numbered checkpoints. It raised again on every .pt file, because the W&B caller's own (".pth", ".pt") filter feeds it those (int("100.pt")). Restoring a run whose artifacts include either shape crashes resolve_run_files.

Ranking rules, applied after the unchanged prefix filter:

  • the trailing _<int> in the stem (.pth or .pt) is that file's step
  • a non-integer trailing segment (model.pth, model_final.pth) ranks as stepless (-1), so any numbered checkpoint outranks it
  • an all-stepless set resolves lexicographically. Deterministic where the old code raised.
  • a lone file is still returned as-is

Tests

param_decomp/tests/infra/test_fetch_latest_checkpoint_name.py adds 12 cases: max-step and numeric-vs-lexicographic ordering, the bare-name-among-numbered crash, .pt suffixes alone and mixed with .pth, word suffixes, all-stepless resolution, the single-file short-circuit, and the prefix filter plus its no-match error. Full infra suite: 52 passed. ruff check, ruff format --check, and basedpyright are clean on the touched files.

… files

fetch_latest_checkpoint_name promised <name>.pth support but raised
ValueError on int('model') whenever a stepless name shared the list with
numbered checkpoints, and again on .pt files (int('100.pt')) that the
W&B caller's own (".pth", ".pt") filter feeds it. Rank stepless names
as -1 so numbered checkpoints win, resolve all-stepless sets
lexicographically, and parse both suffixes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants