Resolve companion ranges across filename case differences - #19
Open
imnasnainaec wants to merge 11 commits into
Open
Resolve companion ranges across filename case differences#19imnasnainaec wants to merge 11 commits into
imnasnainaec wants to merge 11 commits into
Conversation
imnasnainaec
force-pushed
the
ranges-companion-filename-case
branch
2 times, most recently
from
August 14, 2026 18:40
dededeb to
cd9cb67
Compare
A LIFT folder written on Windows can spell its pair inconsistently — Dict.LIFT beside Dict.lift-ranges, or the reverse — and load fine there, because the filesystem folds case. On Linux the sibling candidate is built from the .lift's own suffix, so it missed, the companion was skipped without a word, and every range it defined went absent. Candidates that match no file exactly now fall back to one whose name differs only in case. The fallback is reached only after an exact miss, so a case-folding filesystem never enters it and behaves as before; a case-sensitive one gets one directory read per folder, cached across the candidate list. Where several names fold together the lexicographically first wins. The choice is arbitrary but fixed, which matters more than which file it picks: directory order varies between filesystems and runs, and a companion that loads differently on consecutive reads would be worse than one that never loads. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
0.1.0 has not shipped, so there is no released behavior for an Unreleased entry to be fixing — the tolerance is simply part of what companion discovery does in the first release. Fold it into that bullet. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The directory listing the fallback builds called its files "entries", the word this module uses for a LIFT <entry> everywhere else — the same collision that keeps byte regions from being called spans. Name them files. Spell the surrounding prose the way the rest of the package does: a fallback that runs rather than fires, a name that matched no file rather than missed, a helper named for the filesystem it probes rather than abbreviating it, and fixture names deliberately not taken from the corpus file. Unpack the two densest clauses so each reads in one pass. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
imnasnainaec
force-pushed
the
ranges-companion-filename-case
branch
from
August 18, 2026 20:23
cd9cb67 to
a565d67
Compare
Companion lookup folds names with casefold() over NFC rather than lower(), so a Turkish-cased or NFD-spelled name resolves the way it does on the filesystem that wrote it — FLEx mixes normalization forms within one export. Ties break in code point order on every platform; sorting Path objects left the choice to directory order on Windows, where PurePath ordering is itself case-folded. An unstattable exact spelling now falls through to the folded lookup instead of giving up. A candidate that folds onto the .lift itself is skipped: RangesFile.load rejects a <lift> root, so a header href naming the lexicon in another case took the whole load down. One that folds onto a companion already tracked is skipped too — Path.resolve() leaves case alone on macOS, so a single file reached under two spellings was loaded and tracked twice, and written twice by save(). The sibling candidate is built with with_name, which agrees with with_suffix on every name that has an extension and does not raise on a name without one. Nothing upstream requires the .lift extension: parse_document never inspects it. dangling-ranges-href decides existence with that same lookup, so a companion spelled in another case is no longer reported missing on a case-sensitive filesystem. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The 0.1.0 entry said companion names fold on case alone; they fold on Unicode normalization form as well. The folder guide listed the candidates tried but never mentioned the folding at all. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The three helpers each stated a rule, defended it, then disclaimed it. What is left is the reasoning the code cannot show: casefold over lower, NFC, why only the final component folds, why code point order, and what the fold pre-check protects the inode comparison from. The folder guide drops the folding sentence outright — it describes behavior no reader acts on, in a paragraph otherwise about which candidate wins. The 0.1.0 entry keeps the fact and loses the justification, which now lives only in the docstrings. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…tion A candidate that exists as a directory now stops the lookup instead of falling through to the folded listing. An href of "" normalizes to the LIFT folder itself and one of "sub/" to a subfolder, and folding a folder's own name searches its *parent*: any file there spelled like the folder was returned as the companion, and RangesFile.load then rejected its root and failed the whole load. dangling-ranges-href also treats a match that is the .lift itself as no match. _resolve_ranges refuses to take the lexicon for its own companion, so a header href folding onto it resolves to nothing that supplies the range — the reference is as dangling as a missing file, and went unreported on a case-sensitive filesystem. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
_same_file compared the spellings it was handed, so a caller had to canonicalize first to get a true answer. _resolve_ranges did; the dangling-ranges-href check did not, and an href reaching the .lift through a ".." segment or a symlink read there as some other file. The loader skipped that candidate as self-referential while validation counted it as a companion that exists, leaving the header range both unsupplied and unreported. Resolving inside _same_file makes the answer independent of how the caller spelled its arguments, and retires the loader's own pre-resolution — the duplicate that let the two drift apart. A path that will not resolve now compares false instead of falling back to the spelling as given; it would fail the samefile stat on the next line regardless. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
_fold cited FLEx's mixed NFC/NFD content as its reason to normalize filenames. That is a different code path and no evidence for the names; decomposed filenames arrive from macOS, so say that instead. _existing_file described itself against the behavior it replaced, which reads oddly once nothing remembers that behavior. Lexicon.load's candidate list and its matching rules split into separate paragraphs, and "every one that exists is loaded" becomes "every distinct file among them" — candidates resolving to the lexicon, to a directory, or to a file already tracked all exist and are deliberately skipped. The dangling-ranges-href comment leads with what the check catches rather than closing with it, and loses a restatement of _existing_file's own docstring. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Docstrings and comments only, plus one rename. _existing_file claimed a case-folding filesystem never reaches the fallback. That is false on NTFS, which folds case but not normalization: an NFD companion misses the exact stat and only the fallback finds it. The guarantee that does hold everywhere — an exact hit is returned unchanged — leads instead. Its summary named the argument rather than the return value, and its motivation read as though LIFT folders can only be written on Windows, when what matters is whether the authoring filesystem folds case, as macOS also does. The with_name comment justified a choice against with_suffix rather than warning about it. Naming the hazard is what stops someone reaching for the tidier call and reintroducing the raise it avoids. _case_sensitive_filesystem returned a bool under a noun phrase that promises a filesystem; _same_file and _same_dir are the house pattern for a predicate. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
imnasnainaec
marked this pull request as ready for review
August 20, 2026 12:41
"Stable" names a specific property in sorting — preserving the relative order of equal elements — which is close enough to what is meant here to be read as a claim about the sort rather than about the outcome. The choice among fold-equal names is deterministic: same folder, same winner, on every platform and every run. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A LIFT folder written on Windows can spell its pair inconsistently —
Dict.LIFTbesideDict.lift-ranges, or the reverse — and load fine there, because the filesystem folds case. On Linux the sibling candidate is built from the.lift's own name (_model.py:536), so it missed, the companion was skipped silently, and every range it defined went absent.Zip packages inherit the fix:
load_zipextracts and callsLexicon.load, so_zip.pyneeds no change.Approach
A candidate that matches no file exactly now falls back to one whose name folds onto it.
casefold()over NFC, notlower(). Decomposed filenames arrive from macOS, which stores and zips them that way, and neither NTFS nor ext4 folds the two forms.Along the way the sibling candidate switched to
with_name, which unlikewith_suffixdoesn't raise on a document loaded under a name with no extension.Three things the fallback must not resolve to
Each became reachable once a miss could match something other than the exact name:
.liftitself. A header href spellingDict.liftbeside aDict.LIFTfolds onto the lexicon, andRangesFile.loadrejects a<lift>root — failing the whole load rather than skipping one companion.""normalizes to the LIFT folder itself. Folding a folder's own name searches its parent, so any file up there spelled like the folder came back as the companion. A candidate that exists as a directory now stops the lookup.Path.resolve()leaves case alone on macOS, so one file reached under two spellings was loaded twice and written twice bysave().Validation and loading agree on what exists
dangling-ranges-hrefdecides existence with the same lookup and the same refusals, so a companion spelled in another case is no longer reported missing on a case-sensitive filesystem, and an href resolving to the.liftstill warns._same_fileresolves its own arguments rather than trusting the caller's spelling, so the two sides cannot disagree about a path spelled two ways.The tie-break
Where several names fold together, the first in code point order wins (
Dict.LIFT-rangesahead ofDict.lift-ranges). Arbitrary, but deterministic — directory order varies between filesystems and between runs, and a companion that resolved differently on consecutive reads would be worse than one that never resolved.Tests
Ten in
test_ranges_folder.py. Two need both spellings of a name to coexist, so they skip off a case-sensitive filesystem; the NFC/NFD one is what exercises the fallback on the Windows legs, where case alone never reaches it.python scripts/check.pygreen: 546 passed, 2 skipped, 97.34% coverage.Media hrefs have the same case problem and are deliberately untouched here — #34 carries the design question that makes them different from companions.
Draft while the tie-break gets a read; happy to swap it for another rule, so long as it stays deterministic.
🤖 Generated with Claude Code
Devin review: https://app.devin.ai/review/sillsdev/python-sil-lift/pull/19
This change is