Skip to content

Keep files past PATH_MAX as path-table rows - #1656

Open
thekevinbot wants to merge 8 commits into
mainfrom
bot/1582-path-max-1104
Open

thekevinbot wants to merge 8 commits into
mainfrom
bot/1582-path-max-1104

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1582

A path table walking './**' silently dropped every file whose absolute path is longer than PATH_MAX: the directory listing opened each directory by its full path and treated ENAMETOOLONG as an empty directory.

The fix (packages/rust/src/long_path.rs): on Unix, a path the kernel refuses as too long is opened again a stretch of components at a time, each stretch openat-ed relative to the directory the previous one reached. The listing, the stat columns, symlink/unknown-type classification in the scanner, and content reads all go through it, so deep files keep size, mtime and content. Windows needs nothing: std already passes long paths in \\?\ form, and the red test passed on Windows.

Red run (test-only commit f90ef39, Integration tests fails with left: Integer(20) right: Integer(30)): https://github.com/thekevinscott/dirsql/actions/runs/41949428945/job/126989572880

By hand: the issue's repro now prints [{"n":30,"longest":6039}]; the deepest row reports size 1, a non-NULL mtime, and content "x".

Merging main (#1585's open_regular) resolved a conflict in vtab.rs: open_regular now opens through long_path::open_file, which opens O_NONBLOCK so the FIFO guard still holds. The non-unix open_file body is exempt from mutation in testing-conventions.toml, as it is compiled out on the Linux runner.

Not covered here: declared [[table]] freshness stats (RealFs in lib.rs) and a .gitignore sitting past the limit still use plain std paths.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass (impacted subset)
  • TypeScript SDK e2e: pass
  • Rust core e2e (if applicable): pass
  • packages/python/e2e-attestations/<branch>.json receipt written if packages/python changed (just e2e-attest-python)
  • packages/ts/e2e-attestations/<branch>.json receipt written if packages/ts changed (just e2e-attest-ts)
  • Command(s) run:
    • cd packages/python && testing-conventions e2e attest 'uv run python -m pytest tests/e2e/glob_speed_recursive_test.py tests/e2e/glob_speed_symlink_test.py tests/e2e/stat_speed_test.py tests/e2e/query_contract_test.py tests/e2e/root_at_construction_test.py -x -q'
    • cd packages/ts && testing-conventions e2e attest 'pnpm test:e2e'
    • cargo test -p dirsql --features cli --test path_table_cli_e2e --test path_table_gitignore_cli_e2e --test path_max_depth_e2e --test path_table_symlinks --test path_table_globs --test path_table_query --test scanner --test scan_failures
  • Result summary: Python 17 passed; TypeScript 27 passed; Rust 144 passed.

Changelog / Migrations

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason)
  • Migration fragment with the same slug added under <root>/<pkg>/migrations.d/ for each changed package (every subsection but Summary _None._ when nothing breaks)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stop silently dropping files whose absolute path exceeds PATH_MAX

1 participant