Skip to content

feat(sheet): preserve spreadsheet provenance - #163

Open
789987777777 wants to merge 9 commits into
firecrawl:mainfrom
789987777777:feat/sheet-provenance
Open

feat(sheet): preserve spreadsheet provenance#163
789987777777 wants to merge 9 commits into
firecrawl:mainfrom
789987777777:feat/sheet-provenance

Conversation

@789987777777

@789987777777 789987777777 commented Sep 3, 2026

Copy link
Copy Markdown

Why

to_document() currently turns spreadsheet tables into the shared document model without retaining worksheet identity or source coordinates. That prevents downstream consumers from tracing a table or cell back to the original workbook, especially after hidden rows/columns and merged cells are normalized.

What

  • Add public SpreadsheetCoordinate, SpreadsheetRange, and SpreadsheetSource model types.
  • Add optional source metadata to Table and spreadsheet origin Cell values.
  • Preserve workbook-order sheet indices, sheet names, sparse coordinates, hidden-row/column source positions, merged-cell rectangles, and ODS repeated-row/column cursors across XLSX/XLSM, XLSB, XLS, and ODS.
  • Expose the metadata through the Node, Python, and WebAssembly bindings.
  • Document the contract and add format-specific, binding, and compatibility tests.

Coordinates are zero-based and ranges are inclusive. Non-spreadsheet tables and cells keep None/undefined source metadata.

Verification

  • cargo test -p anydoc --all-targets --quiet: 303 passed, 1 ignored, 0 failed.
  • cargo clippy -p anydoc --all-targets -- -D warnings: passed.
  • cargo fmt --all -- --check: passed.
  • cargo check --workspace --target x86_64-pc-windows-msvc --quiet: blocked in this Windows environment because link.exe is unavailable; core Rust and Python checks pass below.
  • cargo check -p anydoc-python --quiet: passed.
  • rustup run stable-x86_64-pc-windows-gnu cargo check -p anydoc-wasm --target wasm32-unknown-unknown --quiet: passed.
  • Independent public corpus check: 917 spreadsheet files across all five extensions; 865 parsed documents, 1,254 tables, and 3,952,561 origin cells, with 100% table and cell source coverage and no coordinate invariant violations.
  • Official WRMP24 XLSB, Invest NI XLSX, and Invest NI ODS samples converted successfully and their source sheet names/order and coordinate bounds were independently checked against the files.

Risk / notes

  • Table and Cell are public Rust structs, so downstream struct literals must account for the new optional source fields.
  • Native Node/Python binding runtime tests were not runnable in this Windows environment because libnode.dll and the MSVC linker are unavailable; the binding Rust code and WASM target compile checks pass, and the JS/Python test sources pass syntax checks.
  • During broad validation, poc-shared-strings.xlsx parsed successfully but its Markdown conversion grew to roughly 15 GB before being stopped. This occurs in the Markdown conversion path and was not attributed to this provenance change; it should be tracked separately.

No internal or non-public CSR files are included in this branch or repository.


Summary by cubic

to_document() now preserves worksheet identity and source coordinates for spreadsheet files, so tables and cells can be traced back to their original workbook positions. Table and Cell carry optional source metadata for XLSX/XLSM, XLSB, XLS, and ODS; non-spreadsheet tables and cells keep None/undefined. Merged origins keep their complete source rectangle even when hidden rows or columns collapse the visible span or the merge ends in covered cells.

Migration

  • Table and Cell are public Rust structs; downstream struct literals must include source: None.

Written for commit d252d16. Summary will update on new commits.

Review in cubic

Closes #10

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 25 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread src/formats/sheet/xlsx.rs
Comment thread src/formats/odf/table.rs
Comment thread AGENTS.md Outdated
Comment thread python/tests/test_anydoc.py Outdated
@789987777777

Copy link
Copy Markdown
Author

Review follow-up:

  • 838746b keeps the original XLSX merge rectangle when hidden-row/column normalization collapses it to one visible slot, with a regression test for A1:B2.
  • ccc14cb keeps ODS covered tail rows for spreadsheet tables so the materialized grid, clamped span, and source range stay consistent, with a trailing-merge regression test.
  • 663b101 replaces raw StopIteration fixture lookups in the Python binding test with clear assertions.
  • d252d16 removes the out-of-scope AGENTS.md from this PR and corrects the plan's branch reference.

Verification: cargo test -p anydoc --all-targets --quiet — 303 passed, 1 ignored; Clippy, rustfmt, Python syntax, Python binding check, WASM target check, and public official XLSB/XLSX/ODS conversions pass. Native Node/workspace checks remain limited by this Windows host's missing libnode.dll and link.exe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose worksheet identity and source coordinates in to_document()

1 participant