test: complete v3 row lineage test coverage - #869
Open
wgtmac wants to merge 1 commit into
Open
Conversation
wgtmac
commented
Aug 2, 2026
wgtmac
commented
Member
- enable RewriteFiles V3 coverage and verify DV cleanup
- cover V2-to-V3 assignment and incremental/changelog lineage
- verify delete-aware readers preserve lineage
- enable RewriteFiles V3 coverage and verify DV cleanup - cover V2-to-V3 assignment and incremental/changelog lineage - verify delete-aware readers preserve lineage
There was a problem hiding this comment.
Pull request overview
This PR expands the C++ test suite to fully cover Iceberg format v3 “row lineage” behavior, including how row IDs and last-updated sequence numbers flow through rewrites, upgrades, incremental scans, and delete-aware readers.
Changes:
- Extend
RewriteFilestests to run against format v3 and add DV-/row-id-specific assertions (including DV cleanup when removing a data file). - Add format v2→v3 upgrade tests verifying when existing rows remain unassigned vs. when row IDs become assigned on the next commit.
- Add incremental scan planning and file reader tests that validate row lineage fields are preserved/propagated correctly across added/deleted files and delete types (pos deletes, DVs, equality deletes).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/iceberg/test/rewrite_files_test.cc | Adds/adjusts rewrite tests for v3, including DV semantics, sequence-number assertions, DV cleanup, and expands instantiated format versions to include 3. |
| src/iceberg/test/merging_snapshot_update_test.cc | Adds v2→v3 upgrade coverage and validates row lineage assignment behavior across snapshots/manifests after upgrade. |
| src/iceberg/test/incremental_changelog_scan_test.cc | Adds v3-specific incremental changelog planning tests verifying first_row_id and data sequence numbers for added/deleted tasks. |
| src/iceberg/test/incremental_append_scan_test.cc | Adds v3-specific incremental append planning test verifying per-file row lineage propagation into planned tasks. |
| src/iceberg/test/file_scan_task_reader_test.cc | Adds row-lineage projection helper and new reader tests ensuring delete-aware reads preserve row lineage metadata columns. |
Comment on lines
+87
to
99
| std::shared_ptr<DataFile> MakePositionDeleteFile( | ||
| const std::string& path, int64_t partition_x, | ||
| const std::string& referenced_data_file) { | ||
| auto file = MakeDataFile(path, partition_x); | ||
| file->content = DataFile::Content::kPositionDeletes; | ||
| if (table_->metadata()->format_version >= 3) { | ||
| file->file_format = FileFormatType::kPuffin; | ||
| file->referenced_data_file = referenced_data_file; | ||
| file->content_offset = 0; | ||
| file->content_size_in_bytes = 10; | ||
| } | ||
| return file; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.