Skip to content

Repair missing blob references during manual compaction (#15538) - #15538

Closed
mikechuangmeta wants to merge 1 commit into
meta:mainfrom
mikechuangmeta:export-D128836898
Closed

mikechuangmeta wants to merge 1 commit into
meta:mainfrom
mikechuangmeta:export-D128836898

Conversation

@mikechuangmeta

@mikechuangmeta mikechuangmeta commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary:

What changed?

  • Adds CompactRangeOptions::skip_missing_blob_files, default false, for manual repair when referenced blob files are absent from the compaction's input Version.

https://pxl.cl/j8V47

  • Rewrites selected SSTs. Lost scalar values become deletions. Wide entities keep surviving columns.
  • Manual CompactRange only. Runs locally; compaction services are bypassed. FIFO is unsupported.

Why?

Best efforts recovery can leave dangling blob references in SSTs. These can cause later compactions to fail. Offline repair prepares the DB for normal reopening.

Reviewed By: xingbowang

Differential Revision: D128836898

@meta-cla meta-cla Bot added the CLA Signed label Nov 1, 2026
@meta-codesync

meta-codesync Bot commented Nov 1, 2026

Copy link
Copy Markdown

@mikechuangmeta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D128836898.

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

⚠️ clang-tidy: 3 warning(s) on changed lines

Completed in 929.1s.

Summary by check

Check Count
bugprone-argument-comment 2
performance-no-automatic-move 1
Total 3

Details

db/compaction/compaction_iterator.cc (1 warning(s))
db/compaction/compaction_iterator.cc:2169:14: warning: constness of 's' prevents automatic move [performance-no-automatic-move]
db/version_builder_test.cc (2 warning(s))
db/version_builder_test.cc:1395:32: warning: argument name 'file_number' in comment does not match parameter name 'file' [bugprone-argument-comment]
db/version_builder_test.cc:1421:35: warning: argument name 'file_number' in comment does not match parameter name 'file' [bugprone-argument-comment]

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Claude Code Review - OBSOLETE

Superseded by a newer AI review. Expand to see the original review.

✅ Claude Code Review

Auto-triggered after CI passed — reviewing commit 2b6e18b


Summary

Well-structured feature with thorough test coverage. The decorator iterator approach is architecturally clean and follows existing patterns. The version builder change and the UDT seqno-zeroing fix are independent concerns bundled in.

High-severity findings (1):

  • [compaction_picker.cc:862-872] The inputs_shrunk swap reordering is a pre-existing bug fix that changes behavior for kForceOptimized/kIfHaveCompactionFilter bottommost compactions -- should be called out separately in the PR description.
Full review (click to expand)

Findings

🔴 HIGH

H1. CompactionPicker inputs_shrunk swap reordering is an independent bug fix — compaction_picker.cc:862-872
  • Issue: The diff moves inputs.files.swap(inputs_shrunk) from before to after the covering_the_whole_range loop. Before this fix, the loop at line 868 iterated over inputs which had already been swapped to the shrunk set, so inputs.size() was the shrunk size and skip_input_index referred to indices in the original (now lost) array. This meant the loop was a no-op (skip_input_index >= inputs_shrunk.size() always), so covering_the_whole_range was never set to false when it should have been.
  • Root cause: The original code swapped inputs.files before iterating inputs[skip_input_index+1, inputs.size()), but skip_input_index was computed on the pre-swap array. After the swap, inputs.size() becomes inputs_shrunk.size() which is always <= skip_input_index, making the loop a no-op.
  • Impact: Correctness fix for kForceOptimized and kIfHaveCompactionFilter bottommost compactions. With the old code, covering_the_whole_range could remain true when it should be false, causing the compaction to not schedule additional passes for files beyond the gap.
  • Suggested fix: This is correctly fixed but should be documented as a separate bug fix in the PR description and release notes, not bundled silently.

🟡 MEDIUM

M1. Version builder SaveBlobFilesTo change affects all version builds — version_builder.cc:1621-1682
  • Issue: Replacing GetMinOldestBlobFileNumber() (linked-SST-based scan of blob metadata) with a scan of all table files across all levels changes blob file retention behavior for every version build. The old approach relied on blob file back-links (LinkedSsts); the new approach uses table file forward-references (oldest_blob_file_number).
  • Root cause: After best-efforts recovery, an SST may reference a blob file via oldest_blob_file_number even though the blob file's LinkedSsts is empty (because the linking SST was removed). The old logic would skip these blob files; the new logic retains them until no table references remain.
  • Impact: For normal operation the two approaches produce identical results. After best-efforts recovery, the new approach correctly retains blob files that were previously (incorrectly) dropped. This is a behavioral improvement but should be documented separately.
  • Suggested fix: Already has a version_builder test and a release note (blob_timestamp_history.md). Consider whether the release note should also mention this retention behavior change.
M2. OutputHasExternalBlobReferences re-parses blob data in PrepareOutput — compaction_iterator.cc:1908-1938
  • Issue: In the UDT seqno-zeroing path, OutputHasExternalBlobReferences() re-parses the blob index or wide column entity even though these may have already been parsed by GarbageCollectBlobIfNeeded or entity deserialization earlier in PrepareOutput.
  • Impact: Minor inefficiency on the bottommost seqno-zeroing path with UDT enabled. Not a hot-path concern since this only runs for bottommost-eligible keys with timestamps.
  • Suggested fix: Consider reusing already-parsed entity_blob_columns_ data. Low priority.
M3. MissingBlobRepairIterator::Seek linear scan after re-seek — missing_blob_repair_iterator.h:58-73
  • Issue: Seek re-creates the target with kValueTypeForSeek then scans forward with Next() because rewriting kTypeBlobIndex to kTypeDeletion changes the type byte, inverting sort order at the same sequence number. The linear scan is O(n) in pathological cases (many consecutive rewrites at the same user key).
  • Impact: Acceptable for repair-only path. The comment explaining the rationale is terse.
  • Suggested fix: Expand the comment with a concrete example of the sort-order inversion.
M4. SingleDelete + repair interaction relies on implicit position synchronization — compaction_iterator.cc:1010-1028
  • Issue: When checking missing_blob_repair_iterator_->IsRepairDeletion(), the code relies on the repair iterator being positioned on the next key (after input_.Next() at line 995-997). This is correct because the SequenceIterWrapper wraps through the repair iterator, but the coupling is implicit.
  • Suggested fix: Add an explicit comment linking IsRepairDeletion() to the input position after input_.Next().

🟢 LOW / NIT

L1. BlobIndex::DecodeFrom empty-input handling change — blob_index.h:108-110
  • Issue: Changing assert(slice.size() > 0) to Status::Corruption changes release-build behavior. Previously UB on empty input; now returns an error. Strictly better.
  • Suggested fix: Already correct. Could mention in release notes as a robustness fix.
L2. (void)status_.ok() pattern in PrepareOutput — compaction_iterator.cc:2030
  • Issue: Used to suppress ASSERT_STATUS_CHECKED. The status IS being checked (set, then validity is invalidated). This is the correct approach per codebase conventions.

Cross-Component Analysis

Context Executes? Assumptions hold? Action
WritePreparedTxnDB YES YES - deletion respects snapshot visibility Safe
ReadOnly DB NO - CompactRange blocked N/A Safe
Universal compaction YES YES - PickCompactionForCompactRange handles it Safe
User-defined timestamps YES - extensively tested YES - kTypeDeletionWithTimestamp, seqno zeroing handles blob refs Safe
CompactionService Bypassed - forced local N/A Safe
FIFO compaction Blocked - NotSupported N/A Safe
MemPurge NO - manual only N/A Safe

Positive Observations

  1. Thorough test coverage: 500+ lines covering scalar repair, wide columns, SingleDelete, snapshots, subcompactions, UDT history, merges, compaction filters, error propagation, and re-openability.
  2. Clean architectural fit: Decorator iterator pattern matches ClippingIterator, BlobCountingIterator, HistoryTrimmingIterator.
  3. Defensive design: Manual-only, local-only, FIFO-blocked, defaults off, debug-asserted.
  4. Garbage flow handling: InstallCompactionResults correctly skips missing blob flows with logging.
  5. Stress test integration: Exposed via --compact_range_skip_missing_blob_files.
  6. Release notes: Both new feature and bug fix have entries.

ℹ️ About this response

Generated by Claude Code.
Review methodology: claude_md/ci_review_prompt.md

Limitations:

  • Claude may miss context from files not in the diff
  • Large PRs may be truncated
  • Always apply human judgment to AI suggestions

Commands:

  • /claude-review [context] — Request a code review
  • /claude-query <question> — Ask about the PR or codebase

@meta-codesync meta-codesync Bot changed the title Repair missing blob references during manual compaction Repair missing blob references during manual compaction (#15538) Nov 2, 2026
mikechuangmeta added a commit to mikechuangmeta/rocksdb that referenced this pull request Nov 2, 2026
Summary:
Pull Request resolved: meta#15538

What changed?

- Adds `CompactRangeOptions::skip_missing_blob_files`, default `false`, for manual repair when referenced blob files are absent from the compaction's input Version.

https://pxl.cl/j8V47

- Rewrites selected SSTs. Lost scalar values become deletions. Wide entities keep surviving columns.
- Manual `CompactRange` only. Runs locally; compaction services are bypassed. FIFO is unsupported.

Why?

Best efforts recovery can leave dangling blob references in SSTs. These can cause later compactions to fail. Offline repair prepares the DB for normal reopening.

Reviewed By: xingbowang

Differential Revision: D128836898
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Claude Code Review - OBSOLETE

Superseded by a newer AI review. Expand to see the original review.

✅ Claude Code Review

Auto-triggered after CI passed — reviewing commit 8c714d8


Summary

Well-structured repair feature with careful handling of snapshot semantics, wide column entities, and the compaction pipeline. The design is sound: a wrapper iterator in the compaction pipeline is the right abstraction level, and forcing local-only compaction avoids compaction service complications. Test coverage is extensive.

High-severity findings (2):

  • [version_builder.cc] The SaveBlobFilesTo refactor changes blob file pruning semantics for ALL compactions, not just repair -- may drop blob files that the old code retained when an SST's oldest_blob_file_number is stale but a linked-SST-based scan would find the blob still referenced.
  • [include/rocksdb/options.h] Missing public API documentation for skip_missing_blob_files field in CompactRangeOptions.
Full review (click to expand)

Findings

🔴 HIGH

H1. Version builder SaveBlobFilesTo refactor changes semantics for all code paths -- version_builder.cc
  • Issue: The diff replaces GetMinOldestBlobFileNumber() (which iterates over blob file metadata checking their linked_ssts sets) with a new approach where SaveSSTFilesTo() computes min_oldest_blob_file_num from the oldest_blob_file_number field of each retained SST. This value is then passed to SaveBlobFilesTo(). This change affects ALL version builds, not just those during repair compactions.

    The oldest_blob_file_number in SST metadata is the smallest blob file number referenced by any key in that SST, set at SST creation time. The linked_ssts in blob file metadata tracks which SSTs currently reference that blob file. These can diverge: after compaction rewrites an SST, the old SST's oldest_blob_file_number may point to a blob file whose linked_ssts no longer includes that SST, while a newer blob file might have the SST in its linked set.

    Concretely, the old code asked: "which blob files have at least one SST linking to them?" The new code asks: "what is the smallest blob file number referenced by any SST's metadata?" These should be equivalent in the steady state, but edge cases during concurrent compactions or partial edits could cause them to diverge.

    The new test SaveBlobFilesToWithMissingOldestReference partially validates this, but only tests the case where the SST referencing the oldest blob file is deleted. It doesn't test the case where oldest_blob_file_number points to a blob file that has zero linked_ssts but the SST still exists (which could happen if the SST was rewritten but the edit hasn't been fully applied yet).

  • Root cause: Coupling the repair-enabling refactor with a general semantic change to blob file pruning.

  • Suggested fix: Either (a) keep the old GetMinOldestBlobFileNumber() for the non-repair path and only use the SST-derived min for repair, or (b) add extensive tests covering concurrent compaction scenarios to validate semantic equivalence, and document why the new approach is correct for all cases.

H2. Missing public API documentation -- include/rocksdb/options.h
  • Issue: The diff adds skip_missing_blob_files to CompactRangeOptions but the diff does not show any documentation comment for this new field. Every other field in CompactRangeOptions has documentation comments. Without documentation, users won't understand the semantics, safety constraints (manual-only, no FIFO), or the fact that this converts missing blob references to deletions (which is destructive).
  • Root cause: Documentation was likely omitted from the diff or planned for a follow-up.
  • Suggested fix: Add documentation similar to other fields, explaining: purpose (repair after best-efforts recovery), behavior (missing blob references become deletions, wide entities keep surviving columns), constraints (manual CompactRange only, FIFO unsupported, local-only execution), and the default value.

🟡 MEDIUM

M1. Java API not updated -- java/src/main/java/org/rocksdb/CompactRangeOptions.java
  • Issue: The C API (c_api_gen and db/c.cc) adds getter/setter for skip_missing_blob_files, but the Java CompactRangeOptions class is not updated. This creates an API inconsistency across language bindings. Java users cannot access this feature.
  • Suggested fix: Add skipMissingBlobFiles() getter and setter to CompactRangeOptions.java, plus JNI bridge in rocksjni/portal.h, following the pattern of blobGarbageCollectionPolicy.
M2. No release notes or HISTORY.md entry
  • Issue: The unreleased_history/ directory has no entry for this feature. Per CLAUDE.md guidelines, new features should have release notes at a high level for external user consumption.
  • Suggested fix: Add a brief release note: "Added CompactRangeOptions::skip_missing_blob_files for repairing SSTs with dangling blob references after best-efforts recovery."
M3. No stress test coverage -- db_stress_tool/
  • Issue: Per CLAUDE.md: "When adding a new feature, make sure stress test covers the new option." No changes to db_stress_tool/ are present. While this is a manual-only repair feature, exercising it in stress tests would catch edge cases under concurrent load.
  • Suggested fix: Add skip_missing_blob_files to db_stress's manual compaction path, perhaps triggered occasionally after simulated blob file loss.
M4. OutputHasExternalBlobReferences error handling in PrepareOutput -- compaction_iterator.cc
  • Issue: When OutputHasExternalBlobReferences returns a non-OK status in PrepareOutput, the code sets status_ and invalidates, but also has (void)status_.ok() which appears to suppress the ASSERT_STATUS_CHECKED check. While this avoids a build failure, it's a pattern discouraged by CLAUDE.md ("Avoid new uses of PermitUncheckedError()"). The status IS checked immediately after via validity_info_.Invalidate() and the return, but the void cast is suspicious.
  • Suggested fix: Verify this void cast is necessary for ASSERT_STATUS_CHECKED and document why, or restructure to avoid it.
M5. compaction_picker.cc reordering of inputs_shrunk swap changes behavior for existing code paths -- compaction_picker.cc
  • Issue: The diff moves inputs.files.swap(inputs_shrunk) to AFTER the covering_the_whole_range loop. Before the change, the loop iterated over inputs which had already been swapped to inputs_shrunk (smaller). After the change, the loop iterates over the original inputs before the swap. The loop checks inputs[i] for i > skip_input_index -- these positions in the original inputs contain files that were filtered out by inputs_shrunk.

    This is a behavioral change that affects existing bottommost compactions (kForceOptimized, kIfHaveCompactionFilter), not just repair. The change actually makes the check MORE correct (checking all original files rather than only the shrunk set), so it appears to be a bug fix. But it should be called out explicitly.

  • Suggested fix: Document this as a bug fix in the commit message / release notes, or split it into a separate commit to make the fix reviewable independently.

🟢 LOW / NIT

L1. BlobIndex::DecodeFrom assert-to-Status change may affect existing callers -- blob_index.h
  • Issue: Changing assert(slice.size() > 0) to return Status::Corruption() for empty input is a robustness improvement. Any caller that relied on the assert to catch programming errors in debug mode will now get a Status return. This is strictly more defensive and the new test validates it.
L2. MissingBlobRepairIterator is header-only with non-trivial logic (284 lines)
  • Issue: While this avoids build system changes, header-only implementations of this size can increase compile times. Only included from 2 .cc files so impact is minimal.
L3. No db_bench support
  • Issue: Per CLAUDE.md: "When adding a performance related feature, support it in db_bench." While this is primarily a correctness/repair feature, benchmark support would help validate repair performance on large DBs.

Cross-Component Analysis

Context Does code execute? Assumptions hold? Action needed?
WritePreparedTxnDB YES (manual compaction) YES - repair iterator doesn't affect read_callback No, but no test coverage
ReadOnly DB NO - CompactRange not available N/A Safe
User-defined timestamps YES YES - uses kTypeDeletionWithTimestamp Tested
Universal compaction YES (manual) YES - same pipeline No specific test
FIFO compaction NO - explicitly rejected N/A Tested
CompactionService Bypassed N/A Tested
allow_ingest_behind YES YES - DeletionIsObsolete checks allow_ingest_behind No specific test

Positive Observations

  • Excellent test coverage: 15+ unit tests covering scalars, wide columns, snapshots, single deletes, timestamps, merge operations, error handling, and seek correctness. 10+ integration tests covering end-to-end repair scenarios.
  • Sound architectural choice: Using a wrapper iterator is clean and composable. The repair logic is isolated from the core compaction iterator.
  • Careful snapshot semantics: Repair deletions preserve sequence numbers, correctly interact with snapshot boundaries, and handle SingleDelete+repair with proper conversion.
  • DeletionIsObsolete refactor: Clean extraction that reduces duplication.
  • Idempotent output: Tests verify repair output is well-formed by running it through a second compaction.
  • Wide column handling: Partial entity repair with correct blob column index adjustment is well-implemented.

ℹ️ About this response

Generated by Claude Code.
Review methodology: claude_md/ci_review_prompt.md

Limitations:

  • Claude may miss context from files not in the diff
  • Large PRs may be truncated
  • Always apply human judgment to AI suggestions

Commands:

  • /claude-review [context] — Request a code review
  • /claude-query <question> — Ask about the PR or codebase

mikechuangmeta added a commit to mikechuangmeta/rocksdb that referenced this pull request Nov 3, 2026
Summary:
Pull Request resolved: meta#15538

What changed?

- Adds `CompactRangeOptions::skip_missing_blob_files`, default `false`, for manual repair when referenced blob files are absent from the compaction's input Version.

https://pxl.cl/j8V47

- Rewrites selected SSTs. Lost scalar values become deletions. Wide entities keep surviving columns.
- Manual `CompactRange` only. Runs locally; compaction services are bypassed. FIFO is unsupported.

Why?

Best efforts recovery can leave dangling blob references in SSTs. These can cause later compactions to fail. Offline repair prepares the DB for normal reopening.

Reviewed By: xingbowang

Differential Revision: D128836898
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Claude Code Review - OBSOLETE

Superseded by a newer AI review. Expand to see the original review.

✅ Claude Code Review

Auto-triggered after CI passed — reviewing commit f594ef9


The review has been completed. Here is the final output:

Summary

Solid implementation of a well-scoped repair feature for dangling blob references after best-efforts recovery. The design correctly limits repair to manual CompactRange, forces local execution, prevents trivial moves, and preserves snapshot semantics. Several independent improvements are bundled (BlobIndex::DecodeFrom hardening, VersionBuilder min-blob computation, inputs_shrunk ordering fix, PrepareOutput timestamp preservation for blob records).

High-severity findings (1):

  • [compaction_iterator.cc: PrepareOutput] The timestamp-preservation change for external blob references during seqno zeroing applies to ALL compactions (not just repair) and changes observable behavior for UDT-enabled databases with blob files, but the rationale and impact analysis are not clearly documented.
Full review (click to expand)

Findings

🔴 HIGH

H1. PrepareOutput timestamp preservation is an unbounded behavioral change — compaction_iterator.cc:PrepareOutput
  • Issue: The diff changes the seqno-zeroing path in PrepareOutput from if (!timestamp_size_) to if (!timestamp_size_ || has_external_blob_refs). This means that for timestamp-enabled CFs with blob files, the seqno is zeroed but the timestamp is preserved (instead of both being zeroed). This applies to ALL bottommost compactions, not just repair compactions. The HistoryGcKeepsScalarBlobTimestamp and HistoryGcKeepsWideBlobTimestamp tests confirm this is intentional, but this is a behavioral change for existing users of UDT + blob files.
  • Root cause: External blob references encode the user key (including timestamp) inside the blob index value. Zeroing the timestamp in the internal key would create a mismatch with the embedded key. This is a pre-existing latent bug that this PR fixes — but it's not clearly called out as a separate fix.
  • Suggested fix: (1) Document this as a separate behavioral fix in the PR description and release notes. (2) Consider whether this should be a separate PR for cleaner bisection. (3) Add an explicit comment at the code site explaining WHY blob references must preserve timestamps.

🟡 MEDIUM

M1. DeletionIsObsolete() duplicates logic from ProcessDeletion() — compaction_iterator.cc
  • Issue: DeletionIsObsolete() checks the same conditions as the first branch of ProcessDeletion(). After the insertion, ProcessDeletion()'s first branch is now dead code for the NextFromInput() call site (it can still be reached from SelectMergeOutput at line 899). This duplication is fragile — future changes to one must be mirrored in the other.
  • Root cause: The extraction was needed to handle repair-generated deletions at a different point in the control flow (before the ProcessDeletion call), but the original ProcessDeletion wasn't refactored to remove the now-redundant first branch.
  • Suggested fix: Refactor ProcessDeletion() to call DeletionIsObsolete() internally, or document why the duplication is intentional. This prevents divergent maintenance.
M2. MissingBlobRepairIterator::Seek() has O(n) linear scan — missing_blob_repair_iterator.h
  • Issue: After seeking the underlying iterator, Seek() does a linear scan: while (Valid() && Compare(key(), target) < 0) Next(). This is because the type rewriting (kTypeBlobIndex -> kTypeDeletion) can change the sort order within the same user key and sequence. While this is correct, it could be O(n) in pathological cases where many entries at the same user key are rewritten.
  • Root cause: kTypeDeletion has a higher type value than kTypeBlobIndex in the internal key encoding, meaning a deletion at the same sequence sorts BEFORE the original blob index. The seek target might point between the original and rewritten positions.
  • Suggested fix: Document the O(n) worst case. In practice this is bounded by the number of versions of a single user key within a single SST, which is typically small.
M3. OutputHasExternalBlobReferences() adds per-key overhead on timestamp-enabled hot path — compaction_iterator.cc
  • Issue: This new method is called for EVERY key during PrepareOutput when timestamp_size_ > 0 and BottommostSeqnoCanBeZeroed() is true. It decodes BlobIndex or iterates ForEachBlobFileNumber on every such key. For databases with timestamps but no blob files, this is wasted work.
  • Suggested fix: Consider a cheaper pre-check (e.g., compaction_->DoesInputReferenceBlobFiles()) to skip the per-key decode when no blob references exist in the compaction inputs.
M4. CompactionPicker inputs_shrunk reorder is an independent bug fix — compaction_picker.cc:863-872
  • Issue: The diff moves inputs.files.swap(inputs_shrunk) AFTER the covering_the_whole_range loop. In the old code, the loop iterated over the already-shrunk inputs, using skip_input_index which was an index into the ORIGINAL inputs. After the swap, inputs.size() was the shrunk size, so the loop for (size_t i = skip_input_index + 1; i < inputs.size(); ...) would iterate over a different range than intended. This is a legitimate bug fix but should be documented separately.
  • Suggested fix: Call out this fix in the PR description. It's a correctness fix independent of the repair feature.
M5. skip_missing_blob_files option is not documented in options.h — include/rocksdb/options.h
  • Issue: The diff does not show the skip_missing_blob_files member being added to CompactRangeOptions in the public header with documentation. It appears the option must be added to the struct but the diff might be truncated. If the option lacks documentation, users won't know how to use it safely.
  • Suggested fix: Ensure the option has thorough documentation in options.h including: purpose, when to use it, interaction with other options (FIFO unsupported, blob GC policy), and the requirement to use best_efforts_recovery first.

🟢 LOW / NIT

L1. BlobIndex::DecodeFrom empty-check is a separate improvement — blob_index.h
  • Issue: Changing assert(slice.size() > 0) to Status::Corruption for empty input is a correctness improvement that benefits all callers, not just repair. It should be noted as a separate fix.
L2. Missing SeekToLast()/Prev()/SeekForPrev() test coverage for MissingBlobRepairIterator
  • Issue: The tests exercise SeekToFirst(), Seek(), and Next() but not SeekToLast(), Prev(), or SeekForPrev(). While compaction typically uses forward iteration, the iterator declares support for these operations.
  • Suggested fix: Add basic tests for reverse operations, or document that these are not expected to be used in the compaction path.
L3. RepairEntity() re-allocates vectors on each entity — missing_blob_repair_iterator.h
  • Issue: columns_, blob_columns_, surviving_columns_, surviving_blobs_ are member vectors that are clear()ed and refilled on each entity. The clear() preserves capacity so this is amortized, but the initial allocation per subcompaction could be avoided with reserve().
L4. MissingBlobRepairIterator::PrepareValue() returns true unconditionally
  • Issue: The comment says "Input values must already be prepared, as they are for compaction input." This is correct for the compaction pipeline where values are pre-prepared, but the unconditional return true means the iterator silently drops the preparation contract if used outside compaction.
L5. Test helper RecoverMissingNonOldestBlob is well-designed
  • Positive observation: The shared helper correctly sets up the repair scenario (create data, flush, compact, delete blob file, best-efforts recovery) and is reused across multiple tests, reducing copy-paste.
L6. Double trivial-move protection
  • Issue: Both Compaction::IsTrivialMove() and RunManualCompaction's disallow_trivial_move parameter prevent trivial moves when skip_missing_blob_files is set. This is redundant but harmless defensive programming.

Cross-Component Analysis

Context Affected? Analysis
WritePreparedTxnDB Safe Repair deletions use the same seqno as the original value, so snapshot visibility through read_callback_ is preserved
ReadOnly DB Not affected skip_missing_blob_files is CompactRange-only; ReadOnly DB doesn't compact
CompactionService Correctly handled Forced to local compaction with explicit bypass
User-defined timestamps Affected, tested Creates kTypeDeletionWithTimestamp; tested by RepairPreservesTimestampHistoryAfterRecovery
FIFO compaction Correctly rejected Returns NotSupported before flush
Universal compaction Handled L0 code path in PickCompactionForCompactRange also sets the flag
Crash during repair Safe Standard compaction atomicity — if crash occurs, input SSTs are preserved
allow_ingest_behind Not explicitly tested DeletionIsObsolete() checks !allow_ingest_behind() which is correct

Positive Observations

  • The design correctly limits repair to manual CompactRange, preventing accidental data loss from automatic compactions
  • The MissingBlobRepairIterator is cleanly layered in the iterator chain, after history trimming but before the CompactionIterator
  • The SingleDelete + repair deletion handling is carefully implemented with proper snapshot stripe awareness
  • The RepairPropagatesKnownBlobFileIoFailureDuringGc test verifies that GC on existing blob files still reports real I/O errors (repair doesn't silently swallow legitimate failures)
  • The RepairRetainsHealthyBlobsReferencedByUntouchedSsts test verifies that partial-range repair doesn't orphan healthy blob files referenced by untouched SSTs
  • Good use of SCOPED_TRACE for parameterized test loops

ℹ️ About this response

Generated by Claude Code.
Review methodology: claude_md/ci_review_prompt.md

Limitations:

  • Claude may miss context from files not in the diff
  • Large PRs may be truncated
  • Always apply human judgment to AI suggestions

Commands:

  • /claude-review [context] — Request a code review
  • /claude-query <question> — Ask about the PR or codebase

mikechuangmeta added a commit to mikechuangmeta/rocksdb that referenced this pull request Nov 3, 2026
Summary:
Pull Request resolved: meta#15538

What changed?

- Adds `CompactRangeOptions::skip_missing_blob_files`, default `false`, for manual repair when referenced blob files are absent from the compaction's input Version.

https://pxl.cl/j8V47

- Rewrites selected SSTs. Lost scalar values become deletions. Wide entities keep surviving columns.
- Manual `CompactRange` only. Runs locally; compaction services are bypassed. FIFO is unsupported.

Why?

Best efforts recovery can leave dangling blob references in SSTs. These can cause later compactions to fail. Offline repair prepares the DB for normal reopening.

Reviewed By: xingbowang

Differential Revision: D128836898
mikechuangmeta added a commit to mikechuangmeta/rocksdb that referenced this pull request Nov 3, 2026
Summary:
Pull Request resolved: meta#15538

What changed?

- Adds `CompactRangeOptions::skip_missing_blob_files`, default `false`, for manual repair when referenced blob files are absent from the compaction's input Version.

https://pxl.cl/j8V47

- Rewrites selected SSTs. Lost scalar values become deletions. Wide entities keep surviving columns.
- Manual `CompactRange` only. Runs locally; compaction services are bypassed. FIFO is unsupported.

Why?

Best efforts recovery can leave dangling blob references in SSTs. These can cause later compactions to fail. Offline repair prepares the DB for normal reopening.

Reviewed By: xingbowang

Differential Revision: D128836898
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

✅ Claude Code Review

Auto-triggered after CI passed — reviewing commit 82d26a1


Summary

Well-designed feature that enables offline repair of dangling blob references left by best-effort recovery. The approach of using a wrapper iterator in the compaction input chain is architecturally clean, and the extensive test coverage is commendable. The VersionBuilder refactoring is a sound improvement.

High-severity findings (1):

  • [compaction_iterator.cc: DeletionIsObsolete insertion point] The new DeletionIsObsolete() else-if branch duplicates logic from ProcessDeletion(), creating a maintenance risk where the two code paths could diverge if either is modified independently.
Full review (click to expand)

Findings

🔴 HIGH

H1. DeletionIsObsolete() branch duplicates ProcessDeletion() logic — compaction_iterator.cc
  • Issue: The diff inserts a new else if (DeletionIsObsolete()) branch in NextFromInput() between the rule-(A) hidden-key drop and the existing ProcessDeletion() call. DeletionIsObsolete() is identical to the first condition in ProcessDeletion(), and both paths exist in the same function. If either is modified without the other, they can diverge silently.

    After detailed analysis: the new branch IS functionally equivalent to what ProcessDeletion() would do for these deletions (the non-bottommost path). The DefinitelyInSnapshot + KeyNotExistsBeyondOutputLevel check in DeletionIsObsolete() exactly matches the condition under which ProcessDeletion() drops the deletion in its non-bottommost path. So behavior is preserved for non-repair compactions.

    However, the duplication is a maintenance hazard — two places that must stay in sync for the same logic.

  • Root cause: Refactoring extracted a predicate but kept the original code path.

  • Suggested fix: Have ProcessDeletion() call DeletionIsObsolete() internally to avoid duplication, or add a clear comment explaining the relationship and that both paths MUST be kept in sync.

🟡 MEDIUM

M1. VersionBuilder change subtly alters blob file retention semantics — version_builder.cc
  • Issue: SaveBlobFilesTo() now uses min_oldest_blob_file_num computed from SSTs' oldest_blob_file_number instead of GetMinOldestBlobFileNumber() (which uses blob metadata's linked_ssts). These compute slightly different things: one asks "what's the minimum blob file with linked SSTs?" and the other asks "what's the minimum blob file referenced by any retained SST?"

    When an SST references a blob file whose linked_ssts set is stale (e.g., a compaction removed the link but didn't update oldest_blob_file_number), the two methods disagree. The new approach is more conservative (retains more blob files) and more correct (trusts the SST's own reference). The test SaveBlobFilesToWithMissingOldestReference validates this.

  • Suggested fix: Add a comment explaining why the SST-based approach is preferred.

M2. Seek() in MissingBlobRepairIterator has O(n) linear scan — missing_blob_repair_iterator.h
  • Issue: Seek() does a forward scan after seeking the underlying iterator because type rewriting changes sort order. Acceptable for manual repair (Seek is rare in compaction), but worth documenting.
  • Suggested fix: Add a comment noting the O(n) worst case is acceptable for manual repair.
M3. Missing public API documentation — include/rocksdb/options.h
  • Issue: The diff was truncated and doesn't show the skip_missing_blob_files declaration in CompactRangeOptions. Ensure it's documented with: purpose (repair after best-effort recovery), scope (manual CompactRange only), behavior (converts missing blob refs to deletions), constraints (no remote compaction, no FIFO).
  • Suggested fix: Verify documentation exists and is thorough.
M4. No stress test coverage — general
  • Issue: Per CLAUDE.md: "When adding a new feature, make sure stress test covers the new option." No db_stress changes in the diff.
  • Suggested fix: Add stress test coverage or document why it's impractical.

🟢 LOW / NIT

L1. BlobIndex::DecodeFrom empty-check is a behavioral change for all callers — blob_index.h
  • Issue: Changed from assert(slice.size() > 0) (UB in release on empty slice) to Status::Corruption. This is a pure improvement but affects all callers. Note in release notes as a bug fix.
L2. compaction_picker.cc reorder is a latent bug fix — compaction_picker.cc
  • Issue: Moving covering_the_whole_range check before inputs.files.swap(inputs_shrunk) fixes a bug where the loop iterated over the wrong (post-swap) data. This is a standalone correctness fix bundled into this PR.
  • Suggested fix: Consider mentioning this fix in the PR description.
L3. Compaction::SetSkipMissingBlobFiles assert is debug-only — compaction.h
  • Issue: assert(is_manual_compaction_ || !skip_missing_blob_files) only fires in debug. The actual safety comes from CompactRangeInternal only calling this for manual compactions, so this is fine as a defense-in-depth assert.
L4. No db_bench update — general
  • Issue: Consider adding --skip_missing_blob_files to db_bench's CompactRange options.

Cross-Component Analysis

Context Affected? Analysis
WritePreparedTxnDB Low risk Repair runs in manual compaction; read_callback not involved
User-defined timestamps Tested Multiple tests cover UDT + repair
CompactionService Handled Forced local with logging
FIFO Handled Returns NotSupported before flush
Universal Safe Same compaction picker path
Snapshots Tested Multiple tests verify snapshot semantics
ReadOnly DB N/A Manual compaction unavailable

Positive Observations

  • Excellent test coverage (~35 tests) covering edge cases, snapshots, UDT, subcompactions, merge operators, wide columns, and error propagation
  • Clean iterator chain architecture
  • BlobIndex::DecodeFrom fix from assert to Status::Corruption is a genuine bug fix
  • covering_the_whole_range reorder is a latent bug fix
  • Proper FIFO rejection before flush prevents unnecessary work
  • RepairPropagatesKnownBlobFileIoFailureDuringGc test verifies GC errors for present-but-deleted blob files aren't suppressed

ℹ️ About this response

Generated by Claude Code.
Review methodology: claude_md/ci_review_prompt.md

Limitations:

  • Claude may miss context from files not in the diff
  • Large PRs may be truncated
  • Always apply human judgment to AI suggestions

Commands:

  • /claude-review [context] — Request a code review
  • /claude-query <question> — Ask about the PR or codebase

Summary:
Pull Request resolved: meta#15538

What changed?

- Adds `CompactRangeOptions::skip_missing_blob_files`, default `false`, for manual repair when referenced blob files are absent from the compaction's input Version.

https://pxl.cl/j8V47

- Rewrites selected SSTs. Lost scalar values become deletions. Wide entities keep surviving columns.
- Manual `CompactRange` only. Runs locally; compaction services are bypassed. FIFO is unsupported.

Why?

Best efforts recovery can leave dangling blob references in SSTs. These can cause later compactions to fail. Offline repair prepares the DB for normal reopening.

Reviewed By: xingbowang

Differential Revision: D128836898
@meta-codesync

meta-codesync Bot commented Nov 4, 2026

Copy link
Copy Markdown

This pull request has been merged in a51c5e2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant