Repository navigation
Conversation
Fix every boundary of the CdcAggregator work (KIKIMR-27694) before the implementation, so the tablet, the node service, the CDC sender, the PersQueue publication and the SchemeShard lifecycle can be developed in parallel. Adds the CdcAggregator tablet type, its event space and log component, the tablet protocol, the local service protocol, the PersQueue publish command, the stream barrier mode with the topic binding and the EnableCdcAggregator feature flag. No behaviour changes.
Diagnostics need the committed state of a stream, in particular the sources that hold the barrier back. Also fixes type references in the protocol.
Model of one CDC stream for CdcAggregator (KIKIMR-27708): the set of sources, their delivered progress and the ready barrier as the minimum over all of them. The aggregate keeps the invariants (monotonic progress and barrier, a silent source blocks the barrier, atomic replacement on split/merge, a retired source never returns); use cases validate requests and persist changes through a repository port. No actors, LocalDB or protobuf, so it is tested with plain command sequences.
Tablet of KIKIMR-27708 that runs the stream use cases in LocalDB transactions: Configure, Activate, ReportDeliveredBatch, ReplaceSource and DescribeStream. Every reply and the publishable barrier leave the tablet only from Complete of their own transaction, so nothing is acknowledged before it is committed; after a restart the stream is restored with its progress, barrier and replacements. The factory is registered for nodes and test runtimes; publication to PersQueue is left to its own stage.
Model of one CDC stream for CdcAggregator (KIKIMR-27708): the set of sources, their delivered progress and the ready barrier as the minimum over all of them. The aggregate keeps the invariants (monotonic progress and barrier, a silent source blocks the barrier, atomic replacement on split/merge, a retired source never returns); use cases validate requests and persist changes through a repository port. No actors, LocalDB or protobuf, so it is tested with plain command sequences.
The tablet is registered in Local, creates its LocalDB tables, loads its state and becomes active. Event handlers come with the tracking logic. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Rebuild the branch as the tablet skeleton on top of main with the merged contracts; the tracking logic moves to a separate pull request. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Pure library for the CdcAggregator tablet: tracked sources, the minimum heartbeat over them in O(log S), split/merge source replacement and use cases speaking the TEvConfigureTracking, TEvWriteHeartbeats, TEvPartitioningChanged and TEvDescribeTracking contracts. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Name DataShard parameters after the contract fields, make the change flags read as flags and drop batch and payload limits that the contract does not define. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The tablet restores the stored tracking config through the same parsing that validates ConfigureTracking requests. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Handle ConfigureTracking, WriteHeartbeats and PartitioningChanged in tablet transactions that persist the stream changes and reply after commit; DescribeTracking answers from memory. TxInit restores the stream from LocalDB. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ilt on the stream use cases
The tablet persists the tracking config and the source DataShards, replaces a parent with its children on PartitioningChanged and restores both on restart. Heartbeats are still answered with NOT_READY. Transaction types are no longer aligned and tabletId moves to the log context, as requested in the review of the skeleton. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Keep the protobuf response alive until the sources have been copied. This fixes SIGSEGV in the five tablet tests that describe non-empty tracking state.
…the tablet test Select the source-list overload explicitly for the empty configuration. This preserves the BAD_REQUEST case after adding the raw-request overload.
✅ Documentation buildRevision built successfully |
There was a problem hiding this comment.
AI Review Summary
Verdict: ✅ No Critical or Major issues found
Critical and Major issues
No Critical or Major issues found.
Other findings
- Minor | Medium: Duplicate source entries in one
TEvWriteHeartbeatsbatch are silently deduplicated last-wins and only one result entry is reported for the two submitted heartbeats —ydb/core/tx/cdc_aggregator/tablet/cdc_aggregator_stream_proto.cpp:108 - Minor | Medium: Replaying an already-applied partitioning change after the completion notification has been processed (history pruned) returns UNKNOWN_SOURCE for a change that was successfully applied —
ydb/core/tx/cdc_aggregator/tablet/cdc_aggregator_stream.cpp:78 - Minor | Medium: No test covers a heartbeat batch containing duplicate source entries, so the last-wins dedup behavior in
ParseHeartbeatsBatchis unspecified by tests —ydb/core/tx/cdc_aggregator/tablet/ut/tablet_ut.cpp:99 - Nit | High: The
!diff->Deleted.empty()guard is misleading — for any freshly applied changeDeletedalways contains the parent, so the guard effectively means "not a duplicate replay" —ydb/core/tx/cdc_aggregator/tablet/tx_partitioning_changed.cpp:58 - Nit | Medium: The old dead
Heartbeatstable (id 3) is not actually dropped from existing deployments (db.Materialize<Schema>()never removes tables absent from the schema), and the "table id 3 must not be reused" constraint is only stated in the PR description —ydb/core/tx/cdc_aggregator/tablet/schema.h:31
This review was generated automatically. Critical and Major issues require attention; Minor and Nit findings are advisory.
If this comment was useful, please give it a 👍 — it helps us improve the review bot.
| batch.InvalidSourceDataShardIds.insert(sourceDataShardId); | ||
| continue; | ||
| } | ||
| batch.HeartbeatsBySourceDataShardId[sourceDataShardId] = std::move(heartbeat); |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
batch.HeartbeatsBySourceDataShardId[sourceDataShardId] = std::move(heartbeat); silently overwrites an earlier entry for the same source, so a TEvWriteHeartbeats request containing the same SourceDataShardId twice produces only one THeartbeatResult (from batch.HeartbeatsBySourceDataShardId) for two submitted heartbeats, and only the last heartbeat survives — the earlier one is neither rejected nor acknowledged.
ParseHeartbeatsBatch carefully validates every other edge case (zero step, empty data, unknown source) but treats duplicate entries as an unspecified last-wins merge. If duplicates are a protocol violation, mark them BAD_REQUEST like other invalid inputs; if they are legal, document the last-wins semantics so callers know the per-source result count may be smaller than the request count.
Suggested fix: detect duplicates (e.g. check count(sourceDataShardId) before assignment or track a seen-set) and add the entry to InvalidSourceDataShardIds with a BAD_REQUEST result.
| } | ||
|
|
||
| const auto parent = LastHeartbeatsBySourceDataShardId.find(change.ParentDataShardId); | ||
| if (parent == LastHeartbeatsBySourceDataShardId.end()) { |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
A replay of an already-applied partitioning change now returns std::nullopt (→ UNKNOWN_SOURCE ack) once the corresponding history rows have been pruned by TTxForgetPartitioningChanges (which runs as soon as the SchemeShard completion notification arrives). Before this PR the ack of a partitioning change was at least persistent across restarts; after completion processing, the same retried TEvPartitioningChanged — e.g. the original ack was lost in transit and the sender retries — gets an error for a change that was applied and is reflected in the source set.
The pruning is clearly intentional (history cleanup), and the retrying sender can recover by re-reading the source set, but the returned error code conflates "change never seen" with "change seen, applied, and forgotten". Consider a distinct status (e.g. SUCCESS/ALREADY_APPLIED for a change whose parent is no longer tracked and whose children are all known) or documenting that UNKNOWN_SOURCE may arrive for acknowledged changes after completion, so the future caller of this API can handle retries safely.
| } | ||
|
|
||
| void RebootTablet() { | ||
| CompletionSubscribers.clear(); |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
All new heartbeat tests submit at most one entry per source per batch. The last-wins deduplication of duplicate SourceDataShardId entries in ParseHeartbeatsBatch (and the fact that the result then contains fewer THeartbeatResult entries than request entries) is not covered by any test, so that behavior is unspecified and could change unnoticed. Consider adding a test that sends {{1,100},{1,200}} and asserts both the resulting sources' heartbeat and the result-entry count.
| return true; | ||
| } | ||
|
|
||
| if (!diff->Deleted.empty()) { |
There was a problem hiding this comment.
Severity: Nit
Confidence: High
if (!diff->Deleted.empty()) reads as a special case for inserts-only diffs, but TCdcAggregatorStream::ApplyPartitioningChange always inserts the parent into diff.Deleted for a freshly applied change; the only case with empty Deleted is a duplicate replay (which returns an empty diff to skip persistence). Suggest naming the intent explicitly, e.g. if (!applied duplicate) via a dedicated flag on the diff (or an early return true before the persistence block for replays), so future readers do not assume inserts-only changes exist.
| struct VersionStep : Column<1, NScheme::NTypeIds::Uint64> {}; | ||
| struct VersionTxId : Column<2, NScheme::NTypeIds::Uint64> {}; | ||
| struct Data : Column<3, NScheme::NTypeIds::String> {}; | ||
| struct PartitioningChanges : Table<4> { |
There was a problem hiding this comment.
Severity: Nit
Confidence: Medium
Table id 3 is being freed by removing Heartbeats from the schema. Note that TTxInitSchema runs db.Materialize<Schema>(), which only creates the tables/columns listed in the schema and never removes tables absent from it — so on existing deployments the old table 3 and its data remain in the tablet. That is harmless (the table was unused), but the constraint "table id 3 must not be reused" is only stated in the PR description and not visible to future editors of this file. Consider adding a short comment near the PartitioningChanges : Table<4> definition, e.g. // Table id 3 was used by the removed Heartbeats table; do not reuse id 3.
Activate the tablet with an explicit configuration load error and reject reconfiguration instead of overwriting persisted sources and partitioning history. Cover malformed and semantically invalid blobs, repeated reboots, and recovery after restoring the original configuration.
Merge the tracking dependency while retaining per-source and current heartbeat persistence and the transactional heartbeat writer.
|
⚪
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
✅ Documentation buildRevision built successfully |
There was a problem hiding this comment.
AI Review Summary
Verdict: ❌ Critical: 1; Major: 1
Critical and Major issues
- Critical | High: The PR head does not build: linking
ydb/core/tx/cdc_aggregator/tablet/utfails with undefinedOut<NKikimrCdcAggregator::TEvConfigureTrackingResult_EStatus>andOut<NKikimrCdcAggregator::TEvPartitioningChangedAck_EStatus>(confirmed by CI run 44007853556). The new tests compare these proto enums withUNIT_ASSERT_VALUES_EQUAL, which requiresOut<>symbols generated byGENERATE_ENUM_SERIALIZATION(cdc_aggregator.pb.h), missing inydb/core/protos/ya.make—ydb/core/tx/cdc_aggregator/tablet/ut/tablet_ut.cpp:216 - Major | Medium:
tablet_impl.hincludes<ydb/core/tx/schemeshard/schemeshard.h>butydb/core/tx/cdc_aggregator/tablet/ya.makedoes not addPEERDIR(ydb/core/tx/schemeshard), so compilation relies on include dirs leaking in transitively from other targets; in stricter/other build configurations the include is not resolvable —ydb/core/tx/cdc_aggregator/tablet/ya.make:16
Other findings
- Minor | Medium: After a split, if the deleted parent had the highest observed heartbeat but the stream heartbeat was still unset (another source lagged), children may report lower versions and the stream heartbeat starts below the version the parent had already reached — the reported watermark can regress relative to data the deleted parent had durably written —
ydb/core/tx/cdc_aggregator/tablet/cdc_aggregator_stream.cpp:60 - Minor | Medium: A
TEvWriteHeartbeatsbatch containing two entries for the same source, one valid and one invalid, produces twoTHeartbeatResultrecords for the sameSourceDataShardId(one SUCCESS and one BAD_REQUEST); the client cannot resolve which entry was applied —ydb/core/tx/cdc_aggregator/tablet/cdc_aggregator_stream_proto.cpp:132 - Minor | High:
TTxWriteHeartbeatspersists every heartbeat in the batch even when it did not advance the source's last heartbeat (RegisterHeartbeatsreturns unchanged entries too), so repeated or stale heartbeat batches rewrite unchangedSourcesrows on every write —ydb/core/tx/cdc_aggregator/tablet/tx_write_heartbeats.cpp:59 - Minor | Medium: A redelivered
TEvPartitioningChangedfor an operation that was already applied, completed at SchemeShard and forgotten getsUNKNOWN_SOURCEeven though the change was applied; senders that treatUNKNOWN_SOURCEas a permanent failure will not re-ack, and the semantics differ from an in-flight replay (which returns SUCCESS) —ydb/core/tx/cdc_aggregator/tablet/tx_partitioning_changed.cpp:55 - Minor | Low: On a failed pipe connect (
TEvClientConnectedwith non-OK status) the tablet closes and reschedules the pipe itself via a 1sTEvWakeup, while the pipe client was created withTClientRetryPolicy::WithRetries()and retries internally; the two overlapping retry loops can postpone resubscription of completion notifications beyond either policy's intent —ydb/core/tx/cdc_aggregator/tablet/tablet.cpp:108 - Minor | Low:
LoadLastHeartbeatsguards onlyLastHeartbeatStepwithHaveValueand then readsLastHeartbeatTxId/LastHeartbeatDataunconditionally; a row whereLastHeartbeatStepis set butLastHeartbeatDatais missing aborts the tablet at boot instead of surfacing a recoverable load error —ydb/core/tx/cdc_aggregator/tablet/cdc_aggregator_stream_repository.cpp:104
This review was generated automatically. Critical and Major issues require attention; Minor and Nit findings are advisory.
If this comment was useful, please give it a 👍 — it helps us improve the review bot.
|
|
||
| void CheckInvalidPersistedTrackingConfig(const TString& config, const TString& error) { | ||
| TTestEnv env; | ||
| UNIT_ASSERT_VALUES_EQUAL(env.ConfigureTracking({1, 2}).GetStatus(), TConfigureResult::SUCCESS); |
There was a problem hiding this comment.
Severity: Critical
Confidence: High
The PR head fails to link: ydb/core/tx/cdc_aggregator/tablet/ut fails with undefined symbol: void Out<NKikimrCdcAggregator::TEvConfigureTrackingResult_EStatus>(IOutputStream&, ...) and Out<NKikimrCdcAggregator::TEvPartitioningChangedAck_EStatus> (see CI run 44007853556, fail_summary.txt). These symbols are required because the new tests compare proto enum values with UNIT_ASSERT_VALUES_EQUAL(...GetStatus(), TConfigureResult::SUCCESS) here and throughout the suite, which instantiates CompareAndMakeStrings/Out<> for the enum type.
The symbols are normally provided by GENERATE_ENUM_SERIALIZATION, but ydb/core/protos/ya.make does not generate enum serialization for cdc_aggregator.pb.h (only blobstorage_pdisk_config.pb.h, datashard_load.pb.h, shared_cache.pb.h are listed there).
Suggested fix: add GENERATE_ENUM_SERIALIZATION(cdc_aggregator.pb.h) to ydb/core/protos/ya.make (or avoid streaming/comparing these enums directly in the tests).
| tx_write_heartbeats.cpp | ||
| ) | ||
|
|
||
| PEERDIR( |
There was a problem hiding this comment.
Severity: Major
Confidence: Medium
tablet_impl.h now includes <ydb/core/tx/schemeshard/schemeshard.h> (needed for NSchemeShard::TEvSchemeShard::TEvNotifyTxCompletion/TEvNotifyTxCompletionResult), but this module does not declare PEERDIR(ydb/core/tx/schemeshard). I traced the PEERDIR closure of all peers listed here (ydb/core/base, ydb/core/tablet, ydb/core/tablet_flat, ydb/core/persqueue/common, etc.) — none of them reaches ydb/core/tx/schemeshard, so the include resolves only via transitive include-directory leakage from unrelated targets. That makes the build fragile and breaks in configurations where those leakage paths differ (Arcadia requires an explicit PEERDIR for cross-module includes).
Suggested fix: add ydb/core/tx/schemeshard to PEERDIR here, and also add it to the ut/ya.make PEERDIR set (its tablet_ut.cpp includes ydb/core/tx/schemeshard/schemeshard.h as well; ydb/core/testlib/default does not export it).
| if (SourceDataShardIdsByLastHeartbeatVersion.size() != LastHeartbeatsBySourceDataShardId.size()) { | ||
| return std::nullopt; | ||
| } | ||
| const auto& [minVersion, minSourceDataShardId] = *SourceDataShardIdsByLastHeartbeatVersion.begin(); |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
The "stream heartbeat never moves back" invariant only holds while CurrentHeartbeat is already set. Scenario: configure sources {1, 2}, source 1 reports version 100 (source 2 has not reported, so CurrentHeartbeat is still unset), then shard 1 splits into 3 and 4. The parent's heartbeat (100) is dropped together with the parent; children start with no heartbeat. When children later report versions lower than 100 (e.g. they restart from a lower resolved timestamp) and source 2 catches up, the stream heartbeat is established at the children's lower version — i.e. below the version the deleted parent had already durably reached.
The existing test ShouldNotMoveCurrentHeartbeatBackWhenChildReportsOlderVersion only covers the case where the stream heartbeat was already advanced before the split. Consider seeding the new children with a floor (e.g. the parent's last heartbeat version at split time) to keep the watermark monotonic across the parent removal, or documenting that a fresh child reset may lower the first-published watermark.
|
|
||
| TResult proto; | ||
| proto.SetStatus(TResult::SUCCESS); | ||
| for (const auto& [sourceDataShardId, _] : batch.HeartbeatsBySourceDataShardId) { |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
A TEvWriteHeartbeats request with two entries for the same SourceDataShardId — one parseable and one invalid — yields two THeartbeatResult records for the same source: one SUCCESS (from HeartbeatsBySourceDataShardId, because the earlier valid entry wins) and one BAD_REQUEST (from InvalidSourceDataShardIds). Conversely, two valid entries silently keep only the last one. A client matching results back to its own entries by position or by id gets an ambiguous outcome.
Suggested fix: on a duplicate source id either reject the whole batch entry set for that source with BAD_REQUEST, or make the last entry win consistently in both maps.
| const auto batch = Self->CdcAggregatorStreamProtoConverter.ParseHeartbeatsBatch(Ev->Get()->Record); | ||
| const auto diff = Self->CdcAggregatorStream->WriteHeartbeats(batch.HeartbeatsBySourceDataShardId); | ||
| NIceDb::TNiceDb db(txc.DB); | ||
| Self->CdcAggregatorStreamRepository.ApplyCdcAggregatorStreamHeartbeatsDiff(db, diff); |
There was a problem hiding this comment.
Severity: Minor
Confidence: High
RegisterHeartbeats includes every heartbeat from the batch in the returned diff, even when it did not advance the source's last heartbeat (an older or equal version returns the unchanged value), and ApplyCdcAggregatorStreamHeartbeatsDiff then rewrites the Sources row with identical values. DataShards resend heartbeats periodically, so batches that change nothing still produce a full set of row updates (3 columns per source) in every such transaction.
Suggested fix: return only sources whose version actually advanced from RegisterHeartbeats (or filter unchanged entries before persisting) and report SUCCESS with the already-written version for the rest.
| const auto diff = Self->CdcAggregatorStream->ApplyPartitioningChange(*change); | ||
| if (!diff) { | ||
| Result->Record.SetStatus(TResult::UNKNOWN_SOURCE); | ||
| Result->Record.SetError(TStringBuilder() << "Unknown partitioning change " << change->OperationId); |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
Once an operation completes at SchemeShard, TTxForgetPartitioningChanges erases the (parent, operationId) history. A TEvPartitioningChanged redelivered after that point reaches ApplyPartitioningChange, no longer finds the parent among the sources, and is answered UNKNOWN_SOURCE — even though the change was applied and acknowledged earlier. An in-flight replay (operation not yet completed) instead returns SUCCESS. The same sender therefore sees two different outcomes for semantically identical retries, and a sender that maps UNKNOWN_SOURCE to "this shard is not tracked, stop retrying" (as opposed to treating it as retryable) would silently drop a retry after an ack was lost.
The tests codify the UNKNOWN_SOURCE behavior, so this may be intended, but the distinction between "never applied" and "applied and forgotten" is lost on the wire. Consider a dedicated status (or SUCCESS) for replays of forgotten-but-applied operations if senders need to distinguish them.
| result->Record.SetStatus(NKikimrCdcAggregator::TEvPartitioningChangedAck::NOT_READY); | ||
| result->Record.SetOrigin(TabletID()); | ||
| ctx.Send(ev->Sender, result.Release(), 0, ev->Cookie); | ||
| void TCdcAggregator::Handle(TEvTabletPipe::TEvClientConnected::TPtr& ev, const TActorContext& ctx) { |
There was a problem hiding this comment.
Severity: Minor
Confidence: Low
The pipe client is created with TClientRetryPolicy::WithRetries() (unlimited internal retries with up to 10-minute backoff), but the owner additionally reacts to every failed connect by closing the client and scheduling a 1-second TEvWakeup retry. With WithRetries the client usually keeps retrying internally and does not report each failure to the owner; when it does report (e.g. resolve failure paths), the owner tears down a client that was about to retry on its own. The two overlapping recovery mechanisms make the actual resubscription delay hard to reason about. Consider relying on the pipe client's own retry policy and only reacting to TEvClientDestroyed, or using WithoutRetries plus the owner-driven wakeup loop.
| TCdcAggregatorStream::TLastHeartbeats lastHeartbeats; | ||
| while (!rowset.EndOfSet()) { | ||
| auto& lastHeartbeat = lastHeartbeats[rowset.GetValue<Schema::Sources::DataShardId>()]; | ||
| if (rowset.HaveValue<Schema::Sources::LastHeartbeatStep>()) { |
There was a problem hiding this comment.
Severity: Minor
Confidence: Low
The row is accepted whenever LastHeartbeatStep has a value, but LastHeartbeatTxId and LastHeartbeatData are then read unconditionally. ApplyCdcAggregatorStreamHeartbeatsDiff always writes all three columns together, so this cannot happen with the current writer, but a partially written row (future schema change, manual edit, backup/restore skew) would abort the tablet during boot inside LoadLastHeartbeats instead of producing a recoverable load error like the TrackingConfig path does. Consider validating all heartbeat columns with HaveValue before constructing the heartbeat.
Keep the recovery assertions unchanged while reporting complete protobuf responses instead of requiring missing enum output specializations.
|
⚪ ⚪
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
✅ Documentation buildRevision built successfully |
There was a problem hiding this comment.
AI Review Summary
Verdict: ✅ No Critical or Major issues found
Critical and Major issues
No Critical or Major issues found.
Other findings
- Minor | Medium:
tablet_impl.hincludes<ydb/core/tx/schemeshard/schemeshard.h>but the tablet LIBRARY does not declare a PEERDIR onydb/core/tx/schemeshard; tests only build via the transitiveydb/core/testlibdependency —ydb/core/tx/cdc_aggregator/tablet/ya.make:16 - Minor | Medium: the removed
Heartbeats(Table<3>) is never dropped from existing LocalDBs (Materialize<Schema>()does not drop removed tables), leaving orphaned data; considerdb.Cleanup<Schema>()inTxInitSchema—ydb/core/tx/cdc_aggregator/tablet/tx_init_schema.cpp:23 - Minor | High: restart tests never assert the persisted heartbeat
Datapayload, so a regression in theLastHeartbeatData/CurrentHeartbeatDatapersistence path would go unnoticed —ydb/core/tx/cdc_aggregator/tablet/ut/tablet_ut.cpp:820 - Nit | High:
!diff->Deleted.empty()is always true for freshly applied changes (parent is always deleted) and only filters replays; the persistence gate would be clearer as an explicit "applied for the first time" flag —ydb/core/tx/cdc_aggregator/tablet/tx_partitioning_changed.cpp:59 - Nit | High: duplicate heartbeats for the same source in one batch silently resolve last-wins instead of being rejected like the other malformed inputs —
ydb/core/tx/cdc_aggregator/tablet/cdc_aggregator_stream_proto.cpp:108
This review was generated automatically. Critical and Major issues require attention; Minor and Nit findings are advisory.
If this comment was useful, please give it a 👍 — it helps us improve the review bot.
| tx_write_heartbeats.cpp | ||
| ) | ||
|
|
||
| PEERDIR( |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
tablet_impl.h includes <ydb/core/tx/schemeshard/schemeshard.h> (for NSchemeShard::TEvSchemeShard::TEvNotifyTxCompletion / TEvNotifyTxCompletionResult), but this LIBRARY does not declare a PEERDIR on ydb/core/tx/schemeshard. I traced the PEERDIR closure of ydb/core/tx/cdc_aggregator/tablet and ydb/core/tx/schemeshard is not reachable from it; the unit tests only build because ut -> ydb/core/testlib/default -> ydb/core/testlib -> ydb/core/tx/schemeshard happens to provide the include path transitively. The library itself relies on permissive include-path resolution, so strict dependency checking (or building/consuming this LIBRARY outside the test world, e.g. from a new caller) can break.
Please add ydb/core/tx/schemeshard to the PEERDIR list (other subscribers to TEvNotifyTxCompletion, like ydb/core/kqp/gateway/actors, declare it explicitly).
| YDB_LOG_DEBUG("Execute"); | ||
|
|
||
| NIceDb::TNiceDb db(txc.DB); | ||
| db.Materialize<Schema>(); |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
The PR removes Heartbeats : Table<3> from the schema, but db.Materialize<Schema>() only creates tables/columns that are missing (EMaterializationMode::All calls AddTable/AddColumn); it never drops tables that are no longer part of the schema. A CdcAggregator tablet whose LocalDB was created while the intermediate schema (with table 3) was deployed keeps the orphaned Heartbeats table and its rows forever. The PR description correctly says table id 3 must not be reused, but nothing cleans up the leftover data.
Consider using db.Cleanup<Schema>() here (it drops tables and columns that are absent from the current schema, see NIceDb::Schema::Cleanup in flat_cxx_database.h), or add a one-time migration that drops table 3.
| UNIT_ASSERT_VALUES_EQUAL_C(describeResult.GetCurrentHeartbeat().GetStep(), 200, describeResult.ShortDebugString()); | ||
| } | ||
|
|
||
| Y_UNIT_TEST(ShouldKeepHeartbeatsWhenTabletRestarted) { |
There was a problem hiding this comment.
Severity: Minor
Confidence: High
Test gap: restart round-trip of the persisted heartbeat Data payload is never asserted. ShouldKeepHeartbeatsWhenTabletRestarted (and the other restart tests) only compare Step values, while LoadLastHeartbeats/LoadCdcAggregatorStream additionally read Sources::LastHeartbeatData and State::CurrentHeartbeatData. Note that TEvDescribeTrackingResult does not expose the data at all (only versions), so if persistence of the data columns regressed (e.g. the LastHeartbeatData/CurrentHeartbeatData update was dropped or reordered), no existing test would notice, and the payload would silently be lost (or replaced by an empty string) after a tablet restart.
Suggest asserting the persisted bytes via a LocalQuery on the Sources/State tables (the test already uses this technique in ReadTrackingConfig) in at least one restart test.
| return true; | ||
| } | ||
|
|
||
| if (!diff->Deleted.empty()) { |
There was a problem hiding this comment.
Severity: Nit
Confidence: High
For every newly applied change diff.Deleted always contains the parent (TCdcAggregatorStream::ApplyPartitioningChange inserts it unconditionally), and the only path that reaches this line with an empty Deleted is the replayed-identical-change early return. So !diff->Deleted.empty() is effectively "this is a fresh apply, not a replay", which is hard to see from the condition and easy to break later (e.g. if a change type that only inserts sources is added, persistence would be silently skipped).
Suggest making the intent explicit, e.g. have ApplyPartitioningChange return a flag (or std::optional vs. empty-diff distinction) that says whether the change was applied for the first time, and gate persistence on that.
| batch.InvalidSourceDataShardIds.insert(sourceDataShardId); | ||
| continue; | ||
| } | ||
| batch.HeartbeatsBySourceDataShardId[sourceDataShardId] = std::move(heartbeat); |
There was a problem hiding this comment.
Severity: Nit
Confidence: High
Duplicate entries for the same SourceDataShardId inside one TEvWriteHeartbeats batch are accepted and silently resolve to last-wins (batch.HeartbeatsBySourceDataShardId[sourceDataShardId] = ... overwrites the earlier entry), and the reply reports SUCCESS for the surviving version only. Every other malformed input (zero step, empty data, missing heartbeat) is rejected with BAD_REQUEST, so silently coalescing duplicates is inconsistent; the sender gets no signal that part of its batch was discarded.
If duplicates are expected from senders, a comment documenting last-wins would help; otherwise mark duplicates as BAD_REQUEST like the other invalid inputs.
KIKIMR-27708
The CdcAggregator tablet now accepts
TEvWriteHeartbeats: it persists the last heartbeat of each source DataShard and advances the stream heartbeat to the minimum once every source has reported. The stream heartbeat never moves back, including after split/merge and tablet restarts. An in-memory index of sources ordered by version keeps a write at O(log S) for tables with ~100k shards.Builds on #59120 (its commits are included until it is merged). The unused
Heartbeatstable (id 3) is dropped from the schema; table id 3 must not be reused.