Skip to content

Stabilize bridge, NodeWarden and State Storage tests - #58181

Merged
mregrock merged 8 commits into
ydb-platform:mainfrom
mregrock:codex/stabilize-bridge-distconf-tests
Nov 5, 2026
Merged

mregrock merged 8 commits into
ydb-platform:mainfrom
mregrock:codex/stabilize-bridge-distconf-tests

Conversation

@mregrock

@mregrock mregrock commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Stabilize NodeWarden and bridge tests with state-based waits, deterministic replica controls and retries of transient bridge statuses. Unmute 29 checks.

Validation: repeated C++/ASAN checks; StateStorage/SchemeBoard stress tests and root restart passed three runs each on the unchanged SDK; full chaos passed 23 steps and final data checks. The existing bridge shutdown regression test is preserved.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-04 20:45:04 UTC Pre-commit check linux-x86_64-relwithdebinfo for dd9289b has started.
⚪ 2026-11-04 20:46:01 UTC Artifacts will be uploaded here
⚪ 2026-11-04 20:48:10 UTC ya make is running...
🟢 2026-11-04 20:55:05 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1707 1707 0 0 0 0

🟢 2026-11-04 20:55:09 UTC Build successful.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-04 21:58:28 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6705741 has started.
⚪ 2026-11-04 21:59:41 UTC Artifacts will be uploaded here
⚪ 2026-11-04 22:01:30 UTC ya make is running...
🟢 2026-11-04 22:03:47 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1711 1711 0 0 0 0

🟢 2026-11-04 22:03:52 UTC Build successful.

@mregrock mregrock changed the title Stabilize bridge and NodeWarden tests Stabilize bridge, NodeWarden and State Storage tests Nov 4, 2026
@mregrock
mregrock marked this pull request as ready for review November 4, 2026 23:47
@mregrock
mregrock requested a review from a team as a code owner November 4, 2026 23:47
Copilot AI balanced review requested due to automatic review settings November 4, 2026 23:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The verification read can itself return UNAVAILABLE and trigger an immediate assertion failure instead of retrying within the existing deadline while checking whether the update already committed.

1 open issue
What changed in this PR

Stabilizes bridge, NodeWarden and State Storage tests by waiting for observed state, controlling replica behavior explicitly, and retrying transient failures. These changes complement the linked runtime fixes and unmute 29 checks.

Changes:

  • Replace timing-sensitive checks with configuration and state observations.
  • Handle retryable bridge statuses and configure deterministic test behavior.
  • Remove standard and AddressSanitizer test mutes.
File Description
ydb/​tests/​functional/​bridge/​test_three_piles.py Retry aborted and unavailable bridge updates.
ydb/​tests/​functional/​bridge/​test_mixed_storage.py Wait for storage pool growth.
ydb/​tests/​functional/​bridge/​common.py Expand transient update status handling.
ydb/​tests/​functional/​bridge/​chaos/​test_bridge_chaos.py Treat aborted operations as transient.
ydb/​core/​blobstorage/​ut_blobstorage/​bridge_mixed_storage.cpp Retry activation until nodes report bridge capability.
ydb/​core/​blobstorage/​ut_blobstorage/​bridge_keep_flags.cpp Enable barrier validation for the relevant test.
ydb/​core/​blobstorage/​ut_blobstorage/​bridge_failover.cpp Wait for configuration propagation before failover.
ydb/​core/​blobstorage/​ut_blobstorage/​bridge_data_kind.cpp Explicitly enable barrier validation.
ydb/​core/​blobstorage/​ut_blobstorage/​bridge_common.h Add bounded retries and generation-aware state switching.
ydb/​core/​blobstorage/​nodewarden/​distconf_ut.cpp Explicitly trigger the rejected-binding scenario.
ydb/​core/​blobstorage/​nodewarden/​blobstorage_node_warden_ut.cpp Wait for matching PDisk reports within a deadline.
ydb/​apps/​bridge_skipper/​ydbd_mock/​tests/​test_replication.py Use observed events and explicit replication controls.
.github/​config/​muted_ya.txt Unmute 26 standard checks.
.github/​config/​muted_ya_asan.txt Unmute three AddressSanitizer checks.

Review effort: Balanced

"No quorum to start propose/commit configuration", "Hop node disconnected",
# nodes are still rebinding after another pile has been disconnected
"No quorum for bridge configuration collection",
transient = response.operation.status in (StatusIds.ABORTED, StatusIds.UNAVAILABLE) or (
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-05 00:14:28 UTC Pre-commit check linux-x86_64-relwithdebinfo for 03546d0 has started.
⚪ 2026-11-05 00:15:14 UTC Artifacts will be uploaded here
⚪ 2026-11-05 00:17:18 UTC ya make is running...
🟢 2026-11-05 00:19:29 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1711 1711 0 0 0 0

🟢 2026-11-05 00:19:34 UTC Build successful.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-05 09:17:42 UTC Pre-commit check linux-x86_64-relwithdebinfo for e10d400 has started.
⚪ 2026-11-05 09:18:05 UTC Artifacts will be uploaded here
⚪ 2026-11-05 09:19:35 UTC ya make is running...
🟢 2026-11-05 09:26:49 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1711 1711 0 0 0 0

🟢 2026-11-05 09:26:54 UTC Build successful.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-05 11:08:05 UTC Pre-commit check linux-x86_64-relwithdebinfo for bfd2a54 has started.
⚪ 2026-11-05 11:09:18 UTC Artifacts will be uploaded here
⚪ 2026-11-05 11:11:31 UTC ya make is running...
⚫ 2026-11-05 11:16:38 UTC Check cancelled

@mregrock
mregrock force-pushed the codex/stabilize-bridge-distconf-tests branch from 8e4c3e4 to 173b8ee Compare November 5, 2026 11:15
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-05 11:36:35 UTC Pre-commit check linux-x86_64-relwithdebinfo for 774d731 has started.
⚪ 2026-11-05 11:37:45 UTC Artifacts will be uploaded here
⚪ 2026-11-05 11:40:01 UTC ya make is running...
🟢 2026-11-05 11:46:48 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1746 1746 0 0 0 0

🟢 2026-11-05 11:46:53 UTC Build successful.

@mregrock
mregrock enabled auto-merge (squash) November 5, 2026 11:59
@mregrock
mregrock merged commit 9b191db into ydb-platform:main Nov 5, 2026
10 checks passed
@ydbot

ydbot commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Backport

To backport this PR, click the button next to the target branch and then click "Run workflow" in the Run Actions UI.

Branch Run
stable-26-3-1, stable-26-4-1 ▶  Backport
stable-26-4-1 ▶  Backport

▶  Backport manual

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants