Skip to content

ci(tests): run the group and lobby Sweettest suites - #170

Merged
Soushi888 merged 2 commits into
devfrom
chore/160-ci-group-lobby-sweettest
Nov 4, 2026
Merged

Soushi888 merged 2 commits into
devfrom
chore/160-ci-group-lobby-sweettest

Conversation

@Soushi888

@Soushi888 Soushi888 commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

SoushAI analysis. Drafted by Soushi's AI assistant, reviewed and posted by @Soushi888.

Summary

The sweettest job in build.yml ran only nondominium_sweettest, so the Group and Lobby DNAs were tested nowhere but on a laptop. This PR adds their suites as further shards of the same job: group and ndo_anchor from group_sweettest at --test-threads 6, and lobby from lobby_sweettest at --test-threads 1. A failure in any of them turns the sweettest check red, and e2e (which needs sweettest) does not start. The existing nondominium shards from #152 are untouched.

Turning the lobby suite on exposed a real defect in it: all five lobby tests shared one network, so on a hosted runner two to three of them failed with Connection Closed or a consistency timeout. Each test now gets its own network seed, the way the Group suite already does. With that, the lobby shard is green.

Changes
  • Three new matrix entries, each naming its package, target and threads.
  • The run step reads PACKAGE and THREADS from the matrix, defaulting to nondominium_sweettest and 2, so the six existing entries stay as they were.
  • dnas/lobby/tests/src/common/conductors.rs: unique_seed() (same shape as the Group suite's) and with_network_seed on the Lobby DNA, so no two tests share a DHT. Test code only; no zome or DNA change.
  • documentation/TEST_COMMANDS.md: a short "Sweettest in CI" section listing which shard runs which binary.
Decisions
Option Rejected because
A separate job per package Repeats the checkout, Nix, cache and happ-build steps, and e2e would need a second needs:. Matrix entries reuse all of it.
Repeat package: nondominium_sweettest on the six existing entries Edits lines tuned in #152; defaults in the expression leave them alone.
One shard for the two group binaries group is 13 tests and ndo_anchor is 11, two conductors per test. Separate shards keep wall clock near the slowest binary, as #152 did.
Raise the consistency timeouts for lobby Out of scope in #160, and it would hide the cause: the failures were shared-network interference, not slowness.
Skip or #[ignore] the two failing lobby tests Defeats the point of the gate.
  • The group shards use 6 threads because CLAUDE.md and TEST_COMMANDS.md give that figure for the suite. The 2-thread cap in the job comment was measured on the nondominium target (OOM-kill at 6 threads), not on these. The runs below show the group shard completing at 6 threads on a hosted runner (389 s and 200 s).
  • Lobby runs at 1 thread. The seed fix alone was not tested at 2 threads; 1 thread costs about a minute on a five-test suite. Raise it later if wanted.
How to test

CI is the test. The gate was proven on this branch before merge:

  1. With the shards added and one assertion in a group test deliberately broken (a throwaway commit, since dropped), the group shard failed on exactly that test (12 passed, 1 failed), the sweettest check went red and e2e was skipped: red run 41744225698. The same break failed the same way in run 41740048680.
  2. The same runs show the lobby failure that led to the seed fix: run 41734028112 (lobby red at 2 threads, red again on rerun) and 41740048680 (still red at 1 thread, so not thread contention). In 41744225698 the lobby shard is green.
  3. Throwaway commit dropped, history force-pushed with lease: green run on the final head.
Documentation

documentation/TEST_COMMANDS.md: new "Sweettest in CI" section.

Related

Closes #160

Pre-open review

READY TO OPEN. Reviewed against origin/dev at 7faf650 (3 files, 34 insertions, 3 deletions).

  • Title is conventional (ci(tests): ..., imperative, scoped); no vendor attribution, no co-author line, no em-dashes.
  • Scope matches ci: run the group and lobby Sweettest suites #160: three suites in CI, group at 6 threads, existing nondominium shards unchanged, gate proven red then green. Consistency timeouts and the e2e job untouched, as the issue asks.
  • The lobby seed change is test code only; the Group suite uses the identical pattern. No zome, DNA or hApp change.
  • Docs updated in the same PR (TEST_COMMANDS.md).
  • No conflict with dev (GitHub reports the PR mergeable and clean).
  • Open point for the reviewer, not a blocker: the group shards run at 6 threads on a hosted runner, which the job comment warned about for the nondominium target. It finished cleanly in both red and green runs (389 s, 200 s, 200 s); if a runner ever OOM-kills it, lower threads on those two entries.

@Soushi888 Soushi888 added enhancement New feature or request testing Test infrastructure, scenarios, integration tests labels Nov 3, 2026
@Soushi888
Soushi888 force-pushed the chore/160-ci-group-lobby-sweettest branch from ddc0922 to b9b1417 Compare November 4, 2026 00:48
@Soushi888
Soushi888 marked this pull request as ready for review November 4, 2026 01:26
@Soushi888

Copy link
Copy Markdown
Collaborator Author

SoushAI analysis. Drafted by Soushi's AI assistant, reviewed and posted by @Soushi888.

Review APPROVE at head b9b1417, merging under Soushi's explicit -mwr authorisation (2026-11-03 21:42).

  • Scope matches ci: run the group and lobby Sweettest suites #160: build.yml gains three matrix entries (group, ndo_anchor at 6 threads, lobby at 1), with PACKAGE and THREADS defaulting to nondominium_sweettest and 2 so the six existing shards are unchanged. The lobby change is test code only (unique_seed() plus with_network_seed, same pattern as the Group suite). Docs updated in TEST_COMMANDS.md.
  • CI at this head: build, rustfmt, nine Sweettest shards (the six existing plus group, ndo_anchor and lobby) and e2e are all SUCCESS. Branch is mergeable and clean against dev.
  • The PR body links the red and green runs that prove the gate fails when a group test is broken.
  • Open point already noted in the body, not a blocker: if a hosted runner ever OOM-kills the 6-thread group shards, lower threads on those two entries.
  • Follow-up: chore(deps): upgrade to Holochain 0.7.1 #130 may conflict on build.yml and is rebased onto dev after this merge if so.

@Soushi888
Soushi888 merged commit a5d1b43 into dev Nov 4, 2026
12 checks passed
@Soushi888
Soushi888 deleted the chore/160-ci-group-lobby-sweettest branch November 4, 2026 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request testing Test infrastructure, scenarios, integration tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: run the group and lobby Sweettest suites

1 participant