contrib/dev-util: make the cluster script work on macOS - #6122

Draft

emelialei88 wants to merge 3 commits into

bloomberg:mainfrom

emelialei88:fix/cluster-script

emelialei88 commented Aug 10, 2026 •

edited

Loading

Contributor

🎯 The "Why" (Intent)

contrib/dev-util/cluster assumed a Linux host and an image that only ran a
host-built binary. On macOS init, run, tmux and test all broke: no systemd, no
docker group, no /proc, and a Mach-O binary Linux containers cannot execute.
Getting it working surfaced breakage that was never host-specific.

🛠️ The "What" (Critical Changes)

update builds Comdb2 inside a node; the bind-mounted source means one build
serves the cluster, in build-docker/ so it does not fight a host cmake cache.
Linux-only checks — systemctl, docker group, /proc, sudo rm — are guarded
by uname; figlet falls back to a plain banner.
Fixes unrelated to the host: the image would not build where the host uid
collides with the stock ubuntu account (uid 1000) in noble and later;
start brought up nothing; setup ran a copy of this script frozen into the
image; the client config went where cdb2api never reads it; and the persist=0
tmpfs mounted on a literal /opt/bb:exec.
db becomes startdb; stopdb, rmdb and set are new; status names the
databases. Matching is by command line — comdb2 renames its main thread, so
matching the process name misses a live server.
tmux targets its own window, so panes cannot land in an unrelated attached
session; gdb refuses under CPU emulation, where ptrace cannot read
registers, and gdb/vg stop a running database first.

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
cdb2jdbc
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
phys_rep_tiered [timeout]
phys_rep_tiered_nosource_generated [timeout]
phys_rep_tiered_firstfile_generated [timeout]

roborivers approved these changes

View reviewed changes

roborivers left a comment

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
cdb2jdbc
reco-ddlk-sql **quarantined**
timepart_retro
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
phys_rep_tiered [timeout]
phys_rep_tiered_firstfile_generated [timeout]
phys_rep_tiered_nosource_generated [timeout]

emelialei88 force-pushed the fix/cluster-script branch 4 times, most recently from bd26dde to 351697e Compare

August 12, 2026 17:58

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_truncate [db unavailable at finish]
cdb2jdbc
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
ssl_set_cmd
ssl_prefer
ssl_dbname
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
queuedb_rollover **quarantined**
cdb2jdbc
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**

emelialei88 force-pushed the fix/cluster-script branch from 351697e to fe7dc20 Compare

August 12, 2026 20:13


          contrib/dev-util: make the cluster script work on macOS

a61cdb7

The helper assumed a Linux host and a runtime-only image. Guard the
Linux-only bits behind uname, and build comdb2 inside the containers so
the same flow works on Linux and Mac.

Fix the host-agnostic breakage that surfaced along the way: "start"
brought up nothing, "setup" ran a copy of this script frozen into the
image, the client config was shipped where cdb2api never reads it, the
persist=0 tmpfs mounted on a literal "/opt/bb:exec", and the image
failed to build on any host whose uid collides with the stock "ubuntu"
account that ubuntu:noble and later ship on uid 1000.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Signed-off-by: Emelia Lei <[email protected]>

emelialei88 force-pushed the fix/cluster-script branch from fe7dc20 to a61cdb7 Compare

August 12, 2026 21:00

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume
cdb2jdbc
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_truncate_multiddl_generated [db unavailable at finish] **quarantined**
queuedb_rollover_noroll1_generated **quarantined**
cdb2jdbc
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**

emelialei88 and others added 2 commits

August 20, 2026 11:21


          contrib/dev-util: let "setup" run in a container with no config dir

879b7eb

"setup" re-runs this script inside m0, where the guard at the bottom
demanded ~/.local/share/comdb2-util and otherwise bailed out. That
directory is only there because "build" bakes a copy into the image,
and the in-container half of setup does not want it anyway: it reads
$CLUSTER from .bash_profile and never opens config.sqlite. On WSL the
copy is missing and setup died before doing any work.

Based on Salil Chandra's fix in bloomberg#6130, without the /proc/self/cgroup
half of the container test - under cgroup v2 that file reads "0::/",
so the second field is always empty and /.dockerenv was carrying the
condition on its own.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Signed-off-by: Emelia Lei <[email protected]>


          contrib/dev-util: build on current Ubuntu, not a pinned 22.04

a839067

The branch had pinned the non-Ubuntu-host default to ubuntu:22.04;
main already tracks ubuntu:latest, so this just gives that back. A
container should follow the current Ubuntu rather than get steadily
more stale, and "set base ubuntu:<release>" pins it again if some
future release ever breaks the build.

Newer releases need one adjustment: /var/run is a symlink to /run,
where installing openssh-server may already have created sshd, and
the bare mkdir then failed and took the image build with it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Signed-off-by: Emelia Lei <[email protected]>

roborivers suggested changes

View reviewed changes

roborivers left a comment

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
truncatesc_offline_generated [timeout] **quarantined**

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet