Skip to content

Latent: in-memory hash node takes on-disk resolution path -> NULL deref (dbgeteof via hashresolvevalue) #828

Open

Description

@jsavin

Found during 2026-08-09 baseline work (unit 1.3 verification; also seen by units 0.1/1.5). Nondeterministic SIGSEGV (30-80% rate, load-dependent) in test_callback_infrastructure and test_window_bridge. Identical stack: dbgeteof <- dbnormalizeaddress <- dbrefhandle <- hashresolvevalue, NULL deref at 0x0 — an in-memory hash node spuriously takes the on-disk resolution path.

test_callback_infrastructure opens NO database file, so cross-worktree/db interference is excluded; this looks like a genuine latent bug in hash node resolution, likely relevant beyond tests. lldb attach was sandbox-blocked during the run — no backtrace beyond the stack above.

Possibly related: the session-history-dependent failures in # (same family suspicion).

Activity

  1. jsavin commented on Aug 10, 2026

    @jsavin
    OwnerAuthor

    Additional data from the fix-848 branch work (2026-08-10): test_window_bridge's intermittent SIGSEGV was measured at 9/15 crashes on clean develop (2ce3d84) vs 10/15 with the #848 fix applied -- statistically identical, confirming pre-existing and unrelated to the headless_window_verbs change. When the binary completes it is 7/7 green every time. The crash occurs BEFORE any suite output is printed (earlier than the previously documented mid-run failure point). lldb attach was unavailable in the measurement environment ("cannot get permission to debug processes"), so no backtrace yet. At ~60-66 percent crash rate this binary is effectively flaky-by-default in any CI loop.

  2. added 2 commits that reference this issue on Aug 16, 2026
  3. jsavin commented on Aug 16, 2026

    @jsavin
    OwnerAuthor

    Additional characterization (2026-08-15, from #891 test windows)

    Fresh data on both binaries this issue names. The failure rate is worse than the 30-80% recorded above, and test_window_bridge in particular can look deterministic in a short sample.

    Standalone repeat runs

    Both built with CC=/usr/bin/clang, run directly (not under the suite), on an otherwise-loaded machine:

    Binary Clean runs
    test_callback_infrastructure 2/6
    test_window_bridge 1/6

    An earlier sample of test_window_bridge segfaulted 3/3, which reads as deterministic and is misleading — a longer sample shows it is not.

    Confirmed pre-existing, not from #891

    Verified on a purpose-built pristine worktree at 2b56ce868 (the #891 base), built and run identically:

    • test_window_bridge on pristine 2b56ce868: 4/5 runs segfaulted

    So the behavior is present on the untouched base commit. Neither file changed by #891 (tests/headless_sys_verbs.c, Common/source/shellsysverbs.c) is in test_window_bridge's link line — it links LANG_RUNTIME_SOURCES + ../frontier-cli/window_registry.c + ../frontier-cli/stubs/headless_shell.c (tests/Makefile:727), so a change to either could not reach that binary.

    Practical impact on full-suite runs

    The suite's set -e loop aborts at the first segfaulting binary (same masking behavior #827 notes), so a single hit takes down the whole make -C tests test invocation with Error 139. Across five full-suite runs during #891 this surfaced as test_callback_infrastructure three times and test_window_bridge once, with one fully clean run (exit 0). Which binary is hit varies with load, so the failure SET rather than a clean exit is the usable signal until this is fixed.

    Note on lldb

    lldb -b -o run -o bt remains sandbox-blocked in this environment (cannot get permission to debug processes), so no backtrace beyond the dbgeteof <- dbnormalizeaddress <- dbrefhandle <- hashresolvevalue stack already recorded above.

  4. added 2 commits that reference this issue on Aug 16, 2026
  5. jsavin commented on Aug 16, 2026

    @jsavin
    OwnerAuthor

    Characterization: which suite crashes is not stable under parallel test runs (2026-08-16)

    Observed while running unit failure-set comparisons for the --diff-roots unit (PR #897, merged as 6520a89). Adding to the characterization started here rather than opening a new issue, since it is a property of this crash family rather than a separate defect.

    The observation. Two runs of the same unit suite, at the same commit, with the same compiler:

    run suites reported crashing suite
    make -C tests test CC=/usr/bin/clang -j8 48 test_window_bridge
    make -C tests test CC=/usr/bin/clang (serial) 26 test_callback_infrastructure, on "Empty callback (no return statement)"

    test_window_bridge passes 1/1 when run standalone. test_callback_infrastructure passed 14/14 in the -j8 run — i.e. the suite that normally crashes completed cleanly, and a different one died instead.

    Reading. Under -j8 the runner interleaves build and execution, so more suites reach execution before the run aborts (48 vs 26 reported), and which suite is executing when the fault lands moves. That is consistent with this issue's existing "latent, state-dependent" framing: the trigger is process/session state rather than anything specific to the suite that happens to be running.

    Practical consequence, and the reason for this comment. A failure-set comparison is only meaningful from a serial run. A -j8 log naming test_window_bridge (or any other suite) reads as a new regression when it is the known crasher landing somewhere else. Anyone using unit failure-set equality as a merge gate should run serially for that comparison, or they will chase a phantom.

    No new reproducer beyond what is already recorded here; the system.macintosh pure-C result from 2026-08-15 (comment above) still stands as the narrowing evidence — that path does not crash without the interpreter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions