Skip to content

[Bubblewrap] Unblock the watchdog's signals in spawned children - #1804

Open
Skye Soss (Skyb0rg007) wants to merge 4 commits into
microsoft:mainfrom
Skyb0rg007:bwrap-unblock-fatal-signals
Open

Skye Soss (Skyb0rg007) wants to merge 4 commits into
microsoft:mainfrom
Skyb0rg007:bwrap-unblock-fatal-signals

Conversation

@Skyb0rg007

@Skyb0rg007 Skye Soss (Skyb0rg007) commented Nov 18, 2026 •

Copy link
Copy Markdown

📖 Description

Warning: the below description is entirely AI-written. I can revise by-hand if desired. The code in this review is human-reviewed, and if you want a human-description of the issue take a look at #1764.

lxc-exec's first act in main() is signal_cleanup::install(), which blocks SIGHUP, SIGTERM and SIGINT with pthread_sigmask and starts a thread that sigwait()s for them, so that a fatal signal destroys the active LXC container before the process exits. The mask is installed before any other thread exists so that every thread inherits it and the watchdog is the only one that can receive those signals.

The signal mask is also inherited by child processes: it survives both fork() and execve(), and std::process::Command deliberately leaves it alone ("Inherit the signal mask from the parent rather than resetting it" in std's unix spawn path, which has a test pinning that behaviour). Unless a spawn site undoes it, every process lxc-exec starts, and everything those processes start in turn, begins life with SIGHUP, SIGTERM and SIGINT blocked. Few programs reset their mask on startup, so for them those signals are simply never delivered: they stay pending until the process exits. A shell's trap installs a handler but does not touch the mask, so traps on them never fire either.

The LXC backend already handles this: lxc_bindings unblocks the three signals in a pre_exec hook (or through PtyOptions::unblock_signals / LivePty for the PTY paths) for every lxc-attach it spawns. The Bubblewrap backend never did, so with lxc-exec as the executor:

  • The sandboxed workload ignores SIGTERM, SIGINT and SIGHUP. bwrap passes its own mask on to the workload, so timeout 5 cmd, kill <pid>, a supervisor's SIGTERM, or Ctrl-C delivered to the workload's process group (Pipes and PTY modes put bwrap in its own group) has no effect; only SIGKILL stops it. Commands that rely on those signals internally -- timeout(1), a shell script that kills its own background jobs, trap ... TERM cleanup handlers -- hang or skip their cleanup. lxc-exec's own --timeout still works because it uses SIGKILL.
  • The private-network supervisor (unshare ... sh -c SUPERVISOR_SCRIPT) installs trap 'kill "$slirp_pid"' TERM INT so that slirp4netns is stopped with it, but that trap can never run. mxc's own teardown is unaffected because it SIGKILLs the supervisor's process group, but a TERM or INT sent to the group from outside is silently dropped.

The PTY path was not covered either: bwrap_runner passes an empty unblock list to LivePty::attach, which only unblocks SIGWINCH.

Fix it at the source rather than per call site:

  • signal_cleanup, which owns the mask, now exports the set as WATCHDOG_SIGNALS and install() builds its mask from it, so there is a single definition of what is blocked.
  • signal_cleanup::unblock_in_child(&mut Command) registers the pre_exec hook that unblocks that set. It replaces lxc_bindings' private unblock_fatal_signals, and the two hand-written UNBLOCK constants in lxc_bindings now use WATCHDOG_SIGNALS, so the LXC and Bubblewrap backends can no longer drift from the watchdog.
  • bwrap_runner calls it on the bwrap command before stdio is set up, so it applies in the Pipes, Inherit and PTY modes alike.
  • proxy_network calls it on the supervisor's unshare command.

The hook is applied unconditionally, not only when the watchdog is installed: a library host that blocks these signals for its own purposes would leak the same mask into the sandbox, and a sandboxed workload should always start with them deliverable. pre_exec runs after Command's own child setup, and LivePty::attach registers its hook afterwards, so its SIGWINCH unblock still applies on top.

Probes that mxc kills with SIGKILL on timeout (the bwrap version probe, the dependency probes) are left alone: nothing in them relies on the three signals and they never outlive lxc-exec.

A new Bubblewrap characterization test has the workload trap each of SIGHUP, SIGINT and SIGTERM, send it to itself, and asserts that every trap ran. It checks delivery rather than the mask, so it also catches the signals being ignored rather than blocked. It fails without this change (no trap runs) and passes with it.

🔗 References

Closes #1764

🔍 Validation

Ran the fixed version of the program, using the example in the linked issue, and ran the test.

✅ Checklist

  • Signed the Contributor License Agreement
  • Linked to an issue
  • For features or changes requiring a spec: linked the issue with maintainer-applied feature-accepted
  • For implementation: linked the required approved and merged design/API spec, or explained why no spec is required
  • Feature issue remains open unless this PR completes all agreed implementation, validation, and documentation
  • Updated documentation (if applicable)
  • Updated Copilot instructions (if build, architecture, or conventions changed)
  • If this PR changes Cargo.lock, the dependency-feed-check check passes (see pull request builds)

📋 Issue Type

  • Bug fix
  • Feature
  • Task
Microsoft Reviewers: Open in CodeFlow

@Skyb0rg007
Skye Soss (Skyb0rg007) requested a review from a team as a code owner November 18, 2026 18:57
Copilot AI balanced review requested due to automatic review settings November 18, 2026 18:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Outstanding correctness and design issues remain: the LXC streaming spawn path still omits the signal-unblocking helper, Bubblewrap retains a backend-layer dependency, and regression coverage does not exercise PTY, inherited-stdio, or proxy-network paths.

4 open issues

Review effort: Lite

lxc-exec's first act in main() is signal_cleanup::install(), which
blocks SIGHUP, SIGTERM and SIGINT with pthread_sigmask and starts a
thread that sigwait()s for them, so that a fatal signal destroys the
active LXC container before the process exits. The mask is installed
before any other thread exists so that every thread inherits it and
the watchdog is the only one that can receive those signals.

The signal mask is also inherited by child processes: it survives both
fork() and execve(), and std::process::Command deliberately leaves it
alone ("Inherit the signal mask from the parent rather than resetting
it" in std's unix spawn path, which has a test pinning that
behaviour). Unless a spawn site undoes it, every process lxc-exec
starts, and everything those processes start in turn, begins life with
SIGHUP, SIGTERM and SIGINT blocked. Few programs reset their mask on
startup, so for them those signals are simply never delivered: they
stay pending until the process exits. A shell's `trap` installs a
handler but does not touch the mask, so traps on them never fire
either.

The LXC backend already handles this: lxc_bindings unblocks the three
signals in a pre_exec hook (or through PtyOptions::unblock_signals /
LivePty for the PTY paths) for every lxc-attach it spawns. The
Bubblewrap backend never did, so with `lxc-exec` as the executor:

  * The sandboxed workload ignores SIGTERM, SIGINT and SIGHUP. bwrap
    passes its own mask on to the workload, so `timeout 5 cmd`,
    `kill <pid>`, a supervisor's SIGTERM, or Ctrl-C delivered to the
    workload's process group (Pipes and PTY modes put bwrap in its own
    group) has no effect; only SIGKILL stops it. Commands that rely on
    those signals internally -- timeout(1), a shell script that kills
    its own background jobs, `trap ... TERM` cleanup handlers -- hang
    or skip their cleanup. lxc-exec's own --timeout still works
    because it uses SIGKILL.
  * The private-network supervisor (`unshare ... sh -c
    SUPERVISOR_SCRIPT`) installs `trap 'kill "$slirp_pid"' TERM INT`
    so that slirp4netns is stopped with it, but that trap can never
    run. mxc's own teardown is unaffected because it SIGKILLs the
    supervisor's process group, but a TERM or INT sent to the group
    from outside is silently dropped.

The PTY path was not covered either: bwrap_runner passes an empty
unblock list to LivePty::attach, which only unblocks SIGWINCH.

Fix it at the source rather than per call site:

  * signal_cleanup, which owns the mask, now exports the set as
    WATCHDOG_SIGNALS and install() builds its mask from it, so there is
    a single definition of what is blocked.
  * signal_cleanup::unblock_in_child(&mut Command) registers the
    pre_exec hook that unblocks that set. It replaces lxc_bindings'
    private unblock_fatal_signals, and the two hand-written UNBLOCK
    constants in lxc_bindings now use WATCHDOG_SIGNALS, so the LXC and
    Bubblewrap backends can no longer drift from the watchdog.
  * bwrap_runner calls it on the bwrap command before stdio is set
    up, so it applies in the Pipes, Inherit and PTY modes alike.
  * proxy_network calls it on the supervisor's `unshare` command.

The hook is applied unconditionally, not only when the watchdog is
installed: a library host that blocks these signals for its own
purposes would leak the same mask into the sandbox, and a sandboxed
workload should always start with them deliverable. pre_exec runs
after Command's own child setup, and LivePty::attach registers its
hook afterwards, so its SIGWINCH unblock still applies on top.

Probes that mxc kills with SIGKILL on timeout (the bwrap version
probe, the dependency probes) are left alone: nothing in them relies
on the three signals and they never outlive lxc-exec.

A new Bubblewrap characterization test has the workload trap each of
SIGHUP, SIGINT and SIGTERM, send it to itself, and asserts that every
trap ran. It checks delivery rather than the mask, so it also catches
the signals being ignored rather than blocked. It fails without this
change (no trap runs) and passes with it.

Assisted-By: claude-opus-5.5
Bubblewrap imported lxc_common::signal_cleanup only for its child
signal-mask hook, which crossed backend boundaries. The signal list and
unblock_in_child now live in mxc_common::termination_signals. The LXC
watchdog builds its mask from the same list, so there is still a single
definition of what it blocks.

LxcContainer::attach_spawn, the LXC streaming Pipes path, now unblocks
the signals as well. Its doc comment said a host application's mask was
deliberately left alone. That disagreed with the PTY path, which already
unblocked them, and with the policy that a sandboxed workload always
starts with these signals deliverable.

A shared streaming test, run from the Bubblewrap and LXC suites, blocks
SIGHUP, SIGINT and SIGTERM on the calling thread, as a host or lxc-exec's
watchdog would, then checks that the workload's traps for all three run
under both spawn and spawn_with_pty.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI lite review requested due to automatic review settings November 18, 2026 23:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Pipes regression test does not reliably exercise the blocked-signal scenario because the singleton spawner thread may retain an unblocked mask, allowing the test to pass without validating the required hook.

1 open issue
4 resolved since last review

Review effort: Lite

Bubblewrap spawns every sandbox from the process-lifetime spawner
thread, which inherits the signal mask of whichever thread starts it.
Blocking the signals only on the test thread therefore proved nothing
when another test in the same binary had started the spawner first: the
spawn ran with an unblocked mask and passed even without the fix.

The test now re-runs itself as a child process started with SIGHUP,
SIGINT and SIGTERM blocked. A blocked mask survives execve, so every
thread in the child, the spawner included, starts with them blocked, as
under lxc-exec's watchdog. The child first checks that its mask really
blocks all three, then runs the Pipes and PTY checks.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI lite review requested due to automatic review settings November 18, 2026 23:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The change still has an active moderate test-isolation issue because the re-executed child uses a different container name, allowing leaked containers to go undetected.

1 open issue
1 resolved since last review

Review effort: Lite

Comment thread src/mxc-sdk/tests/streaming_lxc.rs Resolved
The signal-delivery test re-runs itself in a child process, and
container_name() includes the PID, so the parent's assert_container_released
checked a name that never existed. A leak was still caught, because the
child runs the same check under its own name and the parent requires the
child to succeed, but the parent's check was dead code that read as if it
mattered.

assert_delivered_despite_host_mask now returns whether this process ran
the workloads, and the LXC test checks cleanup only there.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI lite review requested due to automatic review settings November 18, 2026 23:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Final approval should wait for Linux runtime validation across the affected Bubblewrap, LXC PTY/pipe, and proxy-network supervisor paths to confirm signal delivery and cleanup behavior.

No open issues 🎉

1 resolved since last review

Review effort: Lite

@Skyb0rg007
Skye Soss (Skyb0rg007) requested a balanced review from Copilot November 18, 2026 23:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bwrap] Bubblewrap backend masks SIGTERM and other POSIX signals

2 participants