Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/Build.Linux.Job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,12 @@ jobs:
fi
done

- name: Test Bubblewrap user-namespace denial (end-to-end)
working-directory: ${{ github.workspace }}
env:
LXC_EXEC: ${{ github.workspace }}/src/target/${{ matrix.target }}/release/lxc-exec
run: bash tests/scripts/run_bwrap_user_namespace_test.sh

- name: Upload binaries
if: matrix.upload_artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
Expand Down
29 changes: 22 additions & 7 deletions docs/backends/bwrap/bubblewrap-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,8 +179,8 @@ Bubblewrap creates a namespace-isolated process by:
3. Layering filesystem policy overrides (read-write, read-only, denied paths)
4. Setting up minimal `/dev`, `/proc`, and `/tmp`
5. Installing a seccomp filter that permits `socket()` and `socketpair()` only
for Unix-domain, IPv4, IPv6, and netlink address families, and disables
io_uring
for Unix-domain, IPv4, IPv6, and netlink address families, disables
io_uring, and blocks creation of new user namespaces
6. Clearing the environment and applying only requested variables
7. Dropping all workload capabilities with `--cap-drop ALL`, including for root callers
8. Executing the command via `sh -c`
Expand Down Expand Up @@ -209,22 +209,34 @@ a dedicated thread that lives as long as the process, so a sandbox started from
a pooled or short-lived thread (for example a Tokio `spawn_blocking` worker)
survives that thread and dies only with the embedding process.

The address-family filter is always active. Attempts to create VSOCK, packet,
The seccomp filter is always active. Attempts to create VSOCK, packet,
or other unlisted socket families fail with `EPERM`. Because io_uring can
create sockets from submission-queue entries that seccomp cannot inspect, the
filter also makes `io_uring_setup`, `io_uring_enter`, and `io_uring_register`
fail with `ENOSYS`; runtimes that probe for io_uring fall back to their
non-io_uring paths. Other system calls are not restricted by this filter.
Filter construction, writing, or descriptor inheritance failures abort the
launch instead of running without the filter.
non-io_uring paths. Filter construction, writing, or descriptor inheritance
failures abort the launch instead of running without the filter.

The filter also stops the workload from creating user namespaces. A new user
namespace gives its creator full capabilities inside it, which exposes kernel
interfaces that are otherwise unavailable to the workload. `unshare` and
`clone` calls that request `CLONE_NEWUSER` fail with `EPERM`, so commands such
as `unshare -Ur` fail. `clone3` passes its flags through a pointer that seccomp
cannot inspect, so every `clone3` call fails with `ENOSYS`; glibc then falls
back to `clone`, so threads and child processes still start. Programs that
call `clone3` directly must make the same fallback. Tools that create their
own user namespace, such as a nested `bwrap`, rootless Podman, or Chromium's
sandbox, do not work inside the sandbox; configure them not to create one
(for example, Chromium's `--no-sandbox`). No request option re-enables user
namespaces. Other system calls are not restricted by this filter.

On x86-64 hosts, native and x32 workloads retain their normal syscall ABI, and
i386 workloads using the direct `socket` and `socketpair` syscalls receive the
same family filtering. The legacy i386 `socketcall` interface passes the socket
arguments through a pointer that seccomp cannot inspect, so its socket-creation
and socket-pair subcalls fail with `EPERM`; its other subcalls remain available.
On AArch64 hosts, native and AArch32 direct socket syscalls receive the same
family filtering.
family filtering. User-namespace creation is blocked on each of these ABIs.

MXC supplies one generated program through the legacy `--seccomp FD` option.
Bubblewrap rejects combining that option with `--add-seccomp-fd`, so any future
Expand Down Expand Up @@ -1173,6 +1185,9 @@ These failures carry the `backend_error` code and exit `-1`.
IPv6 destinations remain unreachable until slirp supports IPv6 here.
`runtimeConfig.networkProxy` restricts direct egress to the configured
loopback proxy endpoint; hostname policy belongs to that external proxy.
- **No nested user namespaces** β€” workloads cannot create user namespaces, so
nested sandboxes and rootless container tools do not run inside Bubblewrap
(see [How It Works](#how-it-works)).
- **No container lifecycle operations**: Bubblewrap's `ScriptRunner`
supports create-and-run execution, not `StatefulSandboxBackend` lifecycle calls.
- **Provider-loss detection is exit-based** β€” in the private-namespace modes a
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ test jobs only ever `download-artifact`.
|-----|--------------|
| `dependency-feed-check` | Resolves the locked crate graph through the public `MxcDependencies` feed. Gates the builds. |
| `windows` | `Build.Windows.Job.yml` β€” x64 + arm64 release build, unit tests, uploads `wxc-binaries-<target>` and the separate `telemetry-test-executor-<target>`. The x64 release artifact also includes SDK MicroVM tests and the complete signed NVX bundle staged by `aci_edge_sandboxes`. |
| `linux` | `Build.Linux.Job.yml` β€” x64 + arm64 release build, unit tests including Bubblewrap seccomp, argument generation, host tool lookup, and version checks, Bubblewrap SDK streaming/PTY tests, `wxc_e2e_tests`, uploads `lxc-binaries-<target>`. |
| `linux` | `Build.Linux.Job.yml` β€” x64 + arm64 release build, unit tests including Bubblewrap seccomp, argument generation, host tool lookup, and version checks, Bubblewrap SDK streaming/PTY tests, `wxc_e2e_tests`, Bubblewrap proxy-networking, workload-capability, and user-namespace-denial CLI tests, uploads `lxc-binaries-<target>`. |
| `macos` | `Build.MacOS.Job.yml` β€” arm64 release build, unit tests, Seatbelt SDK streaming/PTY tests, `wxc_e2e_tests`, uploads `mxc-binaries-aarch64-apple-darwin`. |
| `windows-sdk-bundle` | `Package.WindowsSdk.TestBundle.Job.yml` β€” uploads `windows-sdk-test-bundle-<target>` for x64 + arm64. |
| `test-nightly` | Calls the matrix job with `plan: nightly`. Runs on every schedule tick and on a `nightly` dispatch. |
Expand Down Expand Up @@ -319,7 +319,7 @@ get fixed or wired.
| Backend | Status | Notes |
|---------|--------|-------|
| ProcessContainer | βœ… Good | Requires native PSEC; verifies public SDK streaming, filesystem policy and aliases, CLI contract boundaries, DOS/NT final paths, egress/loopback/proxy rules, the functional UI matrix, HANDLES, and global atoms. Unsupported host capabilities must reject requests. Privilege removal uses a host-token control. Application workload tests remain manual-only. |
| Bubblewrap | βœ… Good | The scheduled CLI suite checks controlling-terminal isolation with private PTY fixtures, including redirected stdio. Private resolver fixtures check host IPC isolation, descriptor-backed snapshots, symlink layouts, filesystem-policy precedence, and nonregular-file rejection. The Linux build explicitly runs the resolver safety unit tests. It also checks SDK pipes under a controlling terminal, cancellation without parent-death protection, and private-PTY input/output and resizing. These checks do not require enabling legacy TIOCSTI. The proxy, firewall, and version-gate suites also run with host tools missing from `PATH`, so tool discovery and launch must find them in the standard system directories. |
| Bubblewrap | βœ… Good | The scheduled CLI suite checks controlling-terminal isolation with private PTY fixtures, including redirected stdio. Private resolver fixtures check host IPC isolation, descriptor-backed snapshots, symlink layouts, filesystem-policy precedence, and nonregular-file rejection. The Linux build explicitly runs the resolver safety unit tests. It also checks SDK pipes under a controlling terminal, cancellation without parent-death protection, and private-PTY input/output and resizing. These checks do not require enabling legacy TIOCSTI. The scheduled CLI suite and the Linux build check that workloads cannot create user namespaces in isolated, firewall, and proxy modes, while threads and child processes still start. The proxy, firewall, and version-gate suites also run with host tools missing from `PATH`, so tool discovery and launch must find them in the standard system directories. |
| LXC | βœ… Good | Some networking tests fail on distros other than Ubuntu 24.04; seems to be an issue with MXC. The in-process streaming handle is covered at PR time by `lxc-e2e.yml` instead, because this matrix runs prebuilt binaries. |
| WSLC | βœ… Good | Might have to retry hung jobs - this is an issue with overzealous agent reclaiming. |
| IsolationSession | βœ… Good | Runs the one-shot and state-aware suites, isolated lifecycle ETW capture, the Rust SDK in-process and helper tests, the C# end-to-end tests, the Node SDK suite and the COM apartment probe. Fails when the host cannot run isolation sessions, when a suite executes nothing, or when the run changes the set of local accounts. |
Expand Down
147 changes: 145 additions & 2 deletions src/mxc-sdk/src/backends/bubblewrap/common/seccomp_filter.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

//! Builds Bubblewrap's always-on address-family seccomp filter.
//! Builds Bubblewrap's always-on seccomp filter.

#[cfg(target_os = "linux")]
use std::fs::File;
Expand All @@ -14,7 +14,6 @@ use std::os::fd::{AsRawFd, FromRawFd};
use super::proxy_network::above_stdio;

const BPF_LD_W_ABS: u16 = 0x20;
#[cfg(any(target_arch = "x86_64", test))]
const BPF_ALU_AND_K: u16 = 0x54;
const BPF_JMP_JEQ_K: u16 = 0x15;
const BPF_RET_K: u16 = 0x06;
Expand Down Expand Up @@ -65,10 +64,30 @@ const ARM_SOCKET: u32 = 281;
#[cfg(any(target_arch = "aarch64", test))]
const ARM_SOCKETPAIR: u32 = 288;

#[cfg(any(target_arch = "x86_64", test))]
const X86_64_CLONE: u32 = 56;
#[cfg(any(target_arch = "x86_64", test))]
const X86_64_UNSHARE: u32 = 272;
#[cfg(any(target_arch = "x86_64", test))]
const I386_CLONE: u32 = 120;
#[cfg(any(target_arch = "x86_64", test))]
const I386_UNSHARE: u32 = 310;
#[cfg(any(target_arch = "aarch64", test))]
const AARCH64_UNSHARE: u32 = 97;
#[cfg(any(target_arch = "aarch64", test))]
const AARCH64_CLONE: u32 = 220;
#[cfg(any(target_arch = "aarch64", test))]
const ARM_CLONE: u32 = 120;
#[cfg(any(target_arch = "aarch64", test))]
const ARM_UNSHARE: u32 = 337;

const IO_URING_SETUP: u32 = 425;
const IO_URING_ENTER: u32 = 426;
const IO_URING_REGISTER: u32 = 427;
const IO_URING_SYSCALLS: [u32; 3] = [IO_URING_SETUP, IO_URING_ENTER, IO_URING_REGISTER];
const CLONE3: u32 = 435;

const CLONE_NEWUSER: u32 = 0x1000_0000;

const AF_UNIX: u32 = 1;
const AF_INET: u32 = 2;
Expand Down Expand Up @@ -197,6 +216,8 @@ fn build_x86_program() -> Vec<Instruction> {
let x86_64_socketpair = program.len();
program.push(Instruction::jump(BPF_JMP_JEQ_K, X86_64_SOCKETPAIR));
let mut io_uring_checks = push_io_uring_checks(&mut program);
let mut user_namespace_checks = UserNamespaceChecks::default();
user_namespace_checks.push(&mut program, X86_64_UNSHARE, X86_64_CLONE);
program.push(Instruction::statement(BPF_RET_K, SECCOMP_RET_ALLOW));

let i386_start = program.len();
Expand All @@ -209,6 +230,7 @@ fn build_x86_program() -> Vec<Instruction> {
let i386_socketcall = program.len();
program.push(Instruction::jump(BPF_JMP_JEQ_K, I386_SOCKETCALL));
io_uring_checks.extend(push_io_uring_checks(&mut program));
user_namespace_checks.push(&mut program, I386_UNSHARE, I386_CLONE);
program.push(Instruction::statement(BPF_RET_K, SECCOMP_RET_ALLOW));

let socketcall_check = program.len();
Expand Down Expand Up @@ -244,6 +266,7 @@ fn build_x86_program() -> Vec<Instruction> {
set_true_target(&mut program, socket_check, family_check);
}
append_io_uring_denial(&mut program, &io_uring_checks);
user_namespace_checks.append_denial(&mut program);

program
}
Expand All @@ -268,6 +291,8 @@ fn build_aarch64_program() -> Vec<Instruction> {
let aarch64_socketpair = program.len();
program.push(Instruction::jump(BPF_JMP_JEQ_K, AARCH64_SOCKETPAIR));
let mut io_uring_checks = push_io_uring_checks(&mut program);
let mut user_namespace_checks = UserNamespaceChecks::default();
user_namespace_checks.push(&mut program, AARCH64_UNSHARE, AARCH64_CLONE);
program.push(Instruction::statement(BPF_RET_K, SECCOMP_RET_ALLOW));

let arm_start = program.len();
Expand All @@ -278,6 +303,7 @@ fn build_aarch64_program() -> Vec<Instruction> {
let arm_socketpair = program.len();
program.push(Instruction::jump(BPF_JMP_JEQ_K, ARM_SOCKETPAIR));
io_uring_checks.extend(push_io_uring_checks(&mut program));
user_namespace_checks.push(&mut program, ARM_UNSHARE, ARM_CLONE);
program.push(Instruction::statement(BPF_RET_K, SECCOMP_RET_ALLOW));

let family_check = append_family_filter(&mut program);
Expand All @@ -290,6 +316,7 @@ fn build_aarch64_program() -> Vec<Instruction> {
set_true_target(&mut program, socket_check, family_check);
}
append_io_uring_denial(&mut program, &io_uring_checks);
user_namespace_checks.append_denial(&mut program);

program
}
Expand All @@ -315,6 +342,50 @@ fn append_io_uring_denial(program: &mut Vec<Instruction>, checks: &[usize]) {
}
}

#[derive(Default)]
struct UserNamespaceChecks {
flag_checks: Vec<usize>,
clone3_checks: Vec<usize>,
}

impl UserNamespaceChecks {
fn push(&mut self, program: &mut Vec<Instruction>, unshare: u32, clone: u32) {
for number in [unshare, clone] {
self.flag_checks.push(program.len());
program.push(Instruction::jump(BPF_JMP_JEQ_K, number));
}
self.clone3_checks.push(program.len());
program.push(Instruction::jump(BPF_JMP_JEQ_K, CLONE3));
}

fn append_denial(self, program: &mut Vec<Instruction>) {
// clone3 passes its flags in a structure that seccomp cannot read, so it
// fails as unimplemented and callers fall back to clone.
let deny_clone3 = program.len();
program.push(Instruction::statement(BPF_RET_K, errno(ENOSYS)));
for check in self.clone3_checks {
set_true_target(program, check, deny_clone3);
}

// The kernel ignores the upper half of clone's flags and rejects unshare
// flags it does not know, so the low word decides.
let flag_check = program.len();
program.extend([
Instruction::statement(BPF_LD_W_ABS, SECCOMP_DATA_ARG0_LOW_OFFSET),
Instruction::statement(BPF_ALU_AND_K, CLONE_NEWUSER),
]);
let without_new_user = program.len();
program.push(Instruction::jump(BPF_JMP_JEQ_K, 0));
program.push(Instruction::statement(BPF_RET_K, errno(EPERM)));
let allow = program.len();
program.push(Instruction::statement(BPF_RET_K, SECCOMP_RET_ALLOW));
set_true_target(program, without_new_user, allow);
for check in self.flag_checks {
set_true_target(program, check, flag_check);
}
}
}

fn append_family_filter(program: &mut Vec<Instruction>) -> usize {
let family_check = program.len();

Expand Down Expand Up @@ -358,6 +429,10 @@ mod tests {

const AF_PACKET: u64 = 17;
const AF_VSOCK: u64 = 40;
const SIGCHLD: u64 = 17;
const CLONE_FILES: u64 = 0x400;
const CLONE_NEWNS: u64 = 0x2_0000;
const GLIBC_THREAD_FLAGS: u64 = 0x003d_0f00;

// Syscall and audit values follow Linux v6.18's UAPI headers and syscall tables:
// https://github.com/torvalds/linux/tree/v6.18
Expand All @@ -374,6 +449,20 @@ mod tests {
(NativeArchitecture::Aarch64, 0x4000_0028, 281, 288),
];

const USER_NAMESPACE_ABIS: [(NativeArchitecture, u32, u32, u32, u32); 5] = [
(NativeArchitecture::X86_64, 0xc000_003e, 272, 56, 435),
(
NativeArchitecture::X86_64,
0xc000_003e,
0x4000_0110,
0x4000_0038,
0x4000_01b3,
),
(NativeArchitecture::X86_64, 0x4000_0003, 310, 120, 435),
(NativeArchitecture::Aarch64, 0xc000_00b7, 97, 220, 435),
(NativeArchitecture::Aarch64, 0x4000_0028, 337, 120, 435),
];

fn evaluate(program: &[Instruction], data: &[u8; 64]) -> u32 {
let mut accumulator = 0_u32;
let mut pc = 0_usize;
Expand Down Expand Up @@ -532,6 +621,60 @@ mod tests {
}
}

#[test]
fn denies_new_user_namespaces_on_every_supported_abi() {
let new_user = u64::from(CLONE_NEWUSER);
for (native, audit, unshare, clone, _) in USER_NAMESPACE_ABIS {
for number in [unshare, clone] {
for flags in [
new_user,
new_user | CLONE_NEWNS | SIGCHLD,
(0xffff_ffff << 32) | new_user,
] {
assert_eq!(
result(native, audit, number, flags),
0x0005_0001,
"flags {flags:#x} should be denied for syscall {number:#x} on arch {audit:#x}"
);
}
}
}
}

#[test]
fn allows_clone_and_unshare_without_a_new_user_namespace() {
let new_user = u64::from(CLONE_NEWUSER);
for (native, audit, unshare, clone, _) in USER_NAMESPACE_ABIS {
for (number, flags) in [
(clone, SIGCHLD),
(clone, GLIBC_THREAD_FLAGS),
(clone, new_user << 32),
(unshare, CLONE_FILES),
(unshare, CLONE_NEWNS),
] {
assert_eq!(
result(native, audit, number, flags),
0x7fff_0000,
"flags {flags:#x} should be allowed for syscall {number:#x} on arch {audit:#x}"
);
}
}
}

#[test]
fn denies_clone3_as_unimplemented_on_every_supported_abi() {
for (native, audit, _, _, clone3) in USER_NAMESPACE_ABIS {
assert_eq!(result(native, audit, clone3, 0), 0x0005_0026);
for neighbor in [clone3 - 1, clone3 + 1] {
assert_eq!(
result(native, audit, neighbor, 0),
0x7fff_0000,
"syscall {neighbor:#x} should stay allowed on arch {audit:#x}"
);
}
}
}

#[test]
fn denies_i386_socketcall_creation_subcalls() {
for subcall in [1, 8] {
Expand Down
2 changes: 1 addition & 1 deletion tests/scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ overrides are assigned only to child processes, never to the invoking shell.

| Script | Description | Extra prerequisites |
|--------|-------------|---------------------|
| `run_bwrap_all_tests.sh` | All Bubblewrap tests | `lxc-exec`, `bwrap`, Python 3 for terminal isolation |
| `run_bwrap_all_tests.sh` | All Bubblewrap tests | `lxc-exec`, `bwrap`, Python 3 |
| `run_bwrap_capabilities_test.sh` | Workload capability removal, refusal when the capability drop is blocked, read-only mount protection, firewall tamper resistance, and supervisor networking in all three modes; CI runs as both root and non-root | `lxc-exec`, Python 3, Bubblewrap network prerequisites, x86_64 or aarch64 |
| `run_lxc_all_tests.sh` | All LXC tests | `lxc-exec`, LXC stack, root |
| `run_seatbelt_all_tests.sh` | All Seatbelt tests; missing prerequisites are failures rather than skips | macOS, `mxc-exec-mac`, unprivileged user, backend prerequisites |
Expand Down
Loading
Loading