Skip to content

feat: run OCI containers as lab devices - #45

Open
Frando wants to merge 3 commits into
mainfrom
feat/containers
Open

feat: run OCI containers as lab devices#45
Frando wants to merge 3 commits into
mainfrom
feat/containers

Conversation

@Frando

@Frando Frando commented Jul 29, 2026

Copy link
Copy Markdown
Member

Description

Integration tests written against patchbay often need an auxiliary service running alongside the code under test: an ACME test server like Pebble, a database, a real DNS server. These are usually easier to run as a container than to embed. This lets patchbay run one as an ordinary lab node.

A container is a device. Lab::add_container(name, image) returns a builder that mirrors the device builder for wiring (uplink, iface, default_via, mtu) and adds container configuration on top. It creates the device namespace and veth uplink like any device, then runs the image joined to that namespace, so the service is reachable from other lab devices at the container's lab IP.

let pebble = lab
    .add_container("pebble", "ghcr.io/letsencrypt/pebble:latest")
    .uplink(net.id())
    .env("PEBBLE_VA_NOSLEEP", "1")
    .volume_ro(config_dir.join("pebble.json"), "/test/config/pebble.json")
    .args(["-config", "/test/config/pebble.json"])
    .ready_tcp(14000)
    .build()
    .await?;

// Reachable from any lab device.
let directory = format!("https://{}:14000/dir", pebble.ip().unwrap());

The returned Container dereferences to Device, so the device accessors work directly, and it adds container methods: exec, logs, stop, write_file/read_file, and copy_to/copy_from. Volumes come in read-write, read-only, and tmpfs forms. The container is removed when the handle is dropped.

Runtime and requirements

Containers run with podman, which must be on PATH. Podman is daemonless, so the podman run process, forked from inside the device's namespace worker, joins that namespace with --network=host and lands the container on the device's lab IP. Docker cannot do this rootlessly: its daemon runs in a separate namespace, so a docker container would join the daemon's namespace rather than the device's.

Running a container inside patchbay's rootless user namespace needs a full subuid/subgid range, the standard rootless-podman setup: newuidmap/newgidmap on PATH and an /etc/subuid and /etc/subgid entry for the user. The user-namespace bootstrap now maps that range via newuidmap/newgidmap when the host provides it, and falls back to the previous single-uid map otherwise. A single uid cannot unpack image layers or mount overlay; the range can. Router and device networking work either way, so this changes nothing for labs that do not use containers.

Why it took some plumbing

patchbay maps the invoking user to uid 0, so podman would otherwise treat itself as rootful and use unwritable system paths. Two internal but stable environment variables (_CONTAINERS_USERNS_CONFIGURED, _CONTAINERS_ROOTLESS_UID/GID) tell it it is the already-configured rootless namespace, so it uses the user's rootless storage and reuses already-pulled images. The image is pulled on the host before the container starts, because the device namespace has no route to a registry, and the pull runs on a thread with its own mount namespace, since podman makes its storage mounts private and the plain worker threads share the host mount namespace.

Notes and dead ends

Two alternatives to the environment-variable approach turned out not to be worth it, and the reasons are recorded next to the code:

  • Running patchbay as a non-zero uid so podman detects rootless on its own. The rootful-versus-rootless storage choice is keyed on euid, not the namespace, so this only trades one variable for a non-root uid that would break patchbay's own uid-0 assumptions.
  • Letting podman create its own nested user namespace for the container. The automatic, subuid-driven nesting fails because the inner user has no subuid delegation; an explicit --uidmap 0:0:65536 does work with --network=host, but it is redundant, since patchbay's namespace already maps the full range the container needs.

Making "run in an already-configured user namespace as uid 0" a first-class podman flag is an open upstream request (podman-container-tools/podman#7774).

The API borrows its shape from testcontainers-rs (bind mounts with an access mode, tmpfs, copy in and out) but stays smaller on purpose: ready_tcp covers the common readiness case rather than a full set of wait strategies, which can be added when a service needs them.

Testing

Unit tests cover the run-argument and mount rendering and the container-name sanitizing. Integration tests, gated on podman so they skip where it is absent, run nginx and fetch it from a separate client device, check that a read-only bind mount is visible, and round-trip write_file/read_file and copy_to/copy_from. The full library suite passes with the range-map user namespace, so existing behavior is unchanged.

🤖 Generated with Claude Code

Frando and others added 3 commits July 29, 2026 00:39
Integration tests using patchbay often need auxiliary services that are
easier to run as a container than to embed: an ACME test server (Pebble),
a database, a real DNS server. Add a way to run these as lab nodes.

A container is a device. `Lab::add_container(name, image)` returns a
builder that mirrors the device builder for wiring (uplink, iface,
default_via, mtu) and adds container config (env, args, run_arg,
ready_tcp, runtime). It creates the device namespace and veth uplink like
any device, then runs the image joined to that namespace, so the service
is reachable from other lab devices at the container's lab IP. The
returned Container derefs to Device and adds exec/logs/stop; it is removed
when the handle drops.

Containers run with podman, which must be on PATH. Podman is daemonless,
so the `podman run` process forked from inside the device's namespace
worker joins that namespace via --network=host. Docker cannot do this
rootlessly: its daemon runs in a separate namespace.

Making rootless podman work inside patchbay's user namespace needed three
things:

- userns: map the full subuid/subgid range via newuidmap/newgidmap when
  the host provides them (/etc/subuid, /etc/subgid entries), falling back
  to the previous single-uid map otherwise. A single uid cannot chown
  image files or mount overlay; the range can. This is the standard
  rootless-podman prerequisite and is generally useful for any nested
  container runtime.
- Tell podman it is the already-configured rootless namespace
  (_CONTAINERS_USERNS_CONFIGURED, _CONTAINERS_ROOTLESS_UID/GID), since
  patchbay maps the user to uid 0 and podman would otherwise use unwritable
  rootful paths. This also makes it reuse the user's rootless image store.
- Run the image pull and the non-run podman commands on a thread with its
  own mount namespace, because podman makes its storage mounts private and
  the plain blocking-pool threads share the host mount namespace. The pull
  runs on the host (the device namespace has no route to a registry); the
  run is forked from the device worker, which already has a private mount
  namespace.

Tested: unit tests for run-arg construction and name sanitizing, and an
integration test that runs nginx:alpine and fetches it from a client
device (gated on podman, skips where absent). The full lib suite passes
with the range-map userns, so existing behavior is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Integration services usually need their config or data handed in from the
host: a Pebble config file, a seeded database directory. Add builder
methods for it, following the testcontainers shape (bind mounts with an
access mode, plus tmpfs):

- `volume(host, container)` / `volume_ro(host, container)`: bind-mount a
  host path read-write or read-only.
- `tmpfs(container)`: mount a fresh tmpfs, for a writable scratch dir on an
  otherwise read-only image.

These render to `podman run --volume host:container[:ro]` / `--tmpfs`
before the image. `run_arg` remains the escape hatch for exotic cases such
as a `:z`/`:Z` SELinux label.

Also documents why the `_CONTAINERS_*` rootless environment variables are
used: podman keys "am I rootless" off euid with no containers.conf
override, `--userns=ns:path` is broken for rootless, and a supported flag
is an open request (podman-container-tools/podman#7774). They are internal but stable.

Tested: unit test for mount rendering, and an integration test that
bind-mounts a host file read-only and reads it back from inside the
container (gated on podman).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add methods to move files between the host and a running container,
following testcontainers' shape:

- copy_to(host, container) / copy_from(container, host): copy a file or
  directory in or out (podman cp).
- write_file(path, bytes) / read_file(path): write or read a single file
  without a bind mount, staged through a host temp file so binary content
  and any image (no shell needed) work.

Also records, on runtime_command, why podman must run as uid 0 with the
_CONTAINERS_* variables rather than as a non-zero uid: a non-zero uid
makes podman detect rootless and try to create its own nested user
namespace, which fails inside patchbay's ("cannot clone"), so
_CONTAINERS_USERNS_CONFIGURED is required regardless, and a non-root uid
would break patchbay's own uid-0 assumptions. Verified by reproducing
both cases under `unshare`.

Tested: an integration test round-trips write_file/read_file and
copy_to/copy_from against an alpine container (gated on podman).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant