Skip to content
EmiBSDPublic

About

EmiBSD's own userland: the programs, libraries and configuration above the Kernel, written from scratch in Rust

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

58 Commits

Folders and files

Repository files navigation

EmiBSD Userland

Ferffy, the EmiBSD mascot: a centolla, the spiky orange crab of the southern seas, in front of a shield, above the EmiBSD name

Ferffy, the EmiBSD mascot: a centolla, the child of Ferris (Rust's crab) and Puffy (OpenBSD's pufferfish).

The base system above the kernel: one tree, every program placed by what it does.

License: ISC Status: experimental Website: www.emibsd.org

Not for production. EmiBSD is an experiment. It runs only in QEMU, and its crypto, network and storage code is unaudited. Do not use it to protect real data or real networks.

The monorepo of EmiBSD's userland: the programs, libraries and configuration that run above the Kernel. Phase 3 of the project. Its base is OpenBSD's base system at the pin, the same C the kernel runs today, in an order of our own: every program placed by what it does instead of where thirty years of accretion left it. The written programs grow from that base one at a time, as the kernel's redesign replaces its inherited tree subsystem by subsystem.

Where it sits

flowchart LR
    obsd["OpenBSD-src (mirror)"] -- "the base system's sources, at the pin;<br/>the imports (ssh, LibreSSL, ...)" --> ul["Userland"]
    lz["LandingZone"] -. "the pin" .-> ul
    dev["DevTools"] -- "clang, lld, libc++; rustc, cargo, std" --> ul
    ul -- "the base sets" --> os["OS"]
    ul -. "sysroot: libc, crt, headers" .-> dev
Loading

The pin is LandingZone's, because it fixes the ABI the kernel implements; the toolchains come from DevTools; OS assembles the sets with the kernel into a release.

Rules (the user's decisions of 2026-10-23, revised on 2026-11-04)

  • Three classes of program (docs/PROGRAM.md; the user's decision of 2026-11-04, replacing the two classes of 2026-10-23). Inherited: the base system's C at LandingZone's pin, from the EmiBSD/OpenBSD-src mirror, every notice kept whole, placed in this layout and changed only by patches/, never rewritten in place: the base EmiBSD runs. Imported: cryptography, the security and network protocols with a living upstream and a CVE history, and the legacy giants (ssh, LibreSSL, unbound, smtpd, vi, tmux, mandoc, ncurses, ...) are compiled from their upstream at the pin of their UPSTREAM.toml, never rewritten; our patches are a numbered series, ideally empty. Written: from scratch, under a definition, in Rust, one program at a time where a program is worth it; nothing is a port or a translation of another system's program. just status counts the three. No program is imported today: the entries once imported (ssh, LibreSSL, ncurses, unbound, ...) come from the same mirror at the same pin and are inherited since 2026-11-04, their CPE kept in UPSTREAM.toml for their CVE history; the class stays for an upstream outside the base system's tree. In the userland __OpenBSD__ becomes __EmiBSD__ (2026-11-04): the inherited C sees only the macro the compiler predefines, and the build renames every __OpenBSD__ in its copy of the tree to __EmiBSD__.
  • The first written programs were a proof of concept. cli, sys, ls and cp showed the written class and its grammar; their code was dropped on 2026-11-08. ls and cp are inherited here like every other program; the documents of the two shared crates stay at designed. main carries no Rust code, no workspace and no toolchain pin until a written program returns.
  • Rust for what is written; C only where the ABI demands it, assembly only in libc's syscall stubs. #![forbid(unsafe_code)] in every program; unsafe only in base/lib/libc, base/libexec/rtld and base/lib/crates/sys (the thin FFI the programs share), each site with a written proof, as the kernel's hal.
  • One grammar for every program (docs/PROGRAM.md, "The common grammar"; the user's decision of 2026-10-24, taken with ls): a program is a query over records with a schema; -o name|table|wide|json|yaml|custom-columns=... picks the renderer, --sort-by=.field the order, --no-headers the table's header; --request takes the whole invocation as one JSON document (a file, or inline) and --show-request prints it; under -o json the diagnostics are records too (severity, code, subject), one per line on standard error, so a person and a machine drive the same program the same way. POSIX's short flags keep their meaning, and to a pipe the output is POSIX's byte for byte; to a terminal the defaults are modern (ISO dates, human sizes, headers). BSD-only flags nobody remembers are not kept.
  • LLVM and Rust, nothing GPL. The toolchains live in DevTools: upstream LLVM at a pinned release, one compiler for the C and the backend of rustc, and rust-lang/rust at a pinned stable tag, so the *-unknown-openbsd targets, tier 3 in Rust, get their std from a stable release's source, not from nightly. Nothing of them comes from OpenBSD's tree.
  • Upstreams are mirrored in the organisation (EmiBSD/OpenBSD-src, EmiBSD/rust, ...), so an upstream that moves or disappears does not take the build with it. A tarball upstream gets a repository where each release is one commit. Fetched sources live under upstream/, gitignored and read-only; a build applies patches/ to a copy under target/, never to upstream/. tools/upstream.py fetches, verifies (commit, SHA-256, signature) and pins.
  • just and make. Every program carries a justfile that imports tools/program.just; the root justfile runs over all of them. cargo builds the written programs; the inherited C is built by its own Makefiles with the base system's own make (the user's decision of 2026-11-04): just host-make builds that make for the development host from its inherited src/ with DevTools' clang, without any make, and it reads the share/mk of the same pin. Nothing else is installed for it; tools/host/compat.h gives the host tools what macOS lacks, and is never shipped. just world arm64 (tools/world.py) builds for EmiBSD: it copies every inherited src/ back to its path in the base system's tree under target/, builds the host tools the Makefiles run (rpcgen, yacc, lex), then /usr/include, csu, the libraries and the programs, each by its own Makefile and the pin's share/mk, with DevTools' clang for aarch64-emibsd-emibsd8.0. It runs as a user: tools/host/install.sh records in target/<arch>/metalog the owner, group and mode a Makefile asks for instead of setting them. just is the recipes.
  • Versions. Semver per program (src/Cargo.toml, or version in UPSTREAM.toml with our patch level as +emibsd.N); a calendar version, YEAR.MONTH, for the system release. A commit that changes a program's code carries Change: breaking|feature|fix and the bump it calls for; the commit-msg hook (just setup) refuses it otherwise, and just changelog writes each program's CHANGELOG.md from the trailers (docs/PROGRAM.md, Versions).
  • Performance, concurrency and security are part of every definition: a performance budget measured against a reference system's program on the same input (bench/, just bench); the concurrency model, and stress tests on four CPUs for what serves many; a threat model, the pledge/unveil profile, and the program's CVE history (tests/cve/), each entry made impossible by a type, a check or the language, or tested.

Layout

Under base/, the system that is built, the BSD split: bin/ and sbin/ are what works with only the root partition mounted (linked static), usr/bin/ and usr/sbin/ the rest; bin and usr/bin are for the user, sbin and usr/sbin for operating the system. Inside each, the programs are grouped by what they do. A concept the tradition spreads over two trees keeps a distinct name in each (bin/files and usr/bin/file-tools, sbin/network and usr/sbin/network/), and the README of each says where its other half is. Every program is listed once, in the tree the tradition keeps it in; tools/layout.py regenerates the READMEs and the skeletons from the map and refuses to write when one is missing or doubled.

base/bin/
├── shells               The shells.
├── files                The basic file operations.
├── processes            Listing, signalling and pacing processes; the shell's primitives.
├── system               The machine's name, the terminal line, a checksum.
└── tapes                Tape and changer control.
base/sbin/
├── boot                 Process 1, going down, the console and the terminals.
├── storage/disks        Partitions, labels, device nodes and controllers.
├── storage/filesystems  Making, checking and tuning file systems.
├── storage/mounts       Mounting, and serving file systems over NFS.
├── storage/swap-backup  Swap, dumps, restores and quotas.
├── network              Interfaces, routes, the packet filter, address configuration and name resolution.
├── ipsec                IPsec and its key daemons.
└── kernel               Reading and setting the kernel's state.
base/usr/bin/
├── text                 Filters and transformers of text streams.
├── records              Querying the records the written programs emit (`-o json`); EmiBSD's own, no counterpart elsewhere.
├── editors              Editors and the pager.
├── file-tools           Finding, describing and installing files (the basic operations are bin/files).
├── process-tools        Running things under a changed environment, priority or limit; the shell's helpers (the primitives are bin/processes).
├── users                Logging in, identities, who is on the machine, talking to them.
├── network-clients      Network clients a user runs.
├── development          Build tools, source generators, version control and tracing.
├── docs                 Manual pages and terminal descriptions.
├── calc                 Calculators and calendars.
├── mail                 Reading and sending mail from the command line.
├── scheduling           Running things later.
├── media                Audio, MIDI, radio and CD.
├── system-info          What machine this is, how it is doing, and small system chores.
├── crypto               Certificates, signatures and keys from the command line.
├── yp-clients           Querying the YP/NIS directory (its servers are usr/sbin/yp).
└── misc                 What fits nowhere else.
base/usr/sbin/
├── base-daemons         What rc starts on every system, and rc's control program.
├── accounts             Creating users and groups, the databases built from them, accounting, tokens (what a user does with their own identity is usr/bin/users).
├── storage-admin        Boot blocks, images, quotas, iSCSI, media (the root-partition tools are sbin/storage).
├── network/admin        Neighbours, traces, captures, link state (the root-partition tools are sbin/network).
├── network/dhcp         DHCP, DHCPv6 and router advertisements: the servers and the lease controls.
├── network/routing      Routing daemons and their control programs.
├── hardware             Buses, firmware, GPIO and the guest-host channel.
├── stats                Kernel and I/O statistics.
├── services             Servers: web, mail, time, DNS, LDAP, SNMP, domains.
├── yp                   The YP/NIS servers (the clients are usr/bin/yp-clients).
├── auth                 RADIUS, authpf, wireless authentication and certificate renewal.
├── vpn                  IKE control, PPP and L2TP (the IPsec daemons are sbin/ipsec).
├── nfs-rpc              The RPC services and network boot.
├── printing             Printing.
├── virtualization       vmm(4)'s daemon and control program.
├── packages             Packages, firmware, patches and upgrades.
├── kernel-dev           Configuring, tracing and packaging kernels (the running kernel's state is sbin/kernel).
└── wscons               Console screens, fonts and the console mouse.
base/usr/games/
└── games                Every game, as the base system keeps them, one tree.
base/libexec/
├── login                The authentication back ends login(1) calls.
├── rtld                 The run-time linker.
├── network-daemons      Daemons inetd and the RPC services start.
├── mail                 Local delivery and the spam daemon.
└── system               The login prompt, the nightly security check, the kernel relinker, the traditional cpp.
base/lib/
├── libc                 The C library, its start-up files, threads and libm.
├── crypto               Cryptography and authentication.
├── terminal             Terminal handling and line editing.
├── system               The rest.
└── crates               The Rust crates the written programs share, each born with its first user and laid out as a program (docs/PROGRAM.md): `cli`, the common grammar (one declaration per program with typed options, --help as text and as a record, usage and diagnostics, exit codes; records, -o, --sort-by, tables, JSON, YAML, columns); `sys`, the thin FFI over libc (passwd and group lookups, the terminal, local time, pledge, uid and umask, mkfifo and mknod, file flags, a link's times and mode), the one place beside lib/libc and libexec/rtld where `unsafe` is allowed, each site with its proof. EmiBSD's own; no counterpart elsewhere.
base/etc/
├── rc                   The start-up scripts and the terminal table.
└── base                 Users, groups, shells, the protocol tables, device nodes, the file-system skeleton, root's dotfiles, the per-architecture files.
base/data/
├── zoneinfo             The IANA time zone database (tzdata; zic and zdump are usr/sbin/services).
├── locale               Locale definitions and character maps.
├── dict                 The word lists spell(1) and look(1) read.
├── termtypes            The terminal descriptions, source of terminfo (tic is usr/bin/docs).
└── misc                 Small tables of share/misc: ascii, airport codes, units, the magic of file(1) is in file itself.
base/distrib/
├── sets                 The install sets and their lists; the release version is YEAR.MONTH (docs/PROGRAM.md, Versions).
├── installer            The installer, its ramdisk and the per-architecture media.
├── include              /usr/include as include/Makefile assembles it: the C library's public headers (include/) and the kernel's, the ABI the kernel implements (sys/, headers only, without dev/microcode and dev/pci/drm, as that Makefile leaves them out). The libraries' own exported headers stay with each library.
├── share                /usr/share's own trees: share/mk, the rules every Makefile of the base system includes, and the manuals' shared pages.
├── kernel               The kernel's sources the userland compiles too, reached by the Makefiles' .PATH into sys/: bpf_filter for libpcap, pf_ruleset for pfctl, pf_osfp for tcpdump, ffs_subr and ffs_tables for fsck_ffs and fsdb, ext2fs_bswap for the ext2fs tools, syscalls for btrace, the three DRM device lists fw_update matches firmware with, and the DRM ioctls kdump decodes; and the files that declare the kernel's devices (sys/conf/files, the architectures' files.<arch> and the files.* of sys/dev and sys/scsi), which world.py reads to tell a device's manual page from a protocol's.
└── build                The base system's build glue, which belongs to no program: the top Makefile and Makefile.cross, and the Makefile and Makefile.inc of each tree (bin/Makefile.inc makes every /bin program static).
regress/
├── scenarios            One directory per scenario: the steps, the expected lines, the comparison with a reference system.
└── harness              Booting the images, driving the console, collecting and comparing.

The lists are the map to what a complete base system provides, taken from the names at the pin of Kernel/reference/PINNED.md: every program directory of the base system at the pin has exactly one place in it, and each place is filled by its class.

Open

  • The inherited class is in the tree, not built. Every program of the base system has its directory at the pin in src/, copied by tools/layout.py (committed, the user's decision of 2026-11-04), and so does every tree that is not a program (etc/, data/, distrib/: share/mk, /usr/include with the kernel's headers, the build glue). just world arm64 builds the whole inherited base system for EmiBSD in about four minutes from a clean target: /usr/include, the 37 libraries of lib/ (static and shared) and ld.so, and every program of bin, sbin, usr.bin, usr.sbin, libexec, share and games, each by its own Makefile, dynamic where the base system links dynamically. Left out: what is GPL and outside the tree (bgplgsh's readline, amd's info manual; perl, cvs, texinfo of gnu/), and nsd and unbound, pending (their configure must run its tests, which a cross build cannot). just world arm64 sets makes base, comp, game and man as distrib/sets/maketars does, from DESTDIR by the lists, with the owners and modes of the metalog, and writes what the lists name and was not installed (target/arm64/sets/*.missing). just world arm64 image makes the single-user ramdisk that boots on the Kernel. The build's host tools are the tree's own, built for the host by the same make (tic, strfile, mkdict, makewhatis; pax, mtree, sha256 and pwd_mkdb with tools/host/owners.h, which gives them the tree's owners from the metalog), so terminfo, the games' data, mandoc.db, /etc with pwd.db, and var/sysmerge/etc.tgz with its etcsum are made as the base system makes them. Still to come: the locate database (makelocatedb names the target's own tools), the firmware of etc/firmware, the compiler's part of comp (DevTools), Perl (the user's decision), amd64. gnu/ stays out; the base system's own regress/ is brought per program when needed.
  • Decisions the user has still to take about the inherited class: whether the Kernel keeps its own userland build until N9 (proposed) or consumes this repository's ramdisk now; the milestone letter and the first milestone (proposed: the 110 programs the Kernel builds today, in this layout, booting its smokes).
  • Perl. pkg_add, adduser, libtool, pkgconf and security are traditionally Perl, and Perl (Artistic or GPL, dual) is not in the tree: inherit it under the Artistic licence for those five, leave them out, or write them in Rust. The user's decision; their skeletons say so.
  • cargo audit and cargo deny (just audit, part of just check when installed) are tools installed outside the repository with cargo install; tools/cve.py (`just cve `) queries NVD over the network. Both written, neither run: pending the user's OK.
  • data/ entries are imports like the programs (tzdata, the locales): their UPSTREAM.toml files are to be written.

New code is ISC (LICENSE).

About

EmiBSD's own userland: the programs, libraries and configuration above the Kernel, written from scratch in Rust

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages