Dynblk Self-Contained Block Storage

Dynblk provides /dev/dynblk0 for Linux 6.12 with 4096-byte pages, backed by a private directory of ordinary files. The kernel creates and manages volume000.db and lazy siblings volume001.db through volume063.db. There is no JSON manifest, userspace layout writer, separate userspace journal, daemon or manual compactor. On-disk format 1 is specified in FORMAT.md. Format migration and legacy command aliases are not provided; use matching driver and userspace versions for a volume.

Storage Protocol

Logical I/O and independent COW overwrites are 4 KiB. With compression enabled, complete aligned 64 KiB regions in an incoming write may share one immutable compressed blob across sixteen subblocks. A later 4 KiB overwrite replaces only that mapping, without rewriting its neighbors. none is the default and stores nonzero blocks as raw 4 KiB records. This uses kernel crypto providers, not a zram device or the zram module; no zram dependency or private kernel codec exists.

The authoritative map is a four-level COW radix tree. Each update writes dirty leaves and ancestor paths plus a rooted transaction page, then fsyncs backing parts and publishes both root copies with separate fsync barriers. Old references are released only after both root barriers. A one-block update writes at most four tree pages, not a complete map checkpoint. Capacity-only growth rewrites no tree nodes. The transaction records the selected tree's changes; it is not an external append-only journal or a historical log that recovery replays. Recovery validates the entire selected live tree, transaction and payload before repairing roots or reclaiming storage. Automatic synchronous GC preserves shared encoded blobs and relocates live tree metadata. FORMAT.md defines the exact protocol.

Commands

Run the native ./dynblk binary from this directory or /usr/sbin/dynblk after installation. Mutations are dry-runs unless --execute is supplied; execution also requires root. Read operations use file/device permissions and do not require --execute.

dynblk create /srv/private/volume000.db --size 16GiB --compression none \
  --part-size 4000MiB --module /path/to/dynblk.ko --execute
dynblk status /dev/dynblk0
dynblk status /dev/dynblk0 --json
dynblk grow /dev/dynblk0 32GiB --execute
dynblk unload /dev/dynblk0 --execute
dynblk inspect /srv/private/volume000.db --metadata-only
dynblk check /srv/private/volume000.db --json
dynblk load /srv/private/volume000.db --module /path/to/dynblk.ko --execute

create creates and attaches, by passing filename, create=1, size_bytes, algorithm and part_limit_bytes to the module. Defaults are 16 GiB virtual, none compression and 4000 MiB per part. It does not create the parent directory or write any volume bytes. Existing part names reject at preflight; exclusive kernel creation is authoritative. Failed creation leaves diagnostics intact; the CLI never deletes partial files or retries by overwriting them.

load passes only filename; it never overrides the stored capacity, codec or part limit. Without --module, create/load use modprobe dynblk; with it they use insmod. The CLI verifies the loaded module's backing path and status after the loader returns, so a concurrent load for another volume is rejected. Loading an already loaded module is refused before invoking the loader. Codec allocation and lower-filesystem admission errors come from the actual kernel operation. Names are none, lz4, lz4hc, lzo, lzo-rle, zstd, deflate, 842; recognizing a name does not promise that the running kernel has that transform available. There is no userspace codec fallback or probing by loading modules.

Sizes are integer bytes or binary KiB, MiB, GiB, TiB, multiples of 4096. Virtual size is positive and at most 128 GiB. --part-size is create-only and 4096-aligned between 1 and 4000 MiB; kernel admission remains authoritative. Virtual size is not allocated host space, guaranteed writable space, or a RAM reservation. Host ENOSPC and memory exhaustion remain possible.

status DEVICE reads the block GET ioctl 0x8080d700, exactly 128 bytes, through the shared dynblk_uapi.h structure. Text is the default; --json is a single sorted JSON object. Byte counts are integers, UUID is text and fenced is a boolean. tree_height is 4 and compression_region_bytes is 65536. The earlier checkpoint-period/reserved interpretation is not supported. stored_bytes counts each surviving shared encoded blob once; original_bytes counts live logical 4 KiB blocks. map_memory_bytes includes resident mapping leaves and the fixed tree-pointer index, not all kernel/VFS/crypto allocations. physical_bytes is summed file lengths, not allocated host sectors or free space. See FORMAT.md for all ABI fields and their current meanings.

grow DEVICE SIZE sends absolute capacity through GROW 0x4008d701, native unsigned 64-bit bytes, not an increment. Shrinking is rejected by the kernel. It does not resize an upper filesystem. unload DEVICE verifies the block ABI then invokes plain rmmod dynblk. Normal module pins must reject busy devices; there is no forced unload, unmount, filesystem check, format or mount helper. Dry-run grow/unload never open the device.

Detached Inspection

inspect PATH --metadata-only [--json] requires the explicit flag because it is not a complete volume validator. It opens read-only with nonblocking shared BSD flocks and checks all existing part headers' CRCs, UUIDs, incarnations, format, codec, reserved bytes, part limits and lengths. It checks both root CRCs and structural fields, selects the newest valid root without stale fallback on checksum-valid malformed metadata, and rejects missing active parts. Root tree and transaction pointers must fit active part geometry. At most 66 metadata pages are read; there is no unbounded tree walk or payload decompression. Initial roots may activate only part 0; later transaction pointers must bind the publishing generation. Impossible root-level page counts, direct root-pointer aliases and malformed pointers in either checksum-valid root reject. These checks do not establish the topology or reference consistency of the unvisited tree.

Output includes inspection_scope: headers-roots-part-geometry, fully_validated: false, a warning, capacity/generation, active/present parts, tree page count and root_repair_needed. A torn root copy is reported, never repaired. COW tree contents, transaction contents and compressed payloads are not checked by this inspector. A successful report does not establish attachability or integrity. Kernel attachment performs authoritative recovery validation and may write recovery metadata; it is not a read-only inspection substitute.

An attached volume's exclusive lock prevents inspection. Locks are cooperative; outside writers that ignore them, administrative renames, remounts and direct backing-file changes are forbidden. No inspection writes, repairs or cleanup are performed, including after errors.

Full Read-Only Check

dynblk check PATH/volume000.db [--json] validates the complete selected live state, including the radix tree, rooted transaction and every retained payload. It never loads a module, opens a block device, repairs roots, truncates/unlinks parts, writes a layout or creates temporary files. Both check and inspect open parts with O_RDONLY|O_NOFOLLOW|O_NONBLOCK|O_NOATIME and retain shared BSD flocks on all existing parts. O_NOATIME failure is an error, not a retry that updates atime; use root for root-owned storage. Cooperative locks are mandatory: outside writers that ignore them are unsupported.

The checker selects roots using the same generation/CRC rules as inspection, then checks every edge's generation and CRC, node identity/level/base/count, reserved/unused entries, capacity bounds, cycles, metadata aliases and mapping counts. It verifies transaction ordering, versions, resulting capacity and exact agreement with the selected tree. The active-part incarnation list must equal the set of reachable parts, with part 0 always present. Valid inactive candidates are preserved, not reclaimed.

Payload checks include encoded IEEE CRC32, bounded decompression, exact decoded length and the original CRC of every live 4 KiB slice. A shared 64 KiB blob is decoded once per logical region, even with independent 4 KiB replacements among its surviving members. Shared members must agree on the complete blob identity. Metadata and payload may not share a page. Distinct payload byte ranges must not overlap, including aliases across logical regions. This last interval check is stronger than the current kernel's page-reference counters; it enforces the writer's disjoint-blob invariant rather than accepting malformed byte aliases.

On success, text or JSON includes validation: passed, fully_validated: true, validation_scope: selected-live-state, verified_live_bytes, verified_blobs, verified_stored_bytes, verified_tree_pages and verified_mapping_count. root_repair_needed can still be true: a torn/stale root copy is reported, not repaired. Unreachable tails and inactive payloads are not replayed or declared valid. This is not an upper-filesystem check, lower-filesystem admission test, authentication/rollback defense, proof of host durability or production readiness.

Any corruption, missing decoder, resource exhaustion or I/O failure returns nonzero with an error on stderr and no successful/partial JSON result. The checker traverses the tree twice with iterative depth bounded to four, without retaining the full mapping table. Decode buffers are at most 64 KiB. At most 528417 nodes and 33554432 unique blobs are admitted. Page counters use two bytes per physical 4 KiB page; interval indexes use eight bytes per unique blob and are sorted independently for each backing part. Checking a maximally populated volume can require several hundred MiB of RAM and read all live data; allocation failure aborts instead of skipping checks.

Decoder Dependencies

The checker is part of the native dynblk executable, but codec support is not a mandatory runtime dependency of the program. Control commands, metadata-only inspection and check of raw/none storage work without codec libraries. Ordinary lzo and lzo-rle use the GPL-2.0-only Linux-v6.12-derived decoder compiled directly into dynblk. For compressed lz4/lz4hc, zstd and raw deflate payloads the checker lazily opens liblz4.so.1, libzstd.so.1 or libz.so.1 only when such a blob is actually encountered. A missing optional library fails that check with a codec-specific error; it does not prevent dynblk from starting or managing a device. The kernel module uses its own kernel crypto providers and is independent of these userspace decoder libraries. There is no Python interpreter, ctypes adapter or decoder subprocess in the runtime path. See source attribution and tests.

Compressed 842 still fails explicitly: no decoder is provided and the target kernel does not compile that codec. Checking raw-only bytes under such a header does not imply that the kernel can attach it. Missing decoder support never turns into a metadata-only success or a recommendation to rewrite an immutable codec.

Storage Safety

Backing paths must be canonical absolute paths ending in volume000.db, without symlinks or dot components. Every ancestor and file must be root-owned and not group/other writable; backing files must be single-link regular files. All parents must already exist. /tmp is unsuitable even with a private child. The kernel repeats admission using pinned directories and loader credentials; CLI preflight alone is not a race-proof filesystem security boundary.

Use independent local storage. The CLI does not maintain a lower-filesystem whitelist or call filesystem utilities; kernel admission is authoritative. Accepted lower filesystems and geometry constraints are documented in FORMAT.md. FAT/exFAT ownership and masks must satisfy the same rules. Ext2 is supported when it is served by the ext4 driver on kernels configured with CONFIG_EXT4_USE_FOR_EXT2; ntfs3 means the in-kernel driver, not ntfs-3g. FAT, exFAT and ext2 are non-journaled, so lower-filesystem power-loss behavior depends on the underlying filesystem and storage device. Btrfs support assumes a single-device ordinary-COW filesystem; NOCOW backing files are rejected.

After detach, preserve the complete directory of existing parts when copying a volume; absent unused siblings are normal. Durably sync the copy independently. Do not remove apparently unused files manually. Destination permissions, filesystem/sector geometry and space still need to pass kernel admission.

Build And Install

make -C /lib/modules/$(uname -r)/build M="$PWD" W=1 KCFLAGS=-Werror modules
make test
make install DESTDIR=/tmp/dynblk-stage

The installed runtime is the single native /usr/sbin/dynblk executable. Its hard ELF dependency is libc; kmod supplies the normal-system modprobe/rmmod commands. Codec libraries are optional and are opened only by offline check for the corresponding compressed payload. Python is test tooling only, not a runtime or installation dependency. make install installs the CLI and matching documentation only; it does not register, build, install or load DKMS modules. No mount helper or legacy aliases are installed.

Debian packaging lives in the root debian/ directory. The dynblk-dkms package stages the minimal module source under /usr/src/dynblk-<version> and uses standard dh_dkms integration. The upstream dkms.conf remains suitable for explicit manual development use with AUTOINSTALL=no; package autoinstall policy is defined separately in debian/dynblk-dkms.dkms.

The host make dynblk-initrd target is a static userspace test build. The binary shipped in MiniOS initramfs is rebuilt separately as static i686/musl with the MiniOS Buildroot toolchain; see the initramfs Buildroot documentation.

make test-userspace builds the native CLI/checker and runs LZO/LZO-RLE round-trip and malformed-input tests under ASan/UBSan together with black-box checks for all supported checker codecs, corruption handling, path/sizing safety, JSON output and dry-run command scope. make test additionally runs the source-extracted storage/admission harness in tests-storage. VM acceptance scenarios are documented separately under tests/vm/.