Skip to content

Repository files navigation

libumsh

UMSH is a unified-memory sharing library for integrated CPU/GPU systems. It separates memory binding attributes from data-access policy and records the effective backing, cache/coherency contract, and implementation used by each experiment.

The public Core allocation facade takes a size and BindOptions:

umsh::binding::BindOptions options{
    .page_size = umsh::binding::PageSize::THP,
    .cpu_cache = umsh::binding::CpuCachePolicy::WriteBack,
    .gpu_cache = umsh::binding::GpuCachePolicy::RuntimeDefault,
    .coherency = umsh::binding::Coherency::RuntimeDefault,
};
auto memory = core->bind(size, options);

The canonical data plane follows the paper taxonomy in include/umsh/policy.hpp: reads are Alone, AfterInvalidate, or Bypass; writes are Alone, ThenFlush, or Bypass. CUDA cache operators and fences are retained as raw characterization primitives and are not aliases for canonical bypass or flush policies.

Artifact evaluation

Start with the guide for the target system. Each platform guide covers native deployment, a short correctness smoke test, the full correctness matrix, benchmark prerequisites, and individual benchmark commands.

Target Guide Checked-in campaign IDs
Intel integrated GPU Intel AE guide ultra258v (Lunar), u285h (Arrow), tianx (Raptor)
NVIDIA integrated CPU/GPU NVIDIA AE guide orin-new (Orin), spark (GB10)
A different host or architecture Platform extension guide add a new manifest entry

The default AE build is userspace-only:

cmake -S . -B build-release \
  -DCMAKE_BUILD_TYPE=Release \
  -DUMSH_BUILD_EXAMPLES=ON \
  -DUMSH_BUILD_KERNEL_MODULE=OFF \
  -DUMSH_TARGET_PLATFORM=AUTO
cmake --build build-release --parallel
ctest --test-dir build-release --output-on-failure

Use UMSH_BUILD_KERNEL_MODULE=OFF for the ordinary path. All UMSH helper-module sources and the Spark UVM overlay are included in this checkout, but they still require matching kernel or driver sources and privileged loading. The Intel and NVIDIA guides identify exactly which additional rows need those modules. Missing optional setup must remain a visible skip and must not be relabelled as a pass or failure.

Optional kernel extensions

On Intel, uc_mem enables CPU UC-minus backing and wbinvd_mod enables the coarse all-CPU flush/invalidate rows. Build both against the running kernel:

cmake -S . -B build-intel-modules \
  -DCMAKE_BUILD_TYPE=Release \
  -DUMSH_BUILD_EXAMPLES=OFF \
  -DBUILD_TESTING=OFF \
  -DUMSH_BUILD_KERNEL_MODULE=ON
cmake --build build-intel-modules --parallel \
  --target uc_mem wbinvd_mod

On Orin, arm64_attr_mmap supplies the optional ARM page-attribute backing and arm64_attr_mmap_test is its CPU-only preflight:

cmake -S . -B build-orin-modules \
  -DCMAKE_BUILD_TYPE=Release \
  -DUMSH_BUILD_EXAMPLES=OFF \
  -DBUILD_TESTING=OFF \
  -DUMSH_BUILD_KERNEL_MODULE=ON
cmake --build build-orin-modules --parallel \
  --target arm64_attr_mmap arm64_attr_mmap_test

Each attribute-configuration ioctl requires CAP_SYS_ADMIN, so the two ARM attribute performance cases remain external_setup and are run with the manual privileged procedure in the NVIDIA guide.

The Spark patched-UVM extension is built from the checked-in overlay plus an unmodified NVIDIA open-gpu-kernel-modules 590.48.01 checkout at commit 2ccbad25e1af6a6ee6f38cf569f89f8b65d658ab. The OGKM source, installed driver/userspace/GSP, running kernel headers, and compiler ABI must match:

cmake -S . -B build-spark-uvm \
  -DCMAKE_BUILD_TYPE=Release \
  -DUMSH_TARGET_PLATFORM=SPARK \
  -DUMSH_BUILD_EXAMPLES=OFF \
  -DBUILD_TESTING=OFF \
  -DUMSH_BUILD_KERNEL_MODULE=ON \
  -DUMSH_BUILD_NVIDIA_UVM_MODULE=ON \
  -DNVIDIA_OGKM_SOURCE_DIR=/absolute/path/to/open-gpu-kernel-modules
cmake --build build-spark-uvm --parallel --target nvidia_uvm_ext

Do not replace a live nvidia_uvm manually. The guarded Spark matrix runners capture the stock module identity and parameters and restore them from an exit trap. See the NVIDIA guide and vendored overlay reference.

After building, run the stock-runtime binding probe:

build-release/examples/custom/policy_probe/umsh_policy_probe

BINDING status=PASS confirms one real bind and reports the capability matrix. It is a deployment check, not a CPU/GPU data-transfer correctness result. Run the persistent-kernel smoke command from the selected platform guide for that result.

Campaign workflow

tools/cross_platform/orchestrate.py provides the evidence-producing path. It collects read-only inventory, checks source identity, runs only cases declared ready, validates required CSV/JSON artifacts, and writes checksums. It never builds the repository, invokes sudo, loads a module, or changes a kernel setting.

Validate and inspect the campaign before connecting to a host:

python3 tools/cross_platform/orchestrate.py validate
python3 tools/cross_platform/orchestrate.py plan --profile smoke
python3 tools/cross_platform/orchestrate.py plan \
  --platform u285h --suite correctness --profile full

The checked-in platforms.json is the reference five-host deployment. Its SSH aliases and source/build paths are intentional and are not a portable search list. Use it unchanged only on that deployment. For another machine, create an untracked manifest under build/ as described in the platform extension guide.

On a configured reference host, execute a profile from a clean checkout:

commit=$(git rev-parse HEAD)
python3 tools/cross_platform/orchestrate.py run \
  --platform u285h \
  --suite correctness \
  --profile smoke \
  --expected-commit "$commit" \
  --require-clean \
  --execute

Omitting --execute is always a dry run. The smoke profile is a 4 KiB, one-seed, two-round canonical matrix. The full correctness profile runs 4 KiB, 64 KiB, and 1 MiB with three seeds and 20 rounds. Performance cases are available only in the full profile. The orchestrator currently selects by platform, suite, and profile, not by individual case; use the platform guide's direct binary commands for a focused diagnostic.

Results are written under build/experiment-results/cross-platform/<run-id>/. Keep campaign.json, cases.csv, manifest.snapshot.json, SHA256SUMS, inventory, raw streams, result artifacts, and performance sidecars together.

Campaign status has deliberately narrow meaning:

Status Meaning
RECORDED command exit and artifact contract passed; this alone does not prove a paper claim
SKIP_UNSUPPORTED the implementation is not registered on this platform
SKIP_EXTERNAL_SETUP the case requires an out-of-band privileged or storage setup
SKIP_PRECONDITION a declared runtime prerequisite, such as a helper module, is absent
ERROR_* source, command, timeout, result-contract, or artifact validation failed

Within a correctness CSV, FAIL_STABLE is a bounded, repeatable data observation, not an orchestration failure. Unsupported, skipped, inconclusive, and failed coordinates must remain in the evidence instead of being deleted.

Documentation

  • Intel guide: Intel OpenCL deployment, page-system setup, correctness, benchmarks, optional x86 helper modules, and SSD I/O.
  • NVIDIA guide: stock Orin/Spark deployment, CUDA correctness and benchmarks, optional ARM attributes, patched UVM, and SSD I/O.
  • Platform extension guide: onboard a new host, register honest support states, and archive a new campaign.

Library installation

Installation is optional for AE because all tests can run from the build tree. To install the library, headers, CMake package, tools, and examples under the configured prefix:

sudo cmake --install build-release

The generated shared library is build-release/libumsh.so. Installed projects can consume the package with find_package(umsh) and use the installed umsh_run_policy_matrix runner.

About

Artifact evaluation for libumsh [EuroSys'27]

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages