Repository navigation
Add Google Benchmark harness for HDF5 and Memory backends - #102
Open
olivhoenen wants to merge 12 commits into
Open
olivhoenen wants to merge 12 commits into
olivhoenen wants to merge 12 commits into
Conversation
…t smoke The vertical tracer bullet through the whole build+run+report path (NORTH_STAR.md §8.4): a new benchmarks/ tree, gated by AL_BUILD_BENCHMARKS and its own preset, off in the ordinary build and independent of AL_BUILD_TESTS. Google Benchmark is pinned via FetchContent (v1.9.5, no cluster module dependency). One benchmark drives the public C ABI end-to-end -- a single-leaf write-once-then-read against the in-RAM Memory backend on a tiny inline fixture, deliberately not the equilibrium seed yet -- emitting Google Benchmark JSON. A CTest smoke pair (mirroring tests/CMakeLists.txt's FIXTURES_SETUP/FIXTURES_REQUIRED shape) runs it with a one-iteration budget and asserts clean exit plus structurally parseable JSON, with no timing-value assertion. (cherry picked from commit f24b119)
(cherry picked from commit e2f2d19)
Reuses equilibrium_seed.h (issue #51) verbatim as the workload, driven strictly through the public C ABI. Write + open happen once in untimed setup; the timed region is exactly equilibrium_seed::read_back's begin_global_action(READ) -> traverse every leaf -> end_action, repeated across 20 repetitions so Google Benchmark's JSON reports a distribution (mean/median/stddev/iterations), not a bare mean. Splits BENCHMARK_MAIN() into its own bench_main.cpp now that a second bench_*.cpp exists, and factors the shared CheckOk status-checking helper into bench_common.h so it isn't duplicated across benchmark files. (cherry picked from commit 144206e)
(cherry picked from commit 7246824)
Refactors the HDF5-only equilibrium whole-IDS read benchmark (#53) into one shared BM_EquilibriumWholeIdsRead body parameterized by a BenchBackend enum, registered via BENCHMARK_CAPTURE for both HDF5 and the in-RAM Memory backend. Memory isolates per-node traversal/CPU cost from HDF5's I/O cost, and adding it required only branching the URI-building step -- validating that backend is a cheap sub-axis of the operation rather than a copy-pasted scenario. (cherry picked from commit af0fb25)
…n (issue #55) capture_baseline.py wraps al_benchmarks, tags the resulting JSON with comparability metadata (OS, CPU model, compiler+flags, HDF5 version, filesystem, SLURM partition, commit SHA), and writes it to benchmarks/baselines/<machine-id>/<commit-sha>.json. BASELINE_CAPTURE.md documents the laptop-smoke and ITER-SDCC-cluster procedures; the SLURM partition is always a runtime parameter, never hard-coded. (cherry picked from commit 4a20b81)
(cherry picked from commit c0ee66c)
Moves BenchBackend, the HDF5/Memory URI builder, and pulse-open lifecycle setup out of bench_equilibrium_read.cpp into bench_common.h so the three follow-on benchmarks (issue #57) can reuse them instead of re-deriving the same boilerplate. No change to the existing whole-IDS-read benchmark's behavior or reported numbers (issue #58). (cherry picked from commit 9554df2)
Reads one time_slice element's full leaf set (timebase leaf, radial profile, scalar quantity, constraints subtree) by index, node-by-node, through the public C ABI -- the shared "before" baseline both P2's al_read_subtree and P3's slice-mode buffering (NORTH_STAR.md §7) will diff against, since neither optimization exists yet and this node-by-node walk is the only legal way to fetch one AOS element today. (cherry picked from commit f15d86a)
Write-side counterpart to bench_equilibrium_read.cpp: pulse opened once (untimed), equilibrium_seed::write repeated inside the timed loop on the same open context, closed once (untimed), mirroring the read benchmark's discipline exactly. An explicit extra untimed write after the loop guarantees at least two writes to the same context on every run -- including the CTest smoke run, which caps the timed loop at exactly one iteration -- so the post-loop structural-hash check against equilibrium_seed::expected_hash() is a real, non-vacuous verification that repeated writes never accumulate stale data, on both backends. Surfaced (and relies on the fix for) a genuine HDF5Writer bug where a second whole-IDS write to an already-populated open pulse always re-attempted H5Dcreate2 instead of opening the existing dataset -- fixed separately as part of issue #60's concurrent work (commit 353a66d). (cherry picked from commit e7cfb7a)
Iterates every time_slice element via al_iterate_over_arraystruct, reading all five leaves of each, against both backends -- the "before" baseline for P1/P4's round-trip-count reduction (NORTH_STAR.md §7) and the macro signal that would justify a future P5 per-call primitive micro-benchmark if it comes back slow. Functionally the same traversal bench_equilibrium_read.cpp already drives via equilibrium_seed::read_back, isolated and named as its own operation because its coverage target (P1/P4 round-trip count) is distinct from "read everything in the IDS" as a whole. Reuses equilibrium_seed.h and bench_common.h's shared per-backend boilerplate (issue #58) unchanged; added as a source to the existing al_benchmarks executable, so the existing CTest smoke check and baseline-capture script cover it automatically with no changes to either. (cherry picked from commit 2d7e07c)
(cherry picked from commit e41a944)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the split of #85 (Tests and Benchmarks for regression) into smaller, single-purpose PRs that are easier to review.
Adds an opt-in
benchmarks/tree (AL_BUILD_BENCHMARKS, OFF by default; a benchmark never gates the correctness build) with Google Benchmark and a CTest smoke test that validates the JSON output. Operations: whole-IDS equilibrium read and write, single time-slice element read, and an AOS-heavy traversal, each on HDF5 with the Memory backend as a zero-I/O control. Adds baseline-capture tooling (capture_baseline.py, metadata conventions inBASELINE_CAPTURE.md). Additions only.Yohann's laptop baseline JSON (commit 2b9df07) is deliberately left out: a reference baseline on the ITER cluster is still to be captured (yohannmarguier#56).
Fork history: issues yohannmarguier#50 – yohannmarguier#61 (some still open upstream on the fork: #50, #56, #57).
Stack position: 7/9; depends on #100, because the whole-IDS write benchmark needs that fix.
Authorship: these commits are @yohannmarguier's, cherry-picked unchanged from his fork with
git cherry-pick -x, so each one keeps its original author, date and message plus a(cherry picked from commit …)link back to the fork. Only conflicts with currentdevelopand with the new commit order were resolved.🤖 Generated with Claude Code