Skip to content

Read structs with defaults that hold a Validator without a valid default - #12

Open
clockblocker wants to merge 3 commits into
mainfrom
fix-700-deferred-defaults
Open

clockblocker wants to merge 3 commits into
mainfrom
fix-700-deferred-defaults

Conversation

@clockblocker

@clockblocker clockblocker commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

Fixes munich-data-engineering#700 at the cost of the library's first thread_local.

The only other acceptable solution I found was to document the suggested workaround in the code and docs and leave it be.

AI-authored

Before and after

struct Person {
  rfl::DefaultVal<std::string> name = "Homer Simpson";
  rfl::Validator<std::vector<std::string>, rfl::Size<rfl::Minimum<1>>> favorite_foods;
};

rfl::json::read<Person>(R"({"favorite_foods": ["donuts", "pizza"]})");
main:        error  Size validation failed: Value expected to be greater than or equal to 1, but got 0.
this branch: ok     {"name":"Homer Simpson","favorite_foods":["donuts","pizza"]}

On main no input can be read into this struct. The parser builds Person{} before it looks at the input, and the empty favorite_foods fails its own validator.

A field without a valid default still has to come from the input:

rfl::json::read<Person, rfl::DefaultIfMissing>(R"({})");
main:        error  Size validation failed: Value expected to be greater than or equal to 1, but got 0.
this branch: error  Field named 'favorite_foods' not found, and its default value is invalid: Size validation failed: Value expected to be greater than or equal to 1, but got 0.

Outside a read, Person{} and Validator() throw as before.

Approaches

Defaults of missing fields exist only as default member initializers, which C++ evaluates only while constructing the whole struct. So the parser has to build T{} before it reads, and Validator() has to return normally during that. It takes no arguments, so the parser cannot tell it who is calling.

Approach Why not, or what it costs
The user writes a valid dummy default; docs and a clearer error (fork PR #11) The dummy is real data under DefaultIfMissing and in Person{}.
Validator() never validates its default Person{} in user code holds an invalid value without an error.
Validator can hold "no value": a Result<T> (upstream PR #737) or a flag Every Validator grows and gets accessors that can fail.
Validator() detects the parser with std::source_location (upstream PR #707) Depends on the caller's function name. A missing field keeps the unvalidated default under DefaultIfMissing and DefaultVal<Validator>.
Thread-local scope, then a walk over the type to find invalid defaults (fork PR #8) +926 lines in include/, 647 of them the walk, which broke compilation for some rfl::Skip fields.
A new wrapper for required fields Users change their structs, and the wrapper has an empty state.
Defaults carried in the type instead of = value A second way to declare defaults, limited to compile-time constants.
Thread-local scope plus address records (this PR) The library's first thread_local. +265 −30 lines in include/.

Only the last one reads the issue's struct with no change to user code, leaves Validator's layout and its behaviour outside a read alone, and still rejects a missing field whose default is invalid. It needs no type walk, because each Validator that keeps an invalid default reports its own address.

How it works

  1. read_struct_with_default builds T{} in the result's storage, as on main. If that succeeds, nothing else changes.
  2. If it throws, T{} is built again while a DeferredDefaults scope is active on this thread (RAII, restores the previous scope).
  3. Inside the scope, a Validator() whose default fails validation and whose address lies inside T's storage keeps T() and records its address and error. Every other Validator() (temporaries, the heap, other threads) throws as before.
  4. The fields are read. A field found in the input is replaced as a whole, so a record inside it no longer matters.
  5. A record inside a missing field fails the read with the error shown above, with the path in details().
  6. A per-type relaxed std::atomic<bool> remembers that T{} throws, so later reads skip step 1.

Two related changes, each in its own commit:

  • rfl::Variant() constructs its first alternative in place, so rfl::Variant and rfl::TaggedUnion fields are covered.
  • to_schema<Person>() threw on main, and called std::terminate when Person was nested in another struct. It now returns a schema without default values.

Limits

One case succeeds with an invalid value. A default member initializer that copies or reads a sibling Validator with no valid default is not checked:

struct Copies {
  Positive a;                        // rfl::Validator<int, rfl::Minimum<1>>
  rfl::DefaultVal<Positive> b = a;
};
rfl::json::read<Copies>(R"({"a": 5})");  // ok, b == 0

Copies{} outside a read still throws. Closing this needs user-provided copy constructors on Validator, which would make Validator<int, ...> non-trivially copyable for everyone. A test pins the behaviour and the docs state it.

Everything else fails with an error:

  • An invalid default built outside the struct's storage: moved in (rfl::DefaultVal<Positive> p = Positive();, rfl::Tuple<Positive, int>, a small struct returned in registers, -fno-elide-constructors) or on the heap (std::vector<Positive>(1), rfl::Box).
  • A default that a helper builds invalid and then fixes in place counts as invalid if the field is missing.
  • A default that catches the exception of an invalid Validator and falls back to a valid value works with clang and GCC. MSVC builds that Validator directly inside the struct, so it counts as deferred, and the read fails if the field is missing.
  • rfl::Skip<Validator> without a valid default is never read, so the read always fails.
  • A struct whose T{} throws for another reason is constructed twice before the same error is returned.
  • Windows DLLs each have their own copy of the thread_local. A Validator() that runs in another DLL than the parser throws, and the read fails as on main.

Measurements

All against main (e91ea28), macOS arm64, Apple clang 17 and Homebrew GCC 16.2.

Run time at -O3, in ns, median of 5 interleaved runs:

clang main clang branch clang Δ GCC main GCC branch GCC Δ
read, the issue's Person 4585 (fails) 822 (ok) −3763 (−82%) 5096 (fails) 685 (ok) −4411 (−87%)
read, DefaultVal struct without a Validator 145.0 144.7 −0.3 (−0.2%) 139.1 144.7 +5.6 (+4.0%)
read, struct with valid-default Validator<int> fields 144.5 149.1 +4.6 (+3.2%) 141.3 149.2 +7.9 (+5.6%)
read, the issue's struct with a valid dummy default 283.8 282.8 −1.0 (−0.4%) 287.6 300.7 +13.1 (+4.6%)
Validator<int, Minimum<0>>() 0.04 0.04 0 0.32 0.32 0
Validator<std::string, Size<Maximum<10>>>() 2.07 2.61 +0.54 (+26%) 3.03 3.36 +0.33 (+11%)
{} of a struct with four of those and one int Validator 7.49 10.67 +3.18 (+42%) 10.54 11.80 +1.26 (+12%)

The read rows vary by up to 12% between runs, so only the first row's delta is outside the noise. The last two rows are the real cost: the failure path of Validator() can now return instead of only throwing, which costs 0.3 to 0.6 ns per default-constructed Validator of a non-trivial type.

Compile time and peak memory, clang -O2 -c, median of 5 interleaved runs:

main branch Δ time main branch Δ memory
self-recursive struct with one DefaultVal 1.68 s 1.73 s +0.05 s (+3.0%) 273 MiB 277 MiB +4 MiB (+1.5%)
4 mutually recursive structs, DefaultIfMissing 1.47 s 1.59 s +0.12 s (+8.2%) 254 MiB 264 MiB +10 MiB (+3.9%)
8 mutually recursive structs, DefaultIfMissing 1.86 s 2.06 s +0.20 s (+10.8%) 276 MiB 290 MiB +14 MiB (+5.1%)

That is 25 to 30 ms per struct type read with defaults. Of each time delta, 0.01 to 0.03 s comes from the base branch.

Tests

clang 17 GCC 16.2
reflect-cpp-json-tests 310 pass 310 pass
reflect-cpp-generic-tests 102 pass 102 pass
reflect-cpp-cli-tests 75 pass not built (fails to build on main too: strtod_l)

The new tests also pass with -std=c++23 -DREFLECTCPP_USE_STD_EXPECTED (both compilers), with -fsanitize=address,undefined and with -fsanitize=thread (clang).

The fork has no CI. On the same commit (4c04f88), all 176 checks of the upstream CI pass. That covers Linux with GCC 11 to 14 and clang 16 to 18 (C++20 and C++23), MSVC with static and shared builds, macOS arm64 and Intel, and the test targets of all formats. Not tested: a Validator() that runs in another DLL than the parser (see Limits).

Stacking

Based on fix-700-prereqs: fork PRs #9, #10 and #11, plus one commit that adapts a test of #10 to the message of #11. On that base the "before" message carries the prefix from #11 (Could not default-construct Person, ...).

🤖 Generated with Claude Code

@clockblocker
clockblocker force-pushed the fix-700-deferred-defaults branch from 4c04f88 to bf178ee Compare November 14, 2026 03:25
@clockblocker
clockblocker changed the base branch from fix-700-prereqs to main November 14, 2026 03:25
@clockblocker
clockblocker force-pushed the fix-700-deferred-defaults branch from bf178ee to 9fa1d1d Compare November 16, 2026 05:18
@clockblocker
clockblocker force-pushed the fix-700-deferred-defaults branch from 9fa1d1d to 2d33403 Compare November 21, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reading JSON into a struct with a DefaultVal field and a size validator field always raises a size validation error

1 participant