perf(yq): build a write's pristine presentation tree without the subtrees that say nothing (#6880) by newhoggy · Pull Request #6944 · rust-works/succinctly

newhoggy · 2026-11-11T07:04:43Z

Summary

A YAML write builds the document's presentation tree (comments, styles, anchors) and then a second one for its result: a node for every node of the document, even where there is nothing to carry. This holds one word, CommentTree::Silent, for each collection that says nothing.

The write's pristine snapshot collapses a child as soon as the walk finishes it (to_owned_with_comments_pruned), after the comments that belong to it are attached (- # note keys on the tree lining up with its value).
reconcile_presentation returns Silent for a collection whose pristine tree is Silent and whose result is a collection of the same kind, instead of walking it, so the result's tree is never built either. The nesting-depth guard still runs over the result.
Withheld wherever something reads the tree by position or by shape: a document with an alias, a metadata write (style =, head_comment =, tag =), and a plain = whose right side is not a closed literal (.c = .a copies the source subtree's presentation).
Silent is its own variant and not a Leaf: a Leaf over a collection is also what a write that changed a node's kind leaves, and the two are not the same to -P's deferral of a flow collection's trailing comment (head_comment_first_member). Reusing Leaf was this PR's first review finding; the variant makes every site that matches the tree's shape a compile error to decide.

Output is byte-identical to main (see the gates below). Fixes #6880.

Measurements

Interleaved release A/B against main @ 13def7c58 (builds from git archive, an A/A control of identical source built twice), .zzz = 1 with -o yaml unless noted. Peak RSS under memcap.py, median of 3 (5 at 20 MiB on the M4 Pro); times are medians of 5.

peak RSS (MiB) / time (s)	7950X before	7950X after	M4 Pro before	M4 Pro after
`users` 20 MiB	595	366	629	388
`users` 20 MiB, `-P`	718	365	730	393
`users` 20 MiB, `del(.users[0])`	635	506	612	454
`users` 20 MiB, `\|= "X"`	596	365	611	387
`users` 100 MiB	2899 / 10.32	1760 / 8.52	3061 / 6.58	1966 / 6.04
`comprehensive` 100 MiB	2093 / 8.31	1454 / 7.29	2094 / 5.44	1507 / 4.95
`nested` 100 MiB	1628 / 7.12	1041 / 6.43	1630 / 4.24	1154 / 3.63

Caveats, stated rather than buried:

The first version was 30% slower on the 7950X (100 MiB users: 10.38 s to 13.43 s) while saving the same memory: it collapsed the finished tree in a pass, which freed millions of cold nodes. Collapsing each child as it is finished (this version) frees it while it is cache-hot and is 10-25% faster on the 7950X and 8-14% on the M4 Pro.
Shapes that cannot collapse do not move. The . fast path, JSON-output write, anchors, flow, strings, sequences and the gated .c = .users[0] copy are within the 5 MiB that two builds of the same source differ by. navigation saves 3%: it carries a quoted timestamp in every record, so no record is silent (perf(yq): a record with one field that says something keeps a presentation node for every field of it (#6880 follow-up) #6955).
M4 Pro peak RSS is noisy for nested (the same binary read 267-271 or 365-378 MiB at 20 MiB, and 1145 to 1553 at 100 MiB). macOS keeps freed pages resident until the kernel purges them; read the 7950X column, whose runs agree to a MiB, for the size of the saving. The M4 Pro also carried other sessions' load while timed (load average 1-2), so its times are interleaved ratios, not quiet-box figures.

Review of this PR (`/code-review max`) and what came of it

A real defect, fixed: with -P, a flow collection's trailing comment was dropped when the next item was a silent block collection. Every Silent site is now decided by the compiler, head_comment_first_member grows the one node that holds the comment, and tests/yq_cli/i6880.rs pins it (it fails with the growth disabled; verified by mutation). Also done: the test module's OutputConfig literal no longer breaks on a new field (one helper, after rebasing onto #6935), the depth-guard tests recurse the dense arms again (a tree that says something at every level), the perf-guard.py override for users_yq_del_select (-11% instructions on both architectures), a named predicate and an enum in place of a bool beside under_alias, and the changelog restated. The identity claim the review falsified is replaced by the gates below.

My own follow-up to that review: a 29k-run structural fuzz, classified against yq, showed that growing every Leaf (the first fix) changed ten outputs main gets right by accident; that is why Silent exists. The cases where main is wrong are #6954.

Test plan

cargo test --features cli,simd,regex,serde (full suite) on the final commit
scripts/pre-push.sh --full: ledger, changelog, fmt, release build, three clippy variants, rustdoc x3, cargo test --no-default-features
Output identity, 7950X, baseline against branch, final build: 7,728 runs over 56 filters (writes, del, |=, +=, *=, copies, constructions, -P, metadata writes, reads) x 15 hand-written documents (plain, head comment, line comment, quoted, flow, tagged, anchored, multi-document, empty, keys) and 8 generated 1 MiB ones x 6 flag sets: 0 differences; and 16,200 runs of a structural fuzz (flow collections with trailing comments beside silent block items, - # note elements, nested sequences, 7 writes, -P and not): 0 differences
Unit tests: what keeps a node from collapsing, Silent reads as the dense tree and unfolds on a write, the pruned snapshot against the dense one through reconcile_presentation and emit_yaml_value (and that the shortcut fired), the gate predicate, the - # note element, the depth guards
CLI pins in tests/yq_cli/i6880.rs, rows captured from yq v4.53.3 (mutation-checked: the growth, and the metadata-write gate)
Peak RSS and wall time A/B on both machines (above)
/code-review ultra for the deepest pass, if a human wants one

…rees that say nothing (#6880) A YAML write builds the document's presentation tree and then a second one for the result: a node for every node of the document, even where there is no comment, style, anchor, tag or spelling to carry. The pristine snapshot now holds the one-word empty tree for each subtree that says nothing, collapsed by the parent as the walk finishes the child (so `- # note` is attached first), and reconcile_presentation answers an empty pristine container of the same kind with the empty tree instead of walking it. Not when something reads or writes the tree by position or shape: a document with an alias, a metadata write, a tag write, or a plain = whose right side is not a literal.

…irst version's slowdown

…ld for its first member (#6880 review) -P defers a flow collection's trailing comment to the next item's first member, and head_comment_first_member only matched Object and Array trees, so a silent block item (now one empty leaf) dropped the comment. A leaf over a non-empty collection now grows the one node that holds it, which also fixes the same loss for an item a write changed the kind of, where yq keeps the comment. Also: one named is_silent_leaf predicate, a SilentSubtrees enum in place of a bool next to under_alias, collapse_if_silent private, depth-guard tests that recurse the dense arms again, one test snapshot helper, and CLI pins in tests/yq_cli/i6880.rs.

…override for the faster write row (#6880 review) users_yq_del_select does about 11% less work (instruction counts, both architectures), which the guard fails on in either direction. Unit pins for assign_reads_tree_by_shape and for the shortcut having fired, one CLI row for the metadata-write gate, and the changelog and plan claims restated (9-25% faster, and the one output change).

…#6880 review) A Leaf over a collection is also what a write that changed a node's kind leaves, and the two are not the same to head_comment_first_member: growing every such Leaf fixed the -P comment loss but changed output where main matched yq by accident (a held comment that did not reach a non-adjacent kind-changed item), found by a 29k-run structural fuzz classified against yq. CommentTree::Silent(SilentKind) stands for the dense, all-empty tree of the collection and unfolds on a write, so every site that matches the tree's shape is now a compile error to decide, the reconcile shortcut is exact, and a Leaf behaves as before.

…ir its plan bullet

…pshot is taken, drop dead Silent arms, narrow visibility The prune gate walked the expression on every call, reads included; it is now a closure the snapshot calls. Silent arms that no gated pass can reach are gone (exhaustive patterns stay), materialize and is_silent are private and set_key_comment only unfolds an object, the perf-guard bound for users_yq_del_select is 15 (it moved -11%), the test row carries whether it expects a silent part, and the changelog says 8-17% faster, 25% with -P.

github-actions · 2026-11-11T10:41:23Z

Coverage

Total: 92.98% ⚪ 0.01 pp vs main

Comparing 89385de..27b83a1 (merge-base → PR head)

File	Before	After	Δ
`src/bin/succinctly/yq_runner.rs`	94.85%	94.93%	🟢 0.08 pp

🔇 0 ignored region(s), 683 tolerated region(s)

ignore removes the lines from both reports; tolerate keeps them in the reported percentage but scores them against the baseline, so a cross-run flip cannot move a delta. Regions are read from each revision's own source.

File	Kind	Lines	Rev	Reason
`src/bin/succinctly/jq_path_mode.rs`	`tolerate`	81	both	unreachable: the same filter compiled above
`src/bin/succinctly/jq_path_mode.rs`	`tolerate`	89-92	both	defensive: reached only if the interpreter panics, which no test provokes
`src/bin/succinctly/jq_runner.rs`	`tolerate`	1930	both	unreachable: widening the shadow-candidate set never rejects a program the first parse accepted, for the identical reason run_jq's own analogous fallback (#2395) is tolerated -- see that line's own comment (#2950)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	2319	both	unreachable while the walk and the loader agree on what fails: both resolve with resolve_module_in/resolve_data_file_in, parse with parse_program and name a cycle by canonical file, which the fuzz in #3573 held over thousands of programs; kept so drift prints the loader's failure and not `0 compile errors`
`src/bin/succinctly/jq_runner.rs`	`tolerate`	3046-3058	both	unreachable: `try_parse_meta_op` only fires under `ParserMode::Yq` (src/jq/parser.rs), and `rewrite_namespaced_calls` is only reached via `ModuleProcessor::process_program`, which jq_runner's own jq-mode `run` is the sole caller of -- so a `MetaAssign` node can never reach this function (#798)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	3662	both	unreachable by construction: every error this wrapper receives today is a decode or nesting-depth failure the evaluator raised itself, never error(v); kept so a future one is rendered rather than dropped (#2999)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	3663	both	see the arm above (#2999)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	4092	both	unreachable in practice today: `def` is only `None` when `occurrences.module_def()` is `None`, which (given `origin` is `Some`) would require an open run whose innermost frame has no def set at a `check` point that isn't itself inside a module-level def body -- but every module run is a strict chain of `def` nodes (the loader's own defs, each wrapping its dependency stubs INSIDE its own body via `wrap_defs`/`dep_stubs_for`) terminated by the run's own end marker, so a diagnosable call/var/break site is always reached either inside a module-level def's body (module_def Some) or outside every run (origin None) -- and even when `def` is `Some`, `self.defs` is always `ModuleSource::read`'s own re-parse of the exact same file `run_id_for` interned this origin's id from, so `collect_def_sites` always finds the matching (name, arity, ordinal) span. Kept as a defensive fallback rather than a panic/unwrap in case that invariant is ever violated (#3085)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	4341-4344	both	unreachable in a single-process run by construction: `run_id_for` (the sole source of an `origin` id) always inserts a `run_origins` entry for the id it hands back -- from a real load's canonical path, or its own literal-path fallback on a resolve failure -- and a def body only ever gets stamped with an `origin` after its module loaded successfully, so `at` always names a file that existed and was readable moments earlier. Reaching this arm needs that same file to vanish (or become unreadable) in the narrow window between that load and this re-read, entirely outside this process's control (#2964)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	4769	both	unreachable: widening the shadow-candidate set never rejects a program the first parse accepted -- a newly covered name only wraps an already-successful dedicated parse, and a failing one would have propagated its error in the first parse too, so the retry budget is charged at the identical sites in both (#2395)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	5339-5348	both	defensive: the splitter stops only at a fault jq's parser also meets, so `slurp_fault` names one; kept so a disagreement between them prints the old line-0 message rather than none (#5555)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	5850-5858	both	defensive: the splitter stops only at a fault jq's parser also meets, so `jq_stream_fault` names one; kept so a disagreement between them prints the old line placement rather than none (#5555)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	5950-5954	both	defensive: the only malformed document `get_inputs` returns here is a slurped stream, which carries the read that met its fault unless `slurp_fault_at` finds none (unreachable, see there); kept as the line-0 placement every other malformed document gets (#5555)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	6976	both	untestable: stdin failing part-way
`src/bin/succinctly/jq_runner.rs`	`tolerate`	8564	both	unreachable: the prefix parse stopped at value `parsed`, which the splitter either delimits or stopped at, so a read is found; kept so a disagreement falls back to line 0 (#5555)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	10711	both	untestable: stdin failing part-way
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12225-12232	both	unreachable: the splitter is the more lenient of the two and the prefix parse materializes from its spans, so it cannot find fewer values -- the same internal-error guard as the non-resuming branch below (#1064), kept per segment (#4311)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12329-12331	both	unreachable: the read starts on a UTF-8 continuation byte, which no JSON token begins with, so the parser always names a fault in it; kept so a read it could not name is still an error and not silently dropped (#5975)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12346-12348	both	unreachable with the arm above: the parser always names the fault, so the dropped bytes are never counted by hand (#5975)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12508-12514	both	untestable: a read that fails after the file opened
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12551-12555	both	unreachable: the bytes were validated above
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12560-12565	both	untestable: a read that fails after the file opened
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12579-12583	both	untestable: a read that fails after the file opened
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12853	both	unreachable: called with the source just read at the front
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12962-12968	both	unreachable: the splitter is the more lenient of the two parsers and the prefix parse materializes from its spans, so it cannot find fewer values (the guard `read_json_tail` keeps per segment, #4311)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	13053-13060	both	unreachable: the splitter is the more lenient of the two parsers and the prefix parse materializes from its spans, so it cannot find fewer values (the guard `read_json_tail` keeps per segment, #4311)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	13220	both	unreachable: a stitch cuts at a token's first byte, which is ASCII
`src/bin/succinctly/jq_runner.rs`	`tolerate`	13877	both	unreachable: the one source pushed above
`src/bin/succinctly/jq_runner.rs`	`tolerate`	13921	both	defensive: an RS inside a value makes the walk warn (a truncated record), which returns above; kept so a walk that stopped warning cannot leave an RS in a value the lazy route then reads
`src/bin/succinctly/jq_runner.rs`	`tolerate`	14003	both	unreachable: a file holds a document, so there is a last one
`src/bin/succinctly/jq_runner.rs`	`tolerate`	16282	both	unreachable: no M2-streamable program halts, so stats.halt is None here, as in the uncolored arm below (#5415)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	16283	both	unreachable: see the line above (#5415)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	16284	both	unreachable: see the line above (#5415)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	21263	both	unreachable: this test builds only ResolveError::Call values (#3313)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	25819	both	unreachable in a passing suite by design -- the fixed b\
`src/bin/succinctly/main.rs`	`tolerate`	1587	both	unreachable given clap 4.6's own unknown_argument() error constructor: every ErrorKind::UnknownArgument it builds sets ContextKind::InvalidArg to ContextValue::String(arg) in the same call, so this arm only guards a future clap release changing that invariant
`src/bin/succinctly/main.rs`	`tolerate`	1756	both	llvm-cov line-attribution artifact, not unreachable: test_negative_filter_accepted_via_sjq_multicall_alias_3389's '-x' row demonstrably reaches the e.exit() two lines below (exit code 2, confirmed by that test passing), which this if-let's own closing brace sits directly above -- the brace itself is never credited a hit, the same class of artifact eval.rs's own tolerate list documents for other closing braces (#3389)
`src/bin/succinctly/main.rs`	`tolerate`	2012	both	llvm-cov line-attribution artifact, not unreachable: test_negative_filter_boundary_characters_3389's '-x'/'-n1'/'--bogus' rows demonstrably reach the e.exit() two lines below (exit code 2, confirmed by that test passing), which this if-let's own closing brace sits directly above -- the brace itself is never credited a hit, the same class of artifact eval.rs's own tolerate list documents for other closing braces (#3389)
`src/bin/succinctly/main.rs`	`tolerate`	2036	both	the CLI suites drive yq through the `syq` multi-call arm above; this arm is the same call reached only when spelled `succinctly yq` (#2999)
`src/bin/succinctly/output.rs`	`tolerate`	1384	both	unreachable: a writer emits a placeholder whole, so the text never ends inside one (#5497)

643 more not shown: the full report is in the coverage-comment-x86_64 artifact.

Touched files absent from every coverage report (5)

changelog.d/6880.performance.md
docs/log.md
docs/plan/yq-memory-optimization.md
scripts/perf-guard.py
tests/yq_cli/i6880.rs

These files may contain no executable code or may have been omitted from the coverage run.

Patch coverage

Patch: 96.91% (314/324 new lines covered)

File	Patch	Uncovered new lines
`src/bin/succinctly/yq_runner.rs`	96.69% (175/181)	5738, 7666, 9213, 9215, 15857, 15935
`src/jq/eval_generic.rs`	97.2% (139/143)	4014, 4049, 4059, 4069

Uncovered new lines (10)

src/bin/succinctly/yq_runner.rs:5738
src/bin/succinctly/yq_runner.rs:7666
src/bin/succinctly/yq_runner.rs:9213
src/bin/succinctly/yq_runner.rs:9215
src/bin/succinctly/yq_runner.rs:15857
src/bin/succinctly/yq_runner.rs:15935
src/jq/eval_generic.rs:4014
src/jq/eval_generic.rs:4049
src/jq/eval_generic.rs:4059
src/jq/eval_generic.rs:4069

Indirect coverage changes

🔴 0 lines lost coverage, 🟢 3 lines gained coverage on unchanged code.

Indirect changes

src/bin/succinctly/yq_runner.rs:7516 🟢 uncovered → covered
src/bin/succinctly/yq_runner.rs:9116 🟢 uncovered → covered
src/bin/succinctly/yq_runner.rs:9160 🟢 uncovered → covered

_{📦 Full per-file coverage summary · run summary}

Baseline: the report published for b96c9d8, 1 commit before the merge-base, which has none. The deltas also include whatever those commits changed.

github-actions · 2026-11-11T10:42:05Z

Coverage

Total: 93.03% ⚪ 0.01 pp vs main

Comparing 89385de..27b83a1 (merge-base → PR head)

File	Before	After	Δ
`src/bin/succinctly/yq_runner.rs`	94.85%	94.93%	🟢 0.08 pp

🔇 0 ignored region(s), 682 tolerated region(s)

ignore removes the lines from both reports; tolerate keeps them in the reported percentage but scores them against the baseline, so a cross-run flip cannot move a delta. Regions are read from each revision's own source.

File	Kind	Lines	Rev	Reason
`src/bin/succinctly/jq_path_mode.rs`	`tolerate`	81	both	unreachable: the same filter compiled above
`src/bin/succinctly/jq_path_mode.rs`	`tolerate`	89-92	both	defensive: reached only if the interpreter panics, which no test provokes
`src/bin/succinctly/jq_runner.rs`	`tolerate`	1930	both	unreachable: widening the shadow-candidate set never rejects a program the first parse accepted, for the identical reason run_jq's own analogous fallback (#2395) is tolerated -- see that line's own comment (#2950)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	2319	both	unreachable while the walk and the loader agree on what fails: both resolve with resolve_module_in/resolve_data_file_in, parse with parse_program and name a cycle by canonical file, which the fuzz in #3573 held over thousands of programs; kept so drift prints the loader's failure and not `0 compile errors`
`src/bin/succinctly/jq_runner.rs`	`tolerate`	3046-3058	both	unreachable: `try_parse_meta_op` only fires under `ParserMode::Yq` (src/jq/parser.rs), and `rewrite_namespaced_calls` is only reached via `ModuleProcessor::process_program`, which jq_runner's own jq-mode `run` is the sole caller of -- so a `MetaAssign` node can never reach this function (#798)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	3662	both	unreachable by construction: every error this wrapper receives today is a decode or nesting-depth failure the evaluator raised itself, never error(v); kept so a future one is rendered rather than dropped (#2999)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	3663	both	see the arm above (#2999)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	4092	both	unreachable in practice today: `def` is only `None` when `occurrences.module_def()` is `None`, which (given `origin` is `Some`) would require an open run whose innermost frame has no def set at a `check` point that isn't itself inside a module-level def body -- but every module run is a strict chain of `def` nodes (the loader's own defs, each wrapping its dependency stubs INSIDE its own body via `wrap_defs`/`dep_stubs_for`) terminated by the run's own end marker, so a diagnosable call/var/break site is always reached either inside a module-level def's body (module_def Some) or outside every run (origin None) -- and even when `def` is `Some`, `self.defs` is always `ModuleSource::read`'s own re-parse of the exact same file `run_id_for` interned this origin's id from, so `collect_def_sites` always finds the matching (name, arity, ordinal) span. Kept as a defensive fallback rather than a panic/unwrap in case that invariant is ever violated (#3085)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	4341-4344	both	unreachable in a single-process run by construction: `run_id_for` (the sole source of an `origin` id) always inserts a `run_origins` entry for the id it hands back -- from a real load's canonical path, or its own literal-path fallback on a resolve failure -- and a def body only ever gets stamped with an `origin` after its module loaded successfully, so `at` always names a file that existed and was readable moments earlier. Reaching this arm needs that same file to vanish (or become unreadable) in the narrow window between that load and this re-read, entirely outside this process's control (#2964)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	4769	both	unreachable: widening the shadow-candidate set never rejects a program the first parse accepted -- a newly covered name only wraps an already-successful dedicated parse, and a failing one would have propagated its error in the first parse too, so the retry budget is charged at the identical sites in both (#2395)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	5339-5348	both	defensive: the splitter stops only at a fault jq's parser also meets, so `slurp_fault` names one; kept so a disagreement between them prints the old line-0 message rather than none (#5555)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	5850-5858	both	defensive: the splitter stops only at a fault jq's parser also meets, so `jq_stream_fault` names one; kept so a disagreement between them prints the old line placement rather than none (#5555)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	5950-5954	both	defensive: the only malformed document `get_inputs` returns here is a slurped stream, which carries the read that met its fault unless `slurp_fault_at` finds none (unreachable, see there); kept as the line-0 placement every other malformed document gets (#5555)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	6976	both	untestable: stdin failing part-way
`src/bin/succinctly/jq_runner.rs`	`tolerate`	8564	both	unreachable: the prefix parse stopped at value `parsed`, which the splitter either delimits or stopped at, so a read is found; kept so a disagreement falls back to line 0 (#5555)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	10711	both	untestable: stdin failing part-way
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12225-12232	both	unreachable: the splitter is the more lenient of the two and the prefix parse materializes from its spans, so it cannot find fewer values -- the same internal-error guard as the non-resuming branch below (#1064), kept per segment (#4311)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12329-12331	both	unreachable: the read starts on a UTF-8 continuation byte, which no JSON token begins with, so the parser always names a fault in it; kept so a read it could not name is still an error and not silently dropped (#5975)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12346-12348	both	unreachable with the arm above: the parser always names the fault, so the dropped bytes are never counted by hand (#5975)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12508-12514	both	untestable: a read that fails after the file opened
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12551-12555	both	unreachable: the bytes were validated above
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12560-12565	both	untestable: a read that fails after the file opened
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12579-12583	both	untestable: a read that fails after the file opened
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12853	both	unreachable: called with the source just read at the front
`src/bin/succinctly/jq_runner.rs`	`tolerate`	12962-12968	both	unreachable: the splitter is the more lenient of the two parsers and the prefix parse materializes from its spans, so it cannot find fewer values (the guard `read_json_tail` keeps per segment, #4311)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	13053-13060	both	unreachable: the splitter is the more lenient of the two parsers and the prefix parse materializes from its spans, so it cannot find fewer values (the guard `read_json_tail` keeps per segment, #4311)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	13220	both	unreachable: a stitch cuts at a token's first byte, which is ASCII
`src/bin/succinctly/jq_runner.rs`	`tolerate`	13877	both	unreachable: the one source pushed above
`src/bin/succinctly/jq_runner.rs`	`tolerate`	13921	both	defensive: an RS inside a value makes the walk warn (a truncated record), which returns above; kept so a walk that stopped warning cannot leave an RS in a value the lazy route then reads
`src/bin/succinctly/jq_runner.rs`	`tolerate`	14003	both	unreachable: a file holds a document, so there is a last one
`src/bin/succinctly/jq_runner.rs`	`tolerate`	16282	both	unreachable: no M2-streamable program halts, so stats.halt is None here, as in the uncolored arm below (#5415)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	16283	both	unreachable: see the line above (#5415)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	16284	both	unreachable: see the line above (#5415)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	21263	both	unreachable: this test builds only ResolveError::Call values (#3313)
`src/bin/succinctly/jq_runner.rs`	`tolerate`	25819	both	unreachable in a passing suite by design -- the fixed b\
`src/bin/succinctly/main.rs`	`tolerate`	1587	both	unreachable given clap 4.6's own unknown_argument() error constructor: every ErrorKind::UnknownArgument it builds sets ContextKind::InvalidArg to ContextValue::String(arg) in the same call, so this arm only guards a future clap release changing that invariant
`src/bin/succinctly/main.rs`	`tolerate`	1756	both	llvm-cov line-attribution artifact, not unreachable: test_negative_filter_accepted_via_sjq_multicall_alias_3389's '-x' row demonstrably reaches the e.exit() two lines below (exit code 2, confirmed by that test passing), which this if-let's own closing brace sits directly above -- the brace itself is never credited a hit, the same class of artifact eval.rs's own tolerate list documents for other closing braces (#3389)
`src/bin/succinctly/main.rs`	`tolerate`	2012	both	llvm-cov line-attribution artifact, not unreachable: test_negative_filter_boundary_characters_3389's '-x'/'-n1'/'--bogus' rows demonstrably reach the e.exit() two lines below (exit code 2, confirmed by that test passing), which this if-let's own closing brace sits directly above -- the brace itself is never credited a hit, the same class of artifact eval.rs's own tolerate list documents for other closing braces (#3389)
`src/bin/succinctly/main.rs`	`tolerate`	2036	both	the CLI suites drive yq through the `syq` multi-call arm above; this arm is the same call reached only when spelled `succinctly yq` (#2999)
`src/bin/succinctly/output.rs`	`tolerate`	1384	both	unreachable: a writer emits a placeholder whole, so the text never ends inside one (#5497)

642 more not shown: the full report is in the coverage-comment-ARM64 artifact.

Touched files absent from every coverage report (5)

changelog.d/6880.performance.md
docs/log.md
docs/plan/yq-memory-optimization.md
scripts/perf-guard.py
tests/yq_cli/i6880.rs

These files may contain no executable code or may have been omitted from the coverage run.

Patch coverage

Patch: 96.91% (314/324 new lines covered)

File	Patch	Uncovered new lines
`src/bin/succinctly/yq_runner.rs`	96.69% (175/181)	5738, 7666, 9213, 9215, 15857, 15935
`src/jq/eval_generic.rs`	97.2% (139/143)	4014, 4049, 4059, 4069

Uncovered new lines (10)

src/bin/succinctly/yq_runner.rs:5738
src/bin/succinctly/yq_runner.rs:7666
src/bin/succinctly/yq_runner.rs:9213
src/bin/succinctly/yq_runner.rs:9215
src/bin/succinctly/yq_runner.rs:15857
src/bin/succinctly/yq_runner.rs:15935
src/jq/eval_generic.rs:4014
src/jq/eval_generic.rs:4049
src/jq/eval_generic.rs:4059
src/jq/eval_generic.rs:4069

Indirect coverage changes

🔴 0 lines lost coverage, 🟢 3 lines gained coverage on unchanged code.

Indirect changes

src/bin/succinctly/yq_runner.rs:7516 🟢 uncovered → covered
src/bin/succinctly/yq_runner.rs:9116 🟢 uncovered → covered
src/bin/succinctly/yq_runner.rs:9160 🟢 uncovered → covered

_{📦 Full per-file coverage summary · run summary}

Baseline: the report published for b96c9d8, 1 commit before the merge-base, which has none. The deltas also include whatever those commits changed.

newhoggy added 7 commits November 11, 2026 19:41

docs(yq): record the #6880 tree collapse, its measurements, and its f…

21bf61d

…irst version's slowdown

docs(yq): restate the #6880 measurements on the final build, and repa…

47d35a9

…ir its plan bullet

docs(yq): name the Silent variant in the pruned builder's doc (#6880)

3552c3f

newhoggy force-pushed the issue-6880-yq-collapse-empty-comment-tree branch from 679e4d3 to 3552c3f Compare November 11, 2026 10:26

newhoggy mentioned this pull request Nov 11, 2026

perf(yq): a write over a document with no presentation still builds two full comment trees, 169 MiB of the 436 MiB peak at 20 MiB (#6815 follow-up) #6880

Closed

docs(yq): link only public items from the Silent variant's doc (#6880)

27b83a1

newhoggy added this pull request to the merge queue Nov 11, 2026

Merged via the queue into main with commit bc38857 Nov 11, 2026
53 checks passed

Navigation Menu

Repository navigation

perf(yq): build a write's pristine presentation tree without the subtrees that say nothing (#6880) - #6944

perf(yq): build a write's pristine presentation tree without the subtrees that say nothing (#6880)#6944
newhoggy merged 9 commits into
mainfrom
issue-6880-yq-collapse-empty-comment-tree

newhoggy commented Nov 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 11, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

newhoggy commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Measurements

Review of this PR (/code-review max) and what came of it

Test plan

Uh oh!

github-actions Bot commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage

Patch coverage

Indirect coverage changes

Uh oh!

github-actions Bot commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage

Patch coverage

Indirect coverage changes

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

newhoggy commented Nov 11, 2026 •

edited

Loading

Review of this PR (`/code-review max`) and what came of it

github-actions Bot commented Nov 11, 2026 •

edited

Loading

github-actions Bot commented Nov 11, 2026 •

edited

Loading