Skip to content

perf: do each block's open-time work once in main's parse (huge parse 325 → 225 ms) - #325

Merged
lidonggui merged 3 commits into
mainfrom
perf/main-parse-once
Oct 11, 2026
Merged

lidonggui merged 3 commits into
mainfrom
perf/main-parse-once

Conversation

@lidonggui

Copy link
Copy Markdown
Collaborator

What

Main's parseDocument repeated per-block work on open. Profiled on the 8 MB document (~320 ms in Node):

  • Fused ownership + key. engineOwnedSemanticKey(kind, md) returns canSkipDialectEnrich ? engineSemanticKey : null in one parse. Lists, tables, footnote and link definitions used to be parsed twice (tables: 24 + 22 ms). enrichSpanNone uses it. engineSemanticKey shares the list and table key builders, so the two can't drift.
  • One-shot native hash. Main installs nativeSha256, which calls crypto.hash('sha256', v, 'hex') (Node ≥ 21.7; Electron ships 24) and falls back to createHash. Same digest, without a Hash object per block: 44 → 19 ms for 43,970 blocks. Saves get the benefit too.
  • Line endings. detectLineEnding answers lf immediately when the text has no \r, otherwise steps through with indexOf (was a 24 ms per-character loop). toLf returns LF text unchanged.

Correctness

tests/unit/main-parse-once.test.ts:

  • the fused answer equals asking the two apart for every golden block under its own kind, and for every golden block plus dialect samples under all nine kinds that parse once (both null and non-null answers exercised)
  • nativeSha256 equals createHash for strings (incl. lone surrogates) and bytes
  • detectLineEnding equals the old scan on edge cases and on LF/CRLF goldens
  • toLf output unchanged

Numbers (huge, 8 MB, 43,970 blocks)

before after
parseDocument, Node, median of 11, ×3 interleaved 335 / 322 / 319 ms 221 / 233 / 224 ms
packaged main parseDocument, profiled ×3 428 / 361 / 377 ms 249 / 255 / 279 ms
packaged open reply, profiled ×3 878 / 738 / 736 ms 579 / 566 / 630 ms
bench open huge, BENCH_RUNS=5, ×2 1,404 / 1,371 ms 1,303 / 1,232 ms
bench open large 548 / 524 ms 513 / 553 ms (noise)

Packaged runs: Linux x64, Xvfb, interleaved with the build before. Typecheck and full unit suite green.

Left in main's parse: engine split ~60 ms, tryInlineNodesFromSource building PM nodes just to test ownership ~40 ms, lazy span objects ~20 ms, GC.

engineOwnedSemanticKey asks canSkipDialectEnrich and engineSemanticKey in
one parse (lists, tables, footnote and link definitions were parsed twice).
Main hashes with Node's one-shot crypto.hash instead of a Hash object per
block. detectLineEnding and toLf skip text with no carriage return.
# Conflicts:
#	docs/performance/measurements.md
@lidonggui
lidonggui merged commit 4a214ea into main Oct 11, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant