Skip to content

perf: decouple preloader read-ahead depth from thread count - #27

Open
brandonocasey wants to merge 3 commits into
encounter:mainfrom
rom-weaver:perf/preloader-read-ahead
Open

brandonocasey wants to merge 3 commits into
encounter:mainfrom
rom-weaver:perf/preloader-read-ahead

Conversation

@brandonocasey

@brandonocasey brandonocasey commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor

While digging into ~1s progress stalls during RVZ extraction, I noticed the preloader stats in the stall window showed io: 98.56%, req: 30.6ms, util: 51% while every other window sat around 2% utilization. The threads were mostly idle and the LRU cache was ~94% empty, but the consumer was still blocked on a slow read. The look-ahead in Preloader::fetch only requests num_threads groups ahead (8 MiB at 4 threads), which is not enough to ride out an I/O latency spike in a dense region of the disc.

With preloading enabled, this gives read-ahead a floor of 16 groups (~32 MiB) and sizes the cache to fit the window plus some reuse room, so prefetched groups do not get evicted before use. More threads still deepen the window like before. With preloader_threads = 0, preloading stays disabled and reads use the existing synchronous path instead of queuing work with no workers. At four threads, the minimum window changes from 8 MiB to ~32 MiB.

@brandonocasey
brandonocasey force-pushed the perf/preloader-read-ahead branch from b56fe1e to e77f73e Compare July 22, 2026 22:36
brandonocasey added a commit to rom-weaver/rom-weaver that referenced this pull request Jul 22, 2026
The read-ahead window was hardcoded to num_threads sector groups
(8 MiB at the browser default of 4), so one OPFS latency spike in a
dense disc region froze RVZ extract progress for the spike duration
while preloader threads sat 93-98% idle and the LRU cache ~94% empty.
Floor the window at 16 groups (~32 MiB) independent of thread count
and size the cache to the window plus a reuse tail so prefetched
groups survive until consumed. Mirrors upstream encounter/nod#27.
Comment thread nod/src/disc/preloader.rs Outdated

/// Minimum number of sector groups to request ahead of the consumer. 16 groups is ~32 MiB,
/// enough to ride out I/O latency spikes without stalling. More threads still deepen the
/// window; this floor only matters when the thread count is low.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind dropping or rewriting this comment?

Comment thread nod/src/disc/preloader.rs Outdated

impl PreloaderCache {
/// Size the cache to fit the read-ahead window plus some reuse room, so prefetched groups
/// aren't evicted before they're used. Keeps at least the old fixed capacity of 64.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

@brandonocasey

Copy link
Copy Markdown
Contributor Author

all resolved

The read-ahead window was hardcoded to num_threads sector groups (4
groups = 8 MiB at the default), so a single I/O latency spike in a
dense disc region stalled the consumer for the full spike duration
while preloader threads sat 93-98% idle and the LRU cache sat ~94%
empty. Give the window a 16-group (~32 MiB) floor independent of
thread count and size the cache to hold the full window plus a reuse
tail so prefetched groups are not evicted before use. More threads
still deepen the window and add I/O concurrency; the floor only
matters when the thread count resolves low.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants