story-131: A filed query answers for a scope of any size an inspection hands it

[jerodw]

The tracker tier of the inspector's dedupe had never run. Every post-story inspection from story-101 through story-130 reports "dedupe did not run" in its run's events.log.

The cause was arithmetic. The query branch searched the tracker once per path, sequentially; one search costs roughly 0.85s against this tracker, and story-130's inspection carried a 60-file scope — close to a minute against a 30-second bound. The command was killed partway through and never answered, and dedupe_ran requires every scope's query to have answered.

The fix

Batch the searches. Path markers are quoted and OR'd into one search, BATCH at a time (default 20), and the pages are unioned through the composition that already deduplicates by URL. The cost of a search is nearly flat in batch size — 5 markers measured 0.83s, 10 measured 0.90s, 20 about 1.0s — so a 60-file scope becomes three searches rather than sixty.

Neither the harness's bound nor the scope handed to the command changed. The scope is deliberately not capped: a capped scope means inventing a partial answer, and an answer here means the whole question was answered.

Batching is only safe with a fallback, and it has one. A search is capped at LIMIT results. With one path per search that cap is per path; with many paths in one search, a filled page could be several paths' worth of issues truncated, and a truncated page read as complete is a duplicate filed. So a batch whose page fills to the limit, and a batch whose search fails, are both re-asked one path at a time. That also keeps it safe on a tracker whose limit on query length is tighter than this one's — a batch too long to search fails, and failing is what re-asks its paths individually. A per-path search that fails after that still fails the whole answer, which is the behaviour this branch has always had.

tests/test_a_filed_query_scales_with_its_scope.py holds the batched answer to what the per-path loop returned, which is what makes the optimisation safe rather than merely faster.

Making the failure visible

The failure was honest — the run said dedupe did not run — but it was a clause at the end of an event line, and it was true on every story for thirty stories without being read as the standing failure it was. dedupe_ran is now written on every inspection record in the tracked history, including the ones where it succeeded: a boolean has a false to say, and absence would mean either that or a record written before the field existed. "Which inspections ran without dedupe" is now a question one read of a tracked file answers.

Planned from the brief filed under 208.

🤖 Generated with Claude Code

https://claude.ai/code/session_01U1HiUf7CrMjZMj32TySdiH