Skip to content

fix(engine-mcp): a one-word name spelled in prose has to earn its lead (PRD-865 follow-up) - #647

Merged
jonit-dev merged 2 commits into
developfrom
fix-name-lead
Nov 10, 2026
Merged

jonit-dev merged 2 commits into
developfrom
fix-name-lead

Conversation

@jonit-dev

Copy link
Copy Markdown
Contributor

Follow-up to #644 (PRD-865), which the merge-queue landing shipped with a defect a third review pass then found. One commit, off develop at a6f32923b.

The defect

A spelled one-word name started at NAME_BASE_LEAD = 2, above the situation scores a prose request produces, so ordinary English that happens to be a symbol took row 1 — while the base commit (ecd2a1a56) answered all of these correctly:

request merged (a6f32923b) base this PR
camera follows the player with a third person view ❌ View (a React rectangle) 2.26 ✅ defineGame ✅ defineGame, View at 0.62
the enemy velocity should push the player back when hit ❌ velocity (three/tsl) ✅ gameplay rows ✅ velocity gone
make the water depth fog darker underwater ❌ depth (three/tsl) ✅ Atmosphere ✅ Atmosphere
make the scene feel foggy and moody ❌ Scene 2.95 ✅ no wrong lead ✅ Scene gone
connect two rigid bodies with a hinge so the door swings connect (WebTransport) in the top 3 Joint3D at row 2 connect gone

The fix

A one-word name spelled inside a longer request now scores (NAME_BASE_LEAD + NAME_SHARE_LEAD) * share, and must hold SINGLE_WORD_SHARE_FLOOR of the request and clear the name index's SYMBOL_WEIGHT_FLOOR. A request that is the name, and multi-word names, are untouched, so the names that should lead still do: add godrays, enable ssgi, use rapier, GPUParticles3D sparks, Area3D trigger, characterbody3d.

The four false leads above are now red-green tests in packages/engine-mcp/__tests__/search.spec.ts (they fail on a6f32923b, pass here), plus one keeping the names that must still lead.

Effect on the frozen held-out set

held out (50 rows) base merged this PR
recall@1 0.400 0.620 0.660
recall@5 0.580 0.760 0.760
MRR 0.475 0.689 0.701
request-scope recall@1 0.200 0.400 0.400
mechanic recall@1 0.500 0.632 0.632

All four A/B gates pass; pnpm caps:recall stays 0.938053 (exit 0).

Also in this commit

  • agent-files/.claude/hooks/discovery-gate.mjs and agent-files/.agents/skills/ponytail/SKILL.md still told an agent to open engine_capability_detail on every hit — the two places the router rule change missed. Both now read every row, and the sixteen scaffold trees are re-measured.
  • The alias comment in capability-manifest.spec.ts named the wrong cause: SurfaceCache/RadianceCache are recalled through the name channel and GIBudget/ShadowFilterBudget through situations after the camelCase split — not through summaries.

Gates

pnpm exec vitest run packages/engine-mcp 158 passed / 1 skipped; packages/create-threenative/__tests__/scaffold.spec.ts green with the re-measured trees; instruction-budget, kit-conventions, capability-manifest, mcp-tool-schemas green; pnpm typecheck, pnpm lint, pnpm check:docs, pnpm sync:agents --check, pnpm quality green; pnpm caps:recall exit 0; scripts/capability-search-ab.ts --base ecd2a1a56 exit 0.

A follow-up review of the merged PRD-865 change found that a spelled one-word
name started at NAME_BASE_LEAD = 2, above the situation scores a prose request
produces, so ordinary English that happens to be a symbol took row 1:
"camera follows the player with a third person view" answered the React `View`
rectangle, "the enemy velocity should push the player back" answered a
`three/tsl` velocity buffer, and `scene`, `depth` and `connect` did the same.
The base commit answered all of them correctly and the merged change did not.

A one-word name spelled inside a longer request now scores
(NAME_BASE_LEAD + NAME_SHARE_LEAD) * share and must hold a real share of the
request (SINGLE_WORD_SHARE_FLOOR) and clear the name index's weight floor.
Names spelled as the whole request, and multi-word names, are unchanged, so
`add godrays`, `enable ssgi`, `use rapier`, `GPUParticles3D sparks` and
`Area3D trigger` still lead.

Held out: recall@1 0.400 -> 0.660 (was 0.620), recall@5 0.580 -> 0.760,
MRR 0.701; every A/B gate passes and `caps:recall` stays 0.938053.

Also closes the two places that still told an agent to open a detail call for
every hit — the discovery-gate hook and the shared ponytail skill — and
corrects the alias comment in the manifest spec: those rows are recalled
through the name channel and the camelCase situation split, not through
summaries.
Conflicts: develop (#632) shipped a stale engine-server.mjs bundle without the
PRD-865 name lead; rebuilt it from engine-mcp source. scaffold.spec.ts moved to
two-hash pins; re-pinned with pnpm scaffold:pin (all 16 kits move shared only).
@jonit-dev
jonit-dev merged commit 86aeb49 into develop Nov 10, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant