Repository navigation
Fix the analysis engine's artifacts (spec 005) - #14
Merged
Merged
Conversation
Investigation verdicts, design research, plan, data model, contracts, quickstart and the evidence scripts of spec 005; spec revised after the investigation (corners as places, cancelled out-lap rule).
Copied unchanged from src/demo/insightsMatch.ts on branch 003-demo-mode (spec 003), with its tests, into a module outside src/demo/ so the local analysis cache can use the same tolerance rule as the demo's seed file. Spec 003 imports it from here when it is updated with main.
…rty tests The cut holds laps 6-8 of sample-recording-v1 with only the fields the analysis engine reads, written with 9 significant digits (lossless for float32), the official lap times and the recording's SHA-256. tools/exportRecordingCut.ts regenerates it byte for byte through a temporary database. CLAUDE.md (Rules) and the README record the exception to 'recordings stay out of git'.
Each lap's elapsed time starts at its first sample extrapolated to 0 m, and both laps are interpolated on the same distances inside the range they both cover, so totalDeltaS is within 0.022 s of the official difference on the published recording and (A,B) is the exact negative of (B,A). cornerLoss interpolates both laps at the stretch bounds. README and CLAUDE.md describe no part of the delta's definition, so neither changes.
…ctions A corner is a stretch of the two laps' mean path (worldPosition on a 5 m grid, curvature smoothed over 35 m) with a radius up to 100 m and at least 20 degrees of turning, so (A,B) and (B,A) have the same corners. Each corner owns the stretch between the fastest points before and after it; brakingStartDistance and exitDistance now hold that stretch, always numeric, contiguous and never overlapping. deltaLossS and the line deviation are measured over it, and severity uses both laps. The geometry thresholds are validated only on Interlagos, a lap without motion gets no corners, and the bounds keep their names for the frontend's contract: all three are recorded in the README and CLAUDE.md.
… stretch Per lap and per corner, the slowest point, the braking start (the first sample of the last braking phase before it), the braking peak and the full-throttle pick-up are read from the lap's samples inside the corner's stretch, so neighbouring corners never share a sample and a brake released before the apex still gives the real braking start. A lap that did not brake or did not reach full throttle is reported with four new booleans (brakedPrimary/Comparison, fullThrottlePrimary/Comparison) and its difference is null, instead of a difference between a braking point and an apex. Coasting and throttle oscillation run from the slowest point to the pick-up or the end of the stretch. The data-model now says the pick-up search includes the slowest sample; the cut's tests require the 8 corners in every pair before any per-corner property.
The system prompt now says what the corrected insights mean (totalDeltaS close to the official difference, corners as places of the track, the stretch bounds, deltaLossS including the exit, the four booleans, coasting from the slowest point) and forbids saying a lap braked or reached full throttle when its boolean is false. The reading writes a phrase when one lap only lifted or did not reach full throttle before the next corner, before the difference phrases. REFERENCE_INPUTS cover each new phrase, so PROMPT_VERSION changes through the prompt, the reading and the reference inputs only. The README and CLAUDE.md no longer state the current prompt's token count, which has not been measured yet.
A cached analysis is served only while its prompt version, model and insights are the current ones: the resolver now builds the pair's insights first and compares them with the stored ones through insightsMatch (the rule of the demo seed). A row whose insights differ is generated again with the key and answers 503 without it, and a pair whose samples are gone is not served from the cache.
The six pairs after the correction, the coverage of the requirements, the CI runs so far and the 400 status for a pair without samples.
The coach's reading puts the full-throttle phrase after the trail braking phrase, where the throttle pick-up phrase goes, so each opportunity follows the order of the corner. PROMPT_VERSION changes to 0cb0df9eaefa147e.
…ending notes The measured 2,600 tokens for PROMPT_VERSION 0cb0df9eaefa147e replace the earlier estimate; the plan states the status codes and the cut's size as implemented; the CI rows and the future work for the request builder are recorded.
Status line with pull request #14, the CI row of the fourth push, and the cache observation of the earlier prompt moved out of the README with the prompt it belongs to.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec 005 (
specs/005-analysis-engine-artifacts/): fixes the artifacts found in the analysis engine's insights. Evidence and verdicts:research.md.What changes
totalDeltaSis within 0.008–0.022 s of the official difference in the six pairs of laps 6–8 (tolerance 0.045 s) and antisymmetric; each corner'sdeltaLossSis measured over its stretch and the losses add up to the total.braking.brakedPrimary/brakedComparisonandthrottle.fullThrottlePrimary/fullThrottleComparison(boolean | null). The braking-point and throttle differences arenullunless both laps braked / reached full throttle.PROMPT_VERSIONchanges to0cb0df9eaefa147e(prompt text, new phrases, reference inputs; engine values alone do not change it, and a test shows it). Measured by the maintainer: 2,600 tokens forclaude-opus-4-8, above the 1,024-token cache minimum.insightsMatch, brought unchanged from003-demo-mode); otherwise it is regenerated with the key and answers 503 without it.Verdicts (research.md, "Summary of verdicts")
totalDeltaSanddeltaLossSnot symmetric / not the official differenceContract (
contracts/insights.md)totalDeltaS,apexDistance,brakingStartDistance/exitDistance(now the stretch's bounds; renaming them is debt for the frontend spec),severity,deltaLossSand the difference fields changes.POST /sessions/:id/analysisfor a pair without samples answers 400 (no samples) whatever the cache holds and whatever the key configuration; an uncached pair without samples and without the key answered 503 before. Same message and shape as today. A pair cached before this change now answers 503 without the key (its insights changed).Tests and the recording cut
src/analysis/recordingCut.jsonis a versioned exception to "recordings stay out of git": laps 6–8 of the published recording, only the fields the engine reads, 643,689 bytes (ceiling 660,000, enforced by a test), regenerated byte-identical bytools/exportRecordingCut.ts. The properties (same corners both ways, no shared points, antisymmetry, total against the official difference) run on it in the CI. The exception is written inCLAUDE.mdand the README. Suite: 265 tests.Dependencies
computeDelta, kept as the reference port, or the API can deliver it. Recorded in spec 003'stasks.md(11b1d8a, pushed).insightsMatchfrommain. Its branch currently conflicts withmain(CLAUDE.md,README.md,src/coaching/coachService.ts), so its PR feat: demo mode for a public demonstration (spec 003) #12 has no CI until it is updated; that decision is the maintainer's.Manual steps done
M1: the six messages read and approved (one change made: the full-throttle phrase moved to the throttle slot). M2: token count measured by the maintainer.
Known limitations added
Geometry thresholds validated only on Interlagos; a lap without motion samples gets no corners; the apex depends on the pair of laps compared (future work: a fixed corner map per track).
Out of scope
The frontend; demo mode and its seed (spec 003); flashback and out-lap (their spec).
CI
Last push: head
eac4ebc, run40220823735(pull_request),typecheckandtestcompleted / success (jobs API).