fix(game-api): compute live swarm experiment metrics - #70
Merged
Merged
Conversation
Live World Lab experiment metrics always read zero: the snapshot passed an empty resolved-action array to calculateExperimentMetrics even though the retained swarm ticks carry every committed action. Derive them instead through a new calculateRetainedExperimentMetrics, which uses the same resolved-action, provider-attempt and control-change derivation as an all-agents, entire-retained export, so live and exported metrics cannot drift apart. It scopes to the initial and current roster, as the export does, so a captured agent's history still counts. movementDirectionDistribution, longestRepeatedDirectionStreak and recentCellRevisits were declared in the metrics schema but never assigned after the legacy turn records were removed, so they always fell back to their defaults. Compute them from accepted agent-moved events, which already carry both cells, using the existing geographicDirectionBetweenCells classifier. The legacy implementation walked every scoped move as one sequence, so an aggregate streak ran across interleaved agents; each agent's path is now walked separately, with aggregates summing counts and revisits and reporting the longest single-agent streak. Both new tests fail against the previous code. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Final PR of the zero-swarm migration. Follows #69.
What
calculateExperimentMetrics. A newcalculateRetainedExperimentMetricsderives them from the retained swarm ticks, using the same resolved-action, provider-attempt and control-change derivation as an all-agents, entire-retained export. It scopes to the initial and current roster as the export does, so a captured agent's history still counts.movementDirectionDistribution,longestRepeatedDirectionStreakandrecentCellRevisitswere never assigned after the legacy turn records were removed, so they always fell back to schema defaults. They are now computed from acceptedagent-movedevents, which already carry both cells, using the existinggeographicDirectionBetweenCellsclassifier. No new plumbing was needed.Tests
experiment-export.test.ts(new) pins the movement semantics. It includes an interleaved move by a second agent that would produce a streak of three under the old single-sequence walk.simulation-service.swarm.test.tsasserts that live snapshot metrics equal the metrics of an all-agents, entire-retained export after three ticks.Both tests fail when the source fix is reverted.
Docs
ROADMAP.md: the known-open-work entries are resolved and recorded as part of the migration.docs/ARCHITECTURE.mdanddocs/TESTING.md: document the metric semantics and the new coverage.Validation
Run locally by the repository owner and reported green:
🤖 Generated with Claude Code