Repository navigation
fix: restore extracted benchmark campaign execution and evidence paths - #8
Merged
Merged
Conversation
…-campaign-contracts-30684
LifeOpsBench (Python) — smokeRun ID:
Full artifacts: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repairs the extracted benchmark campaign so its CLI resolves the checkout correctly, explicit mock-command smoke runs do not require unused OpenCode, campaign inventory describes the current suite layout, and producers, resume, publication, and the viewer share the repository's
benchmark_resultsstore. Nonempty legacy result stores fail with an actionable migration error instead of being silently mixed or ignored. Lifecycle scoring now attests the installed contract and scenario files undersuites/.Closes #4. Closes #5. Closes #6. Closes #7.
Built on #3; preserve its runtime-sensitive resume identity and keyless LifeOps workflow repair. This PR contains no live benchmark scores and does not relax publication requirements.
Validation:
PYTHONPATH=<checkout-parent> ELIZA_TRAINING_ROOT=<restored-training-root> python3 -m pytest tests suites/orchestrator/tests -q: 873 passed, 152.72 seconds on9cb9b16. The subsequent merge of Bind benchmark resume to runtime and source identity #3 changes only its previously tested LifeOps workflow.dae3e1d28cfbcf4b915c04ea1e072030529b4bdausing the historical converter. All 11,578 rows reproduce the pinned SHA-25678dd3bc0b8cc98e0637231ae96dee7a63dbe1b5ae3e1b4fd7d862733ff5e162b; no expected hash was changed to accept a different dataset.The ledger now records 72 entries (58 adapters and 14 direct suites). Full comparative campaign counts remain inventory, not completed evaluation; unknown corpus sizes and manual/non-agent lanes remain explicit.