Skip to content

feat(benchmark): collect harbor atif trajectories - #31

Merged
minixalpha merged 9 commits into
mainfrom
feat/harbor-atif-integration
Sep 3, 2026
Merged

feat(benchmark): collect harbor atif trajectories#31
minixalpha merged 9 commits into
mainfrom
feat/harbor-atif-integration

Conversation

@minixalpha

@minixalpha minixalpha commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • Have the repository-local Harbor adapter request the agent's ATIF-v1.7 trajectory at /logs/agent/trajectory.json, declare native ATIF support, validate the collected document, and reject revision values that Harbor coerces away from strings.
  • Populate Harbor's prompt, completion, cache-token, and cost totals from ATIF final metrics while preserving unknown values and recording explicit complete, partial, missing, or invalid trajectory diagnostics.
  • Document the reproducible Harbor workflow and verified Terminal-Bench run in both Harbor READMEs and the bilingual 0.8.x development notes, add an [Unreleased] changelog entry, and ignore local /jobs/ outputs.

Validation

  • benchmarks/harbor/.venv/bin/pytest benchmarks/harbor/tests — 12 passed.
  • .venv/bin/pytest — 188 passed.
  • terminal-bench/openssl-selfsigned-cert through Harbor 0.21.0 and Docker — reward 1.0, official verifier 6/6, no Harbor exceptions, ATIF-v1.7 validation passed, 11 trajectory steps, and all 10 cost reconciliations resolved on the first attempt.

@minixalpha minixalpha changed the title feat/harbor atif integration feat(benchmark): collect harbor atif trajectories Sep 3, 2026
@minixalpha
minixalpha merged commit 3bb5249 into main Sep 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant