Skip to content

chat: add per-run reasoning timing metadata - #552

Merged
Justin Chen (justschen) merged 2 commits into
mainfrom
agents/reasoning-timing
Oct 30, 2026
Merged

Justin Chen (justschen) merged 2 commits into
mainfrom
agents/reasoning-timing

Conversation

@justschen

@justschen Justin Chen (justschen) commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add optional producer-recorded timing for each contiguous reasoning run, so clients can retain labels such as "Thought for 5s" across subscriptions, reconnects, and restored history instead of timing replay or combining unrelated runs.

tool -> reasoning (5s) -> markdown -> reasoning (12s) -> tool

This represents two reasoning parts with durations 5000 and 12000, not a message/group total of 17000. The interval starts at the first nonempty reasoning text and ends at the next markdown/tool part or turn termination. SDK block IDs, message completion, and sideband metadata do not split a run.

  • Add optional startedAt (ISO 8601) and duration (non-negative integer milliseconds) to ReasoningResponsePart and ChatReasoningAction.
  • Support timing-only updates with empty content; omitted fields preserve existing values, including recorded zero.
  • Update TypeScript, Go, Rust, Swift, Kotlin, and .NET reducers and regenerate client models and JSON schemas.
  • Document preservation of individual run identities and intervening content, with shared conformance fixtures and a changelog fragment.

Companion implementation: microsoft/vscode#345121. It records Copilot runs, preserves their ordering across SDK messages, and displays their separate durations after reload without removing conversation grouping.

Compatibility

The wire fields are optional and require no protocol-version bump. Hosts publish final timing before the turn's terminal action and retain individual runs when restoring history. Missing timing remains unknown; clients must not infer it from their own clock or invent values for older/interrupted history.

Native client consumers should rebuild against the updated generated models. Rust struct literals need the new optional fields (normally None when unknown); Kotlin callers using positional constructor arguments should use named arguments when upgrading. This is wire-compatible, not a promise of native binary compatibility.

Validation

Directory Command Result
Repository root npm run generate Passed; generated output is clean
Repository root npm test Passed: 615 tests; 100% reducer branch coverage; includes type-check, lint, metadata, changelog, and generated-output checks
clients/typescript npm run typecheck Passed
clients/typescript npm test Passed: 77 tests
clients/go go test -count=1 ./... Passed without test cache
clients/rust cargo test --workspace -j 2 Passed
clients/swift/AgentHostProtocol swift test -j 2 Passed: 96 tests
clients/kotlin ./gradlew test --no-daemon --console=plain --max-workers=2 Blocked: Java runtime unavailable; the prescribed container fallback was also unavailable because Podman is not installed
clients/dotnet dotnet build --configuration Release --disable-build-servers -maxcpucount:2 Blocked: .NET SDK unavailable

Validation includes the shared 5s/12s run-separation fixture. Kotlin/.NET execution remains for CI.

Carry producer-recorded reasoning start times and durations through protocol state, timing-only actions, and all client reducers. Preserve unknown timing and recorded zeroes across snapshots and completed turns.

Co-authored-by: Copilot <[email protected]>
Define thinking runs independently of SDK block and message boundaries. Preserve their identities and individual durations around markdown and tool output, with shared 5s/12s conformance coverage.

Co-authored-by: Copilot <[email protected]>
Justin Chen (justschen) added a commit to microsoft/vscode that referenced this pull request Oct 30, 2026
Keep one reasoning identity and clock until markdown, a new tool call, or turn termination. Persist the observed response-part ordering and update runs that span SDK messages, rather than storing message-level totals.

Restore separate run durations and canonical tool positions, retain conversation grouping with per-run labels, and cover 5s/12s boundaries across database restart and UI reload. Pin the clarified contract from microsoft/agent-host-protocol#552.

Co-authored-by: Copilot <[email protected]>
@justschen Justin Chen (justschen) changed the title chat: add optional reasoning timing metadata chat: add per-run reasoning timing metadata Oct 30, 2026
@jackfreemanceo-dotcom

This comment was marked as spam.

@justschen
Justin Chen (justschen) marked this pull request as ready for review October 30, 2026 17:31
@justschen
Justin Chen (justschen) merged commit d7a5579 into main Oct 30, 2026
9 checks passed
@justschen
Justin Chen (justschen) deleted the agents/reasoning-timing branch October 30, 2026 17:31
Justin Chen (justschen) added a commit to microsoft/vscode that referenced this pull request Oct 30, 2026
Reference the merge commit of microsoft/agent-host-protocol#552. The generated protocol sources are unchanged.

Co-authored-by: Copilot <[email protected]>
Justin Chen (justschen) added a commit to microsoft/vscode that referenced this pull request Oct 30, 2026
* chat: persist agent-host reasoning timing

Preserve producer-owned timing across live updates, reconnection, and saved chat history so fallback thinking headers retain their elapsed time. Record Copilot reasoning against durable assistant-message IDs without timing replay or inventing missing historical durations.

Sync the optional protocol fields from microsoft/agent-host-protocol commit f9d401d7.

Co-authored-by: Copilot <[email protected]>

* chat: preserve timing for each contiguous thinking run

Keep one reasoning identity and clock until markdown, a new tool call, or turn termination. Persist the observed response-part ordering and update runs that span SDK messages, rather than storing message-level totals.

Restore separate run durations and canonical tool positions, retain conversation grouping with per-run labels, and cover 5s/12s boundaries across database restart and UI reload. Pin the clarified contract from microsoft/agent-host-protocol#552.

Co-authored-by: Copilot <[email protected]>

* chat: fix imported and late reasoning timing

Persist imported response projections against synthesized Copilot event IDs before resume. Refresh timing labels on finalized reasoning sections, including array and grouped content, and keep AHP timing provider-owned even when optional timing fields are absent.

Add import/reopen, late visible/ARIA label, and untimed reconnect regressions while preserving local-provider measurement.

Co-authored-by: Copilot <[email protected]>

* agentHost: pin merged reasoning timing protocol

Reference the merge commit of microsoft/agent-host-protocol#552. The generated protocol sources are unchanged.

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants