Repository navigation
fix(chat): clean agent sources and a complete trace for streamed answers (v0.7.28) - #474
Merged
Merged
Conversation
- Runtime: a subagent answer given as a bare facts array, a fenced object or with a sentence before it is parsed instead of reaching the answer as raw JSON; facts keep their source_refs. The answering model sees the cited documents next to each fact. - Runtime: streamed turns end with a second, complete `trace` event right before `sources` (agent research, answering model, llm_ms/total_ms). Agent turns report their retrieval (summed searches, hits, reranker) and per subagent name, queries, facts, cited hits and open points. - Weave-API keeps the last trace (comment + test); Weave-Chat shows the new subagent details and the search budget. - contracts/internal-chat.md: additive entry for both. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W9t7FeuDDnBkiM8kKa8MN9
- Parser: JSON inside a reply must have the answer's shape (fact objects
with text, or facts/open_points lists); the first match from any `{`/`[`
wins (raw_decode), so trailing braces or a second object no longer leak
raw JSON. Prose that merely contains an array stays prose, string facts
are kept, a one-object array is unwrapped, and embedded JSON that yields
nothing falls back to the reply as text.
- Closing trace: streamed n8n turns report n8n_ms; the multi-agent path
reports the model its answer call returned; the agent search trace sums
only searches that returned a trace (None otherwise); agent turns report
top_k/final_k as None (the subagents choose them).
- Chat: the search-budget tag stays graph-only, as before.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01W9t7FeuDDnBkiM8kKa8MN9
…ng trace - Parser: a non-list `facts`/`open_points` is ignored instead of being split per character; an intentionally empty fenced or prefixed answer stays empty; JSON starts are found with a regex instead of a per-character loop. - A subagent that searched but answered in prose is asked once more (no tools) for the structured answer, so its facts keep their citations. - Agent sources are the hits a fact cites (all hits if none is cited); the agent retrieval trace's `used` matches them, hits are ordered by score and ranked by position. - The opening stream trace leaves retrieval.hits empty; the hit list goes out once, with the closing trace. - Tests for the multi-agent closing trace, answer model, cited sources and ranks; contract and changelog updated. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W9t7FeuDDnBkiM8kKa8MN9
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W9t7FeuDDnBkiM8kKa8MN9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two problems from an agent answer in Weave-Chat.
1. Raw JSON under "Quellen". The answer listed
1. Feuerwehr: [{"text": …, "source_refs": [{"document_id": …, "chunk_id": 14600}]}].2. Misleading trace. The trace said "No retrieval call for this turn", although the agents searched and used the reranker. It also showed only
router_ms.Root causes
_parse_subagent_answertreated that as plain text, so the raw JSON became the "fact" and its citations were lost.traceevent went out before the agent research. Retrieval, the answering model andllm_ms/total_mstherefore never reached the UI.Runtime
agents.py): accepts bare fact arrays, fenced JSON and JSON with prose before or after it. Embedded JSON must match the answer's shape; the first match found byraw_decodewins. String facts are kept. A non-listfactsis ignored. An intentionally empty answer stays empty. Prose that merely contains an array stays prose.(evidence: vpn.pdf p. 3)), not only the agent's name.traceevent right beforesources. The opening trace leavesretrieval.hitsempty; the hit list goes out once, with the closing trace.retrieval:usedequals the reported sourcestop_k/final_karenullmodel. Per subagent, the trace now carriesname,facts,cited_hits,queriesandopen_points.n8n_msin the closing trace.Weave-API persists the last
trace(comment and test). Weave-Chat shows the new subagent details in the trace panel.Contract:
contracts/internal-chat.mdhas an additive entry (2026-11-18). Release commit v0.7.28.Affected services
services/runtime,services/api,services/chat,contracts/,services/ingest/CHANGELOG.md,README.mdTest plan
pytest -q services/runtime/backend/tests: 704 passed. New tests cover:tests/test_streaming_api.py: 25 passed, including a test that the closing trace is persistedvitest(components/chat): 119 passed, plustypecheckandlintChecklist
contracts/updated (additive, no version bump)python scripts/weave_config.py check: n/a, configuration unchangedNotes for reviewers
error, there is no closing trace, so the trace of that turn has no hit list. Weave-API does not persist such a turn anyway.Generated by Claude Code