fix(harness): guarantee a resolvable user turn for non-native-tool models - #86
Conversation
…dels
A model whose profile reports no native tool calling is driven through
its own Jinja chat template by the serving runtime (LM Studio,
llama.cpp, Ollama), so the outgoing message list has to satisfy that
template and not just the wire schema. Several widely used templates
hard-require a locatable user query; Qwen 3's raises outright:
{%- if ns.multi_step_tool %}
{{- raise_exception('No user query found in messages.') }}
A prompt-guided tool loop reaches that state legitimately. Once the real
user turn has aged out of the window — summarization, a resumed
transcript, a task carried entirely by the system prompt — every
remaining non-system turn is an assistant continuation or a folded tool
result, and the runtime aborts the request with a 400 before the model
is ever called.
Add `ensure_resolvable_user_turn`, applied by the OpenAI-compatible
adapter for no-native-tools profiles after the existing tool-result
coalescing. It inserts a user turn after any leading system turns only
when the list has none. A folded `[Tool results]` turn does not count as
a query — the model requested those itself, and Qwen's template makes
the same distinction — while a non-text turn (JSON, an image) does.
Native-tool models are served through the provider's tool protocol
rather than a template with this guard, so their message list is
untouched.
Refs tinyhumansai/openhuman#5291
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
M3gA-Mind has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 18 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
Comment |
|
Cross-link: the OpenHuman-side half of tinyhumansai/openhuman#5291 is tinyhumansai/openhuman#5309 — it reclassifies the template-parse 400 so the user is told the chat template rejected the request, instead of "your provider rejected the model or temperature setting". The two are independent: #5309 does not bump the |
2 participants
Summary
A model whose profile reports no native tool calling is driven through its own Jinja chat template by the serving runtime (LM Studio, llama.cpp, Ollama), so the outgoing message list has to satisfy that template — not just the wire schema. Several widely used templates hard-require a locatable user query. Qwen 3's raises outright:
A prompt-guided tool loop reaches that state legitimately: once the real user turn has aged out of the window — summarization, a resumed transcript, a task carried entirely by the system prompt — every remaining non-system turn is an assistant continuation or a folded tool result, and the runtime aborts with a
400before the model is ever called.Reported downstream as tinyhumansai/openhuman#5291 (LM Studio +
qwen/qwen3.5-9b,supports_native_tools=false, 80 tools, both retries dead).API Or Behavior Changes
harness::tool::ensure_resolvable_user_turn(&[Message]) -> Vec<Message>(re-exported via the existingpub use prompt::*). Guarantees the list contains a user turn a chat template can resolve, inserting one after any leading system turns only when none is present; a list that already has a real user turn is returned unchanged.translate_request_with) for no-native-tools profiles, after the existing tool-result coalescing.Two judgment calls worth reviewer attention:
[Tool results]turn does not count as a resolvable query. The model requested those itself, and Qwen's template makes the same distinction (it skips user turns that are wholly a tool response). The marker string is now a shared const socoalesce_prompt_tool_resultsand the resolvability check cannot drift.!profile.tool_calling, not toprompt_guided_tools. The guard is a property of the template, not of whether tools were sent — so a no-native-tools model with an empty tool list and no user turn is covered by the same normalization.A non-text user turn (JSON, an image) does count as resolvable — an image-only turn is real user input and must not be displaced.
Tests
Six unit tests in
src/harness/tool/prompt_test.rs(real query left alone · insertion after leading system turns · folded tool results do not count · blank turn ignored · image-only turn accepted · insertion at index 0 when there is no system turn) and two adapter tests insrc/harness/providers/openai/test.rs(prompt-guided wire body carries exactly one resolvable user query in system → user → assistant order; native-tool model's messages untouched).cargo fmt --checkcargo clippy --all-targets -- -D warningscargo clippy --all-targets --all-features -- -D warnings— not runcargo build --all-targets— not run separately (covered by the test build)cargo build --all-targets --all-features— not runcargo test(--lib, 1232 passed / 0 failed)cargo test --all-features— not runDocumentation
docs/modules/harness/tool.mdgains a Prompt-Guided Message Shape section documenting both normalization helpers, why the template (not the wire schema) is the constraint for these models, and why a folded tool result is not a query.