The pinned Cinference runtime rejects strict=true and constrained JSON/grammar output because it cannot guarantee the generated shape. Normal tool calls work, but the caller must validate their arguments. Franci requested tracking of this engine limitation.
Determine whether an upstream contribution or maintained patch can add actual constrained decoding for tool arguments and structured JSON. Define a supported JSON Schema subset and explicit rejection for unsupported schema features. Do not implement this as stripping strict flags or post-hoc JSON cleanup while claiming a generation guarantee.
Validate required fields, nested types, enums, extra properties, escaping, invalid schemas, streaming, reasoning transitions, tool-result replay and all advertised HTTP protocols. Measure prefill/decode/memory costs with NVFP4, K8V4 and DFlash2; correctness must survive speculative verification and batched requests.
Acceptance: a documented feasibility/architecture decision and bounded conformance/performance evidence; if implementation is selected, pin the runtime and add regressions before advertising support. This is tracking, not authorization to fork, deploy, change defaults or borrow the GPU now.
Related: #28/#29 and the separate harness compatibility and tool-choice enforcement issues. Source: https://github.com/satellitedown/cinference/blob/b74044fb0a319cd2a737cb7108012e6344b96dac/docs/serving.md
The pinned Cinference runtime rejects strict=true and constrained JSON/grammar output because it cannot guarantee the generated shape. Normal tool calls work, but the caller must validate their arguments. Franci requested tracking of this engine limitation.
Determine whether an upstream contribution or maintained patch can add actual constrained decoding for tool arguments and structured JSON. Define a supported JSON Schema subset and explicit rejection for unsupported schema features. Do not implement this as stripping strict flags or post-hoc JSON cleanup while claiming a generation guarantee.
Validate required fields, nested types, enums, extra properties, escaping, invalid schemas, streaming, reasoning transitions, tool-result replay and all advertised HTTP protocols. Measure prefill/decode/memory costs with NVFP4, K8V4 and DFlash2; correctness must survive speculative verification and batched requests.
Acceptance: a documented feasibility/architecture decision and bounded conformance/performance evidence; if implementation is selected, pin the runtime and add regressions before advertising support. This is tracking, not authorization to fork, deploy, change defaults or borrow the GPU now.
Related: #28/#29 and the separate harness compatibility and tool-choice enforcement issues. Source: https://github.com/satellitedown/cinference/blob/b74044fb0a319cd2a737cb7108012e6344b96dac/docs/serving.md