Repository navigation
Fix hexadecimal escapes in chat templates with dinja 1.3.1 - #1342
Merged
Merged
Conversation
leehack
marked this pull request as ready for review
October 23, 2026 11:45
Contributor
|
Chat app preview removed for |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dinja ^1.3.1to render valid Jinja\xHHliterals, including Flint's embedded\x00replacement (Refs Flint chat fails because its embedded template uses unsupported hexadecimal escapes #1335; lexer fix).LlamaEngine.create()regressions, typed-history checks, and malformed-escape cases.Production-readiness scope
Completeness checklist
LlamaEngine.chatTemplatebugfix, no new API or model/runtime support claim (Flint chat fails because its embedded template uses unsupported hexadecimal escapes #1335).PR type guidance
Test Plan
dart run tool/prepare_workspace.dart; initial clean preparation and regenerated consumer locks.dart format --output=none --set-exit-if-changed .dart analyzeand platform boundaries.dart test -p vm -j 1 --exclude-tags local-only— exact-head CI split core/runtime/tooling/native lanes and coverage.dart test -p chrome --exclude-tags local-only— exact-head CI.Matrix Evidence
6a4451af1990d4d8b12bc9ede9a12ce9d575c607Review Notes
audit_flint_consumeraccepted the exact final head/base after the validation-lock and Windows fixture corrections. It took no part in implementation.High-risk regression review
6a4451af1990d4d8b12bc9ede9a12ce9d575c607/09c189fab37f6683b771aea8b9211d86a3439a60.Exact-head independent review and schema-v2 evidence
Fresh independent review accepted all 19 changed files after both CI corrections. The JSON preserves audit-time CI notes; all current-head checks subsequently passed, as linked above. Controls verify hosted 1.3.0 fails the 23 consumer cases; all pass after restoration. Parser and compiled-grammar mutations fail causally and are restored. Independent Git checkout controls reproduce CRLF conversion before the two exact-capture attributes, then preserve both captures byte for byte after the fix. The validation lock remains unchanged under dependency resolution.
The documented local evaluator exits 2 (
unverifiedPrerequisites): repository-local evidence is internally consistent; authenticated external enforcement is intentionally unconfigured, as described in the readiness contract. Manual gates still require current-head CI and zero unresolved threads.{ "schema": "llamadart.high-risk-readiness-evidence", "schema_version": "2.0.0", "test_evidence": { "test/integration/core/template/flint_template_test.dart": { "test_case": "Flint keeps typed argument and result values while sanitizing text", "test_snippet": "'Flint keeps typed argument and result values while sanitizing text'", "production_refs": [ { "path": "lib/src/core/template/handlers/hermes_handler.dart", "snippet": "final template = Template(templateSource);" }, { "path": "lib/src/core/template/template_render_context.dart", "snippet": "rendered['content'] = result is String ? result : jsonEncode(result);" } ], "command": "TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/integration/core/template/flint_template_test.dart", "head_result": "pass", "control_result": "fail", "control_kind": "before-fix", "control_description": "In an isolated copy of final 6a4451a source, pubspec_overrides.yaml selects hosted dinja1.3.0. The exact Flint template fails at lexer escape x before rendering. Remove override and offline pub restores1.3.1; source untouched.", "evidence_notes": "Independently repeated at final 6a4451a: consumer-independent-6a-1.3.0-red.log +1 fixture/-23 consumer cases; Unknown escape x at Hermes Template. consumer-independent-6a-1.3.1-restored.log +24. Final isolated Chrome +23 in consumer-independent-6a-chrome.log. Fixture integrity is separate +1 proof, not a causal case." }, "test/integration/core/template/qwen35_tool_result_test.dart": { "test_case": "typed result history ${malformed ? 'rolls back undeclared tools' : 'preserves schema-directed scalar and container types'}", "test_snippet": "'typed result history ${malformed ? 'rolls back undeclared tools' : 'preserves schema-directed scalar and container types'}'", "production_refs": [ { "path": "lib/src/core/engine/chat_completion_stream_parser.dart", "snippet": "ChatTemplateEngine.parse(" }, { "path": "lib/src/core/template/xml_tool_call_format.dart", "snippet": "final decoded = decodeToolSchemaText(raw, schema);" }, { "path": "lib/src/core/template/tool_schema_utils.dart", "snippet": "return validateToolSchemaValue(raw, schema);" }, { "path": "lib/src/core/template/handlers/qwen3_coder_xml_handler.dart", "snippet": "schemas: tools == null ? null : toolSchemas(tools)," } ], "command": "TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/integration/core/template/qwen35_tool_result_test.dart", "head_result": "pass", "control_result": "fail", "control_kind": "mutation", "control_description": "In isolated copy only, return invalid for schema-declared strings in decodeToolSchemaText. Separate negative control removes undeclared XML tool-name rejection. Both restored.", "evidence_notes": "Final 6a4451a consumer-independent-6a-green.log +28 (Flint23+fixture1+parser4). Repeated consumer-independent-6a-parser-mutant.log +3/-1: typed reconstruction falls back to raw envelope. consumer-independent-6a-rollback-mutant.log +3/-1: malformed unknown tool becomes executable call. Covers auto/required with thought prefixes and none/no-grammar history branch. Mutations restored." }, "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart": { "test_case": "Hermes required calls may follow forced-open reasoning", "test_snippet": "test('Hermes required calls may follow forced-open reasoning', () {", "production_refs": [ { "path": "lib/src/core/template/tool_call_grammar_utils.dart", "snippet": "static String afterForcedOpenThought(String grammar, String endTag) => [" }, { "path": "lib/src/core/template/handlers/hermes_handler.dart", "snippet": "grammar: requiredGrammarAfterThought(" } ], "command": "LLAMA_CPP_GBNF_VALIDATOR=/private/tmp/flint-fix-1335/grammar-validator-build/bin/test-gbnf-validator TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart --run-skipped --name 'Hermes canonical alternatives|Hermes and Mistral compile|Hermes required calls|open schema objects constrain'", "head_result": "pass", "control_result": "fail", "control_kind": "mutation", "control_description": "In isolated copy only, replace afterForcedOpenThought wrapper with bare grammar. Restored after observation.", "evidence_notes": "consumer-independent-6a-grammar-green.log +4 maintained compiled cases: malformed/unknown envelopes, missing/wrong-type fields, nullable nested schemas and required thought prefixes. consumer-independent-6a-grammar-mutant.log -1 forced-thought valid sequence rejected at initial N when afterForcedOpenThought returns bare grammar. Fresh validator source clean HEAD=v0.7.0 commit620d2225cc436ab0b001e3200e590e3b286ef9ab matching native wrapper v0.7.0-2; binary sha256 dbdeb6f33d062eeefcab67225329be6001aa7b7e8f49b7b805c49ae6404ce07d. Mutation restored byte-for-byte." } }, "timestamp": "2026-10-23T11:29:03Z", "correlation_id": "pr1342-flint-consumer-audit-6a4451a", "repository": "leehack/llamadart", "pr_number": 1342, "expected_pr_head_sha": "6a4451af1990d4d8b12bc9ede9a12ce9d575c607", "current_base_sha": "09c189fab37f6683b771aea8b9211d86a3439a60", "pr_author": "leehack", "classification": "high-risk", "surfaces": [ "artifactConsumer", "structuredOutput" ], "required_matrix_row_ids": [ "high-risk-exact-head-independent-qa", "structured-output-adversarial" ], "matrix_row_evidence": { "high-risk-exact-head-independent-qa": { "row_id": "high-risk-exact-head-independent-qa", "result": "pass", "command": "Independent blocking-only audit of exact head 6a4451af1990d4d8b12bc9ede9a12ce9d575c607 against base09c189fab37f6683b771aea8b9211d86a3439a60", "evidence_notes": "Fresh independent re-audit of19 final files. New .gitattributes delta only protects two exact upstream captures. Independent old/new core.autocrlf=true checkout: old template60CRLF/capturedJSON176CRLF; final0CRLF/exact Git hashes. Old checkout integrity fails -1; fixed checkout finalVM28/Chrome23/compiled4 pass. Validation package lock remains pub-stable. Root refreshed final GraphQL with zero threads; exact-head CI still running. No earlier broad/parity counts are claimed as final6a readiness gates." }, "structured-output-adversarial": { "row_id": "structured-output-adversarial", "result": "pass", "command": "TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/integration/core/template/flint_template_test.dart test/integration/core/template/flint_fixture_integrity_test.dart test/integration/core/template/qwen35_tool_result_test.dart\nLLAMA_CPP_GBNF_VALIDATOR=/private/tmp/flint-fix-1335/grammar-validator-build/bin/test-gbnf-validator TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart --run-skipped --name 'Hermes canonical alternatives|Hermes and Mistral compile|Hermes required calls|open schema objects constrain'", "evidence_notes": "Final6a isolatedVM28, Chrome23, compiled4. Hosted1.3.0 control +1/-23 then restored1.3.1 +24. Schema decoding +3/-1, unknown-tool guard deletion +3/-1, grammar bypass -1; all restored. Separate Windows checkout integrity control -1→pass. Actual effect rendering/history; unchanged parsing/grammar conservatively covered. No Flint model/tool-output qualification." } }, "independent_audit": { "auditor_identity": "codex-auditor-flint-consumer", "audit_kind": "codex-adversarial", "audit_head_sha": "6a4451af1990d4d8b12bc9ede9a12ce9d575c607", "audit_base_sha": "09c189fab37f6683b771aea8b9211d86a3439a60", "decision": "accepted", "unresolved_review_threads": 0, "known_pr_caused_p1_regressions": 0, "summary": "Independent blocking re-audit accepted for6a4451a vs09c189f. All19 changed files inspected. Increment fromdcc is four .gitattributes lines: comment/blank plus two narrow -text exact-capture rules. Independent core.autocrlf=true old/new checkout reproduces and repairs byte/hash failure; integrity negative control fails and fixed bytes pass. All final-head behavior/parser/compiled tests and causal controls rerun. Zero known PR-caused P1. Root GraphQL refreshed exacthead/base and zero threads; current CI remains pending. Prior head audits are superseded; no Flint architecture/generation/output tools qualified." }, "structured_output_evidence": { "coverage": { "input_rendering_history": [ "test/integration/core/template/flint_template_test.dart", "test/integration/core/template/qwen35_tool_result_test.dart" ], "compiled_grammar_acceptance": [ "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart" ], "compiled_grammar_rejection": [ "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart" ], "schema_reconstruction": [ "test/integration/core/template/qwen35_tool_result_test.dart" ], "streaming_rollback": [ "test/integration/core/template/qwen35_tool_result_test.dart" ], "tool_choice_thinking": [ "test/integration/core/template/qwen35_tool_result_test.dart", "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart" ], "upstream_parity": [ "test/integration/core/template/flint_template_test.dart" ] }, "impacts": { "inputRenderingHistory": { "applicable": true, "rationale": "Hosted dinja 1.3.1 is consumed by every Template constructor. Flint rendering reaches Hermes, shared context and public create. Tests compare upstream bytes and preserve typed history.", "production_refs": [ { "path": "lib/src/core/template/handlers/hermes_handler.dart", "snippet": "final template = Template(templateSource);" }, { "path": "lib/src/core/template/template_render_context.dart", "snippet": "rendered['content'] = result is String ? result : jsonEncode(result);" } ] }, "outputParsingStreaming": { "applicable": true, "rationale": "No parser or grammar implementation bytes change in this PR. Evidence-only structured paths impose this conservative minimum impact; unchanged production boundaries were inspected and validated with maintained cases and isolated causal mutations.", "production_refs": [ { "path": "lib/src/core/engine/chat_completion_stream_parser.dart", "snippet": "ChatTemplateEngine.parse(" }, { "path": "lib/src/core/template/xml_tool_call_format.dart", "snippet": "final decoded = decodeToolSchemaText(raw, schema);" } ] }, "grammarSchema": { "applicable": true, "rationale": "No parser or grammar implementation bytes change in this PR. Evidence-only structured paths impose this conservative minimum impact; unchanged production boundaries were inspected and validated with maintained cases and isolated causal mutations.", "production_refs": [ { "path": "lib/src/core/template/tool_call_grammar_utils.dart", "snippet": "static String afterForcedOpenThought(String grammar, String endTag) => [" }, { "path": "lib/src/core/template/handlers/hermes_handler.dart", "snippet": "grammar: requiredGrammarAfterThought(" } ] } }, "families": [ { "family": "Flint-template-artifact", "status": "tested", "evidence_test_paths": [ "test/integration/core/template/flint_template_test.dart" ], "rationale": "Exact upstream template/captured apply-template emissions at model revision 468307a1b38259e9e4c9d4fe53d4e3b76386572c; seven prompt cases, typed history and public mocked create. Native architecture/generation/output tools are deliberately unqualified (native issue112)." } ] }, "affected_test_paths": [ "test/integration/core/template/flint_template_test.dart", "test/integration/core/template/qwen35_tool_result_test.dart", "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart" ], "evaluation": { "evaluated_at": "2026-10-23T11:31:10.094644Z", "changed_files": [ { "path": ".gitattributes", "status": "modified" }, { "path": "CHANGELOG.md", "status": "modified" }, { "path": "README.md", "status": "modified" }, { "path": "example/basic_app/README.md", "status": "modified" }, { "path": "example/chat_app/pubspec.lock", "status": "modified" }, { "path": "packages/llamadart_validation/pubspec.lock", "status": "modified" }, { "path": "pubspec.yaml", "status": "modified" }, { "path": "skills/llamadart-chat-streaming/SKILL.md", "status": "modified" }, { "path": "test/fixtures/flint/Flint.jinja", "status": "added" }, { "path": "test/fixtures/flint/LICENSE", "status": "added" }, { "path": "test/fixtures/flint/README.md", "status": "added" }, { "path": "test/fixtures/flint/render_upstream.json", "status": "added" }, { "path": "test/integration/core/template/flint_fixture_integrity_test.dart", "status": "added" }, { "path": "test/integration/core/template/flint_template_test.dart", "status": "added" }, { "path": "test/support/flint_template_fixture.dart", "status": "added" }, { "path": "website/docs/changelog/recent-releases.md", "status": "modified" }, { "path": "website/docs/guides/chat-template-and-parsing.md", "status": "modified" }, { "path": "website/docs/platforms/support-matrix.md", "status": "modified" }, { "path": "website/examples/observability/pubspec.lock", "status": "modified" } ], "decision": "unverifiedPrerequisites", "failure_classification": "externalPrerequisitesUnavailable", "message": "Repository-local evidence is internally consistent, but auditor authentication, GitHub App publication, protected-environment provenance, and ruleset enforcement are not available. This is not operational merge readiness.", "external_prerequisites": { "app_installed": false, "protected_environment_configured": false, "independent_auditor_authenticated": false, "ruleset_enforced": false, "diagnostic_message": "No repository-local input can authenticate the dedicated GitHub App, protected environment, independent auditor, or conditional ruleset. See doc/high_risk_pre_merge_readiness.md." } } }