Skip to content

Fix hexadecimal escapes in chat templates with dinja 1.3.1 - #1342

Merged
leehack merged 5 commits into
mainfrom
fix/dinja-1.3.1-template-escapes
Oct 23, 2026
Merged

leehack merged 5 commits into
mainfrom
fix/dinja-1.3.1-template-escapes

Conversation

@leehack

@leehack leehack commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Require hosted dinja ^1.3.1 to render valid Jinja \xHH literals, including Flint's embedded \x00 replacement (Refs Flint chat fails because its embedded template uses unsupported hexadecimal escapes #1335; lexer fix).
  • Add the exact licensed Flint template, seven upstream prompt references, public LlamaEngine.create() regressions, typed-history checks, and malformed-escape cases.
  • Update the docs, consumer skill, example and Unreleased changelogs; regenerate all three tracked dinja consumer locks and preserve exact captured fixture bytes on Windows.

Production-readiness scope

  • User-facing scope: Embedded and custom templates accept two-digit hexadecimal escapes, preserve literal backslashes and Unicode, and report malformed literals as typed inference errors.
  • Supported platforms/paths: Shared Dart rendering on VM and Chrome, including the actual public create/render path.
  • Unsupported or intentionally unavailable paths: Flint model architecture and generated tool-output support remain unqualified on the pinned native runtime.
  • Out of scope / follow-ups: Native runtime qualification stays in llamadart-native #112; runtime pins and parsing/grammar source are unchanged.

Completeness checklist

  • Declared rendering scope is complete.
  • Negative cases fail before backend generation; runtime limits are documented.
  • README, website guide/support matrix, example, skill and both changelogs updated.
  • Exact upstream prompts, public create, Unicode/NUL/marker handling, typed-history preservation and malformed escapes covered.
  • Capability registry: N/A; existing LlamaEngine.chatTemplate bugfix, no new API or model/runtime support claim (Flint chat fails because its embedded template uses unsupported hexadecimal escapes #1335).
  • Security/privacy reviewed: synthetic fixtures; no secrets.
  • Native follow-up tracked above.

PR type guidance

  • Bugfix: Earlier dinja releases reject valid hexadecimal literals before generation. The repaired hosted release fixes the shared renderer on VM/browser.

Test Plan

  • dart run tool/prepare_workspace.dart; initial clean preparation and regenerated consumer locks.
  • dart format --output=none --set-exit-if-changed .
  • dart analyze and platform boundaries.
  • dart test -p vm -j 1 --exclude-tags local-only — exact-head CI split core/runtime/tooling/native lanes and coverage.
  • dart test -p chrome --exclude-tags local-only — exact-head CI.
  • Final-head independent VM 28, Chrome 23, compiled grammar 4; hosted 1.3.0 control fails all 23 consumer cases, restored 1.3.1 passes.
  • Three independent production mutations fail causally: typed reconstruction, unknown-tool rollback, and forced-thought grammar.
  • Final-head desktop bundles and extracted archive help/list smoke pass on all three OSes in CI, preserving clean source provenance.
  • Docs generation/links and all three desktop bundle jobs pass in CI.

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
static-format-analyze Root/examples and platform boundaries Flutter 3.47.1 PASS CI
root-vm Core/runtime/tooling/native regressions Linux, macOS, Windows VM PASS Exact-head CI lanes
root-chrome Shared rendering/browser contracts Chrome PASS Exact-head CI; independent Flint 23/23
coverage-lib Existing library regression coverage Linux VM PASS Exact-head coverage lane
structured-output-adversarial Rendering/history and upstream parity; conservative parser/grammar controls Exact Flint artifact, Qwen35 fixtures, pinned compiled grammar PASS Independent VM28/Chrome23/compiled4 and causal controls below
high-risk-exact-head-independent-qa Blocking adversarial review Head 6a4451af1990d4d8b12bc9ede9a12ce9d575c607 PASS Fresh auditor accepted; zero P1 and unresolved threads
docs-site Site generation and internal links Jaspr PASS Docs Build Check
validation-harness Private consumer dependency and portable artifacts Three desktop OSes, web/Android/iOS inputs PASS Validation bundles; all tracked consumer locks resolve 1.3.1
native Flint qualification Model architecture/generation/tool output Pinned runtime N/A Separate owner issue #112; no support claim

Review Notes

  • Independent review: audit_flint_consumer accepted the exact final head/base after the validation-lock and Windows fixture corrections. It took no part in implementation.
  • CI: all current-head checks passed (selected N/A jobs skipped), run.
  • Initial CI exposed a stale private-harness lock and Windows CRLF conversion of exact capture files. Regenerating the lock and adding two narrow Git attributes fixed both without changing production source or the clean-source guard. Independent Windows-style checkout controls confirm exact fixture hashes.

High-risk regression review

  • Classification: high-risk — artifactConsumer, structuredOutput.
  • Implementation task: flint-consumer-pr; regression worker flint_consumer_regressions; validation-lock worker flint_validation_lock.
  • Independent blocking QA task: audit_flint_consumer; accepted after both CI corrections and re-audit.
  • Exact head / current base: 6a4451af1990d4d8b12bc9ede9a12ce9d575c607 / 09c189fab37f6683b771aea8b9211d86a3439a60.
  • Production-branch deletion, bypass, or miswire proof: Hosted 1.3.0 fails all 23 consumer cases. Three isolated production mutations also fail; all restored. Public create tests prohibit native template fallback.
  • Affected-family real-model/artifact evidence: Exact Flint Q4_K_M embedded template and seven official upstream prompt emissions; hosted dinja 1.3.1 archive independently verified. Fixtures include license, revision and hashes.
  • Explicit unavailable-family or other N/A evidence: Native Flint architecture/generation and emitted tool calls remain outside this rendering repair.
  • Known PR-caused P1 regressions: 0.
  • Unresolved review threads: 0, checked against the exact revision.
Exact-head independent review and schema-v2 evidence

Fresh independent review accepted all 19 changed files after both CI corrections. The JSON preserves audit-time CI notes; all current-head checks subsequently passed, as linked above. Controls verify hosted 1.3.0 fails the 23 consumer cases; all pass after restoration. Parser and compiled-grammar mutations fail causally and are restored. Independent Git checkout controls reproduce CRLF conversion before the two exact-capture attributes, then preserve both captures byte for byte after the fix. The validation lock remains unchanged under dependency resolution.

The documented local evaluator exits 2 (unverifiedPrerequisites): repository-local evidence is internally consistent; authenticated external enforcement is intentionally unconfigured, as described in the readiness contract. Manual gates still require current-head CI and zero unresolved threads.

{
  "schema": "llamadart.high-risk-readiness-evidence",
  "schema_version": "2.0.0",
  "test_evidence": {
    "test/integration/core/template/flint_template_test.dart": {
      "test_case": "Flint keeps typed argument and result values while sanitizing text",
      "test_snippet": "'Flint keeps typed argument and result values while sanitizing text'",
      "production_refs": [
        {
          "path": "lib/src/core/template/handlers/hermes_handler.dart",
          "snippet": "final template = Template(templateSource);"
        },
        {
          "path": "lib/src/core/template/template_render_context.dart",
          "snippet": "rendered['content'] = result is String ? result : jsonEncode(result);"
        }
      ],
      "command": "TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/integration/core/template/flint_template_test.dart",
      "head_result": "pass",
      "control_result": "fail",
      "control_kind": "before-fix",
      "control_description": "In an isolated copy of final 6a4451a source, pubspec_overrides.yaml selects hosted dinja1.3.0. The exact Flint template fails at lexer escape x before rendering. Remove override and offline pub restores1.3.1; source untouched.",
      "evidence_notes": "Independently repeated at final 6a4451a: consumer-independent-6a-1.3.0-red.log +1 fixture/-23 consumer cases; Unknown escape x at Hermes Template. consumer-independent-6a-1.3.1-restored.log +24. Final isolated Chrome +23 in consumer-independent-6a-chrome.log. Fixture integrity is separate +1 proof, not a causal case."
    },
    "test/integration/core/template/qwen35_tool_result_test.dart": {
      "test_case": "typed result history ${malformed ? 'rolls back undeclared tools' : 'preserves schema-directed scalar and container types'}",
      "test_snippet": "'typed result history ${malformed ? 'rolls back undeclared tools' : 'preserves schema-directed scalar and container types'}'",
      "production_refs": [
        {
          "path": "lib/src/core/engine/chat_completion_stream_parser.dart",
          "snippet": "ChatTemplateEngine.parse("
        },
        {
          "path": "lib/src/core/template/xml_tool_call_format.dart",
          "snippet": "final decoded = decodeToolSchemaText(raw, schema);"
        },
        {
          "path": "lib/src/core/template/tool_schema_utils.dart",
          "snippet": "return validateToolSchemaValue(raw, schema);"
        },
        {
          "path": "lib/src/core/template/handlers/qwen3_coder_xml_handler.dart",
          "snippet": "schemas: tools == null ? null : toolSchemas(tools),"
        }
      ],
      "command": "TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/integration/core/template/qwen35_tool_result_test.dart",
      "head_result": "pass",
      "control_result": "fail",
      "control_kind": "mutation",
      "control_description": "In isolated copy only, return invalid for schema-declared strings in decodeToolSchemaText. Separate negative control removes undeclared XML tool-name rejection. Both restored.",
      "evidence_notes": "Final 6a4451a consumer-independent-6a-green.log +28 (Flint23+fixture1+parser4). Repeated consumer-independent-6a-parser-mutant.log +3/-1: typed reconstruction falls back to raw envelope. consumer-independent-6a-rollback-mutant.log +3/-1: malformed unknown tool becomes executable call. Covers auto/required with thought prefixes and none/no-grammar history branch. Mutations restored."
    },
    "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart": {
      "test_case": "Hermes required calls may follow forced-open reasoning",
      "test_snippet": "test('Hermes required calls may follow forced-open reasoning', () {",
      "production_refs": [
        {
          "path": "lib/src/core/template/tool_call_grammar_utils.dart",
          "snippet": "static String afterForcedOpenThought(String grammar, String endTag) => ["
        },
        {
          "path": "lib/src/core/template/handlers/hermes_handler.dart",
          "snippet": "grammar: requiredGrammarAfterThought("
        }
      ],
      "command": "LLAMA_CPP_GBNF_VALIDATOR=/private/tmp/flint-fix-1335/grammar-validator-build/bin/test-gbnf-validator TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart --run-skipped --name 'Hermes canonical alternatives|Hermes and Mistral compile|Hermes required calls|open schema objects constrain'",
      "head_result": "pass",
      "control_result": "fail",
      "control_kind": "mutation",
      "control_description": "In isolated copy only, replace afterForcedOpenThought wrapper with bare grammar. Restored after observation.",
      "evidence_notes": "consumer-independent-6a-grammar-green.log +4 maintained compiled cases: malformed/unknown envelopes, missing/wrong-type fields, nullable nested schemas and required thought prefixes. consumer-independent-6a-grammar-mutant.log -1 forced-thought valid sequence rejected at initial N when afterForcedOpenThought returns bare grammar. Fresh validator source clean HEAD=v0.7.0 commit620d2225cc436ab0b001e3200e590e3b286ef9ab matching native wrapper v0.7.0-2; binary sha256 dbdeb6f33d062eeefcab67225329be6001aa7b7e8f49b7b805c49ae6404ce07d. Mutation restored byte-for-byte."
    }
  },
  "timestamp": "2026-10-23T11:29:03Z",
  "correlation_id": "pr1342-flint-consumer-audit-6a4451a",
  "repository": "leehack/llamadart",
  "pr_number": 1342,
  "expected_pr_head_sha": "6a4451af1990d4d8b12bc9ede9a12ce9d575c607",
  "current_base_sha": "09c189fab37f6683b771aea8b9211d86a3439a60",
  "pr_author": "leehack",
  "classification": "high-risk",
  "surfaces": [
    "artifactConsumer",
    "structuredOutput"
  ],
  "required_matrix_row_ids": [
    "high-risk-exact-head-independent-qa",
    "structured-output-adversarial"
  ],
  "matrix_row_evidence": {
    "high-risk-exact-head-independent-qa": {
      "row_id": "high-risk-exact-head-independent-qa",
      "result": "pass",
      "command": "Independent blocking-only audit of exact head 6a4451af1990d4d8b12bc9ede9a12ce9d575c607 against base09c189fab37f6683b771aea8b9211d86a3439a60",
      "evidence_notes": "Fresh independent re-audit of19 final files. New .gitattributes delta only protects two exact upstream captures. Independent old/new core.autocrlf=true checkout: old template60CRLF/capturedJSON176CRLF; final0CRLF/exact Git hashes. Old checkout integrity fails -1; fixed checkout finalVM28/Chrome23/compiled4 pass. Validation package lock remains pub-stable. Root refreshed final GraphQL with zero threads; exact-head CI still running. No earlier broad/parity counts are claimed as final6a readiness gates."
    },
    "structured-output-adversarial": {
      "row_id": "structured-output-adversarial",
      "result": "pass",
      "command": "TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/integration/core/template/flint_template_test.dart test/integration/core/template/flint_fixture_integrity_test.dart test/integration/core/template/qwen35_tool_result_test.dart\nLLAMA_CPP_GBNF_VALIDATOR=/private/tmp/flint-fix-1335/grammar-validator-build/bin/test-gbnf-validator TMPDIR=\"$(mktemp -d)\" dart test -p vm -j 1 test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart --run-skipped --name 'Hermes canonical alternatives|Hermes and Mistral compile|Hermes required calls|open schema objects constrain'",
      "evidence_notes": "Final6a isolatedVM28, Chrome23, compiled4. Hosted1.3.0 control +1/-23 then restored1.3.1 +24. Schema decoding +3/-1, unknown-tool guard deletion +3/-1, grammar bypass -1; all restored. Separate Windows checkout integrity control -1→pass. Actual effect rendering/history; unchanged parsing/grammar conservatively covered. No Flint model/tool-output qualification."
    }
  },
  "independent_audit": {
    "auditor_identity": "codex-auditor-flint-consumer",
    "audit_kind": "codex-adversarial",
    "audit_head_sha": "6a4451af1990d4d8b12bc9ede9a12ce9d575c607",
    "audit_base_sha": "09c189fab37f6683b771aea8b9211d86a3439a60",
    "decision": "accepted",
    "unresolved_review_threads": 0,
    "known_pr_caused_p1_regressions": 0,
    "summary": "Independent blocking re-audit accepted for6a4451a vs09c189f. All19 changed files inspected. Increment fromdcc is four .gitattributes lines: comment/blank plus two narrow -text exact-capture rules. Independent core.autocrlf=true old/new checkout reproduces and repairs byte/hash failure; integrity negative control fails and fixed bytes pass. All final-head behavior/parser/compiled tests and causal controls rerun. Zero known PR-caused P1. Root GraphQL refreshed exacthead/base and zero threads; current CI remains pending. Prior head audits are superseded; no Flint architecture/generation/output tools qualified."
  },
  "structured_output_evidence": {
    "coverage": {
      "input_rendering_history": [
        "test/integration/core/template/flint_template_test.dart",
        "test/integration/core/template/qwen35_tool_result_test.dart"
      ],
      "compiled_grammar_acceptance": [
        "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart"
      ],
      "compiled_grammar_rejection": [
        "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart"
      ],
      "schema_reconstruction": [
        "test/integration/core/template/qwen35_tool_result_test.dart"
      ],
      "streaming_rollback": [
        "test/integration/core/template/qwen35_tool_result_test.dart"
      ],
      "tool_choice_thinking": [
        "test/integration/core/template/qwen35_tool_result_test.dart",
        "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart"
      ],
      "upstream_parity": [
        "test/integration/core/template/flint_template_test.dart"
      ]
    },
    "impacts": {
      "inputRenderingHistory": {
        "applicable": true,
        "rationale": "Hosted dinja 1.3.1 is consumed by every Template constructor. Flint rendering reaches Hermes, shared context and public create. Tests compare upstream bytes and preserve typed history.",
        "production_refs": [
          {
            "path": "lib/src/core/template/handlers/hermes_handler.dart",
            "snippet": "final template = Template(templateSource);"
          },
          {
            "path": "lib/src/core/template/template_render_context.dart",
            "snippet": "rendered['content'] = result is String ? result : jsonEncode(result);"
          }
        ]
      },
      "outputParsingStreaming": {
        "applicable": true,
        "rationale": "No parser or grammar implementation bytes change in this PR. Evidence-only structured paths impose this conservative minimum impact; unchanged production boundaries were inspected and validated with maintained cases and isolated causal mutations.",
        "production_refs": [
          {
            "path": "lib/src/core/engine/chat_completion_stream_parser.dart",
            "snippet": "ChatTemplateEngine.parse("
          },
          {
            "path": "lib/src/core/template/xml_tool_call_format.dart",
            "snippet": "final decoded = decodeToolSchemaText(raw, schema);"
          }
        ]
      },
      "grammarSchema": {
        "applicable": true,
        "rationale": "No parser or grammar implementation bytes change in this PR. Evidence-only structured paths impose this conservative minimum impact; unchanged production boundaries were inspected and validated with maintained cases and isolated causal mutations.",
        "production_refs": [
          {
            "path": "lib/src/core/template/tool_call_grammar_utils.dart",
            "snippet": "static String afterForcedOpenThought(String grammar, String endTag) => ["
          },
          {
            "path": "lib/src/core/template/handlers/hermes_handler.dart",
            "snippet": "grammar: requiredGrammarAfterThought("
          }
        ]
      }
    },
    "families": [
      {
        "family": "Flint-template-artifact",
        "status": "tested",
        "evidence_test_paths": [
          "test/integration/core/template/flint_template_test.dart"
        ],
        "rationale": "Exact upstream template/captured apply-template emissions at model revision 468307a1b38259e9e4c9d4fe53d4e3b76386572c; seven prompt cases, typed history and public mocked create. Native architecture/generation/output tools are deliberately unqualified (native issue112)."
      }
    ]
  },
  "affected_test_paths": [
    "test/integration/core/template/flint_template_test.dart",
    "test/integration/core/template/qwen35_tool_result_test.dart",
    "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart"
  ],
  "evaluation": {
    "evaluated_at": "2026-10-23T11:31:10.094644Z",
    "changed_files": [
      {
        "path": ".gitattributes",
        "status": "modified"
      },
      {
        "path": "CHANGELOG.md",
        "status": "modified"
      },
      {
        "path": "README.md",
        "status": "modified"
      },
      {
        "path": "example/basic_app/README.md",
        "status": "modified"
      },
      {
        "path": "example/chat_app/pubspec.lock",
        "status": "modified"
      },
      {
        "path": "packages/llamadart_validation/pubspec.lock",
        "status": "modified"
      },
      {
        "path": "pubspec.yaml",
        "status": "modified"
      },
      {
        "path": "skills/llamadart-chat-streaming/SKILL.md",
        "status": "modified"
      },
      {
        "path": "test/fixtures/flint/Flint.jinja",
        "status": "added"
      },
      {
        "path": "test/fixtures/flint/LICENSE",
        "status": "added"
      },
      {
        "path": "test/fixtures/flint/README.md",
        "status": "added"
      },
      {
        "path": "test/fixtures/flint/render_upstream.json",
        "status": "added"
      },
      {
        "path": "test/integration/core/template/flint_fixture_integrity_test.dart",
        "status": "added"
      },
      {
        "path": "test/integration/core/template/flint_template_test.dart",
        "status": "added"
      },
      {
        "path": "test/support/flint_template_fixture.dart",
        "status": "added"
      },
      {
        "path": "website/docs/changelog/recent-releases.md",
        "status": "modified"
      },
      {
        "path": "website/docs/guides/chat-template-and-parsing.md",
        "status": "modified"
      },
      {
        "path": "website/docs/platforms/support-matrix.md",
        "status": "modified"
      },
      {
        "path": "website/examples/observability/pubspec.lock",
        "status": "modified"
      }
    ],
    "decision": "unverifiedPrerequisites",
    "failure_classification": "externalPrerequisitesUnavailable",
    "message": "Repository-local evidence is internally consistent, but auditor authentication, GitHub App publication, protected-environment provenance, and ruleset enforcement are not available. This is not operational merge readiness.",
    "external_prerequisites": {
      "app_installed": false,
      "protected_environment_configured": false,
      "independent_auditor_authenticated": false,
      "ruleset_enforced": false,
      "diagnostic_message": "No repository-local input can authenticate the dedicated GitHub App, protected environment, independent auditor, or conditional ruleset. See doc/high_risk_pre_merge_readiness.md."
    }
  }
}

@leehack
leehack marked this pull request as ready for review October 23, 2026 11:45
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1342.

@leehack
leehack merged commit b06ac94 into main Oct 23, 2026
41 checks passed
@leehack
leehack deleted the fix/dinja-1.3.1-template-escapes branch October 23, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant