Skip to content

Decode HTML character references instead of double-escaping them - #3

Merged
jacksonpires merged 1 commit into
mainfrom
fix/html-entities
Nov 13, 2026
Merged

jacksonpires merged 1 commit into
mainfrom
fix/html-entities

Conversation

@jacksonpires

Copy link
Copy Markdown
Owner

Problem

HTML character references in template text and literal attribute values were copied verbatim into the generated Ruby strings. Phlex escapes everything it outputs, so they were escaped twice:

Template Generated (before) Rendered (before)
<p>&copy; 2026</p> p { "&copy; 2026" } <p>&amp;copy; 2026</p> → shows "© 2026"
a&nbsp;b "a&nbsp;b" shows "a b"
href="/x?a=1&amp;b=2" href: "/x?a=1&amp;b=2" &amp;amp; in the URL

This affected ERB from the start, and HAML text since #2.

Fix

  • HtmlEntities.decode turns references into the characters they stand for. Named references come from the HTML 4 set plus &apos;, generated from Python's html.entities.name2codepoint; decimal and hex references are supported too.
  • The ERB Parser applies it to Text nodes and to the literal parts of attribute values. Phlex then re-escapes only what needs it: &lt;b&gt; → "<b>" → &lt;b&gt;. HAML text goes through the same parser, so it is fixed as well.
  • These stay verbatim:
    • <script>/<style> content (RawText);
    • HAML's :escaped filter (literal text by definition);
    • unknown names, references without a closing ;, and invalid code points.
  • Transformer#escape_inner writes invisible characters, such as a decoded &nbsp;, as   escapes, so the generated source stays readable.

After the fix, rendered with Phlex 2.4: <p title="Tom & Jerry">© 2026 — a b &lt;tag&gt;</p><a href="/x?a=1&b=2">Go</a>.

Testing

  • New test/html_entities_test.rb (12 tests): decoding rules, ERB text and attributes, interpolated attributes, static_classes after decoding, script verbatim, and HAML text vs :escaped. Suite: 169 → 181, all green.
  • The entity and HAML text tests also pass under haml 6.4 and 5.2.
  • README (conversion table and limitations), CHANGELOG and CLAUDE.md are updated.

🤖 Generated with Claude Code

Text and literal attribute values were emitted with their HTML character
references intact, and Phlex escapes everything it outputs, so `&copy;`
rendered as the visible text "&copy;" (`&amp;copy;`), `&nbsp;` as
"&nbsp;", and `href="?a=1&amp;b=2"` as `&amp;amp;`.

The ERB Parser now decodes references (HtmlEntities: HTML 4 named set
plus `&apos;`, decimal and hex) in Text nodes and in the literal parts of
attribute values, letting Phlex re-escape only what needs it. HAML text
goes through the same Parser and is fixed too. `<script>`/`<style>`
content and HAML's `:escaped` filter stay verbatim; unknown names,
unterminated references and invalid code points are left as written.
Invisible characters such as a decoded `&nbsp;` are written as ` `
escapes in the generated source.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@jacksonpires
jacksonpires merged commit c66a44e into main Nov 13, 2026
23 checks passed
@jacksonpires jacksonpires mentioned this pull request Nov 13, 2026
@jacksonpires
jacksonpires deleted the fix/html-entities branch November 13, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant