Skip to content

fix: resolve double-encoded HTML entities in title and description - #456

Merged
Guts merged 2 commits into
Guts:mainfrom
Zerthick:fix/double-encoded-html-entities
Oct 27, 2026
Merged

Guts merged 2 commits into
Guts:mainfrom
Zerthick:fix/double-encoded-html-entities

Conversation

@Zerthick

Copy link
Copy Markdown
Contributor

Description

Resolve double-encoded HTML entities (e.g. &, <, >) in RSS feed title and description fields.

MkDocs pre-escapes content, and then Jinja's |e filter escapes again, producing double-encoded entities. Using html.unescape() in Python before template rendering ensures |e performs a single correct escape.

Changes

  • Add html_module.unescape() for title and description in plugin.py
  • Add test fixture with special characters (& < >)
  • Add test verifying no double-encoded entities in output

Split from: #451


💡 AI Contribution Level: 8 — Based on the VisiData AI contribution scale, this was an AI-assisted contribution where the AI performed the code splitting and PR creation with minimal human guidance.

@github-actions github-actions Bot added bug Something isn't working quality Tests, project resiliency, etc. labels Aug 10, 2026
@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.22%. Comparing base (eeeb232) to head (fa8485b).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #456      +/-   ##
==========================================
+ Coverage   83.20%   83.22%   +0.02%     
==========================================
  Files          11       11              
  Lines         786      787       +1     
  Branches      133      133              
==========================================
+ Hits          654      655       +1     
  Misses         86       86              
  Partials       46       46              
Flag Coverage Δ
unittests 83.22% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
mkdocs_rss_plugin/plugin.py 89.50% <100.00%> (+0.06%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Guts Guts left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this one.

The description part, however, introduces a regression, and the test doesn't exercise the bug:

  1. description is HTML, not text : get_description_or_abstract() returns page.content or markdown.markdown(...) output in almost every branch, and the JSON Feed publishes it as content_html. In RSS, <description> carries that HTML escaped once for XML, so &amp;lt; in the raw XML is the correct encoding of a &lt; in the HTML. Unescaping it first turns code samples into real markup. With Use `<script>` and `a < b` in code.:

    • main: readers get <p>Use <code>&lt;script&gt;</code> and <code>a &lt; b</code> in code.</p>
    • this PR: readers get <p>Use <code><script></code> and <code>a < b</code> in code.</p>

    For a docs plugin, where inline code is everywhere, that's a blocker. Please keep the unescape on title only (see inline suggestions).

  2. fixture doesn't reproduce the bug because its title is set in the front-matter, which MkDocs returns verbatim; only H1-derived titles carry entities. Same for the description meta, which |e already escapes once. The new test passes on main without the fix. I've suggested a fixture with an H1 title instead.

Please also rebase on main.

Comment thread tests/test_double_encoded_entities.py Resolved
Comment thread mkdocs_rss_plugin/plugin.py Resolved
Comment thread mkdocs_rss_plugin/plugin.py Resolved
@Zerthick
Zerthick force-pushed the fix/double-encoded-html-entities branch from ada0a8d to 5214af7 Compare October 25, 2026 20:35
@Zerthick

Copy link
Copy Markdown
Contributor Author

Done:

  • unescape kept on title only, with a comment noting why
  • fixture now derives its title from the H1 (# Test: A & B < C > D), which is the only case where MkDocs returns entities
  • dropped the test module; test_xml_escaping_in_author now asserts Test: A & B < C > D in the feed XML
  • rebased on main

@Guts

Guts commented Oct 26, 2026

Copy link
Copy Markdown
Owner

@Zerthick thanks. Can you rebase again on tour side please?

MkDocs pre-escapes content, then Jinja's |e filter escapes again,
producing &amp;amp;, &amp;lt;, &amp;gt;. Use html.unescape() in Python
before template rendering so |e performs a single correct escape.

- Add html_module.unescape() for title and description in plugin.py
- Add test fixture with special characters
- Add test verifying no double-encoded entities
- keep unescape on title only: description is HTML and must stay escaped
  once, unescaping it turned code samples into real markup
- fixture: derive the title from the H1 so it actually carries entities
  (front-matter titles are returned verbatim by MkDocs)
- drop the duplicate test module; assert the title encoding in
  test_xml_escaping_in_author
@Guts
Guts force-pushed the fix/double-encoded-html-entities branch from 5214af7 to fa8485b Compare October 27, 2026 06:40
@sonarqubecloud

Copy link
Copy Markdown

@Guts Guts left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Guts
Guts merged commit 17a140b into Guts:main Oct 27, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working quality Tests, project resiliency, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants