Skip to content

fix: use a raw string for EMAIL_CATCH_ALL_PATTERN - #292

Merged
bokysan merged 1 commit into
bokysan:masterfrom
pando85:fix/anonymizer-invalid-escape-sequences
Jul 29, 2026
Merged

bokysan merged 1 commit into
bokysan:masterfrom
pando85:fix/anonymizer-invalid-escape-sequences

Conversation

@pando85

@pando85 pando85 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Problem

image_root/scripts/email-anonymizer.py:37 builds the catch-all pattern from a normal string
literal containing \[, \. and \{. None of those are valid Python escape sequences — they
survive only because Python leaves unknown escapes as-is.

Python 3.12 promoted that from a DeprecationWarning to a visible SyntaxWarning, and since
v5.0.0 moved the image to Debian Trixie (Python 3.13) it now shows up at runtime:

SyntaxWarning: "\[" is an invalid escape sequence. Such sequences will not work in the future.
Did you mean "\\["? A raw string is also an option.

It is more than cosmetic noise. With ANONYMIZE_EMAILS set, rsyslog runs this script via
mmexternal and reads its replies over a pipe, so the warning lands on that channel and rsyslog
rejects it as a bad reply:

rsyslogd: mmexternal: invalid reply '/scripts/email-anonymizer.py:37: SyntaxWarning: invalid
escape sequence '\['   EMAIL_CATCH_ALL_PATTERN = ...' from program '/scripts/email-anonymizer.sh'

This happens once per anonymizer process start — and again whenever the wrapper's
while ! ... ; do sleep 1; done retry loop restarts it. The log message being processed at that
moment doesn't get anonymized.

Fix

Make the literal a raw string.

One thing worth flagging for review: just adding the r prefix would silently change the
pattern.
The literal already contains \\[ and \\] in the first character class, which in a
raw string would mean a literal backslash followed by [ — altering what the class matches. So
the pre-escaped sequences are collapsed in the same change (\\[ → \[, \\] → \],
\\{ → \{), which leaves the compiled pattern byte-for-byte identical to before.

Verification

  • The resulting pattern string is unchanged (compared repr() of the literal before and after).
  • The SyntaxWarning is gone; no other warnings appear.
  • The input vectors from unit-tests/024_email-anonymizer-smart.bats and
    unit-tests/025_email-anonymizer-regexs.bats produce identical output before and after,
    across the noop, hash, paranoid and smart filters — 190 input × filter comparisons,
    zero differences. 025's own expectations still pass unchanged.

On Python 3.12+ the pattern emits a SyntaxWarning at import time:

    SyntaxWarning: "\[" is an invalid escape sequence.
    Such sequences will not work in the future.

`\[`, `\.` and `\{` are not valid Python escapes, so they only survive by
accident. Python 3.12 turned the old DeprecationWarning into a visible
SyntaxWarning, so since v5.0.0 moved the image to Debian Trixie
(Python 3.13) it shows up at runtime.

It is not only noise. With ANONYMIZE_EMAILS set, rsyslog runs this script
through mmexternal and reads its replies over a pipe, so the warning ends
up on that channel and rsyslog logs it as a bad reply:

    rsyslogd: mmexternal: invalid reply '.../email-anonymizer.py:37:
    SyntaxWarning: invalid escape sequence '\['   ...' from program
    '/scripts/email-anonymizer.sh'

That happens once per anonymizer process start (and again each time the
wrapper's retry loop restarts it), and the message being processed at
that moment is not anonymized.

Note that simply prefixing the existing literal with `r` would change the
pattern: the parts already written as `\\[` would become a literal
backslash followed by `[` inside the character class. The pre-escaped
sequences are therefore collapsed (`\\[` -> `\[`, `\\]` -> `\]`,
`\\{` -> `\{`) at the same time, which keeps the resulting pattern
byte-for-byte identical to before.

Verified: the compiled pattern string is unchanged, the warning is gone,
and the vectors from unit-tests/024 and 025 produce identical output
before and after across the noop, hash, paranoid and smart filters
(190 input x filter comparisons, no differences).
@bokysan

bokysan commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Nice catch! Thank you for the fix!

@bokysan
bokysan merged commit cb0e48a into bokysan:master Jul 29, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants