Skip to content

Support matching URLs with Unicode and Japanese characters - #49

Merged
Alt-Charlie merged 2 commits into
alt-design:mainfrom
andreasbergqvist:andreasbergqvist-fix-unicode-redirect-matching
Oct 30, 2026
Merged

Alt-Charlie merged 2 commits into
alt-design:mainfrom
andreasbergqvist:andreasbergqvist-fix-unicode-redirect-matching

Conversation

@andreasbergqvist

Copy link
Copy Markdown
Contributor

This pull request improves the application's support for Unicode paths in redirects, ensuring that both URL-encoded and raw Unicode paths are correctly matched and redirected. It updates the redirect logic to handle Unicode-aware regular expressions and adds comprehensive tests for these scenarios.

Unicode path handling:

  • Updated the redirect middleware (CheckForRedirects.php) to check for redirects using both URL-encoded and decoded (raw Unicode) paths, ensuring that Unicode paths are matched regardless of encoding.
  • Enhanced regex redirect matching to use Unicode-aware patterns and to check both encoded and decoded URIs, improving support for Unicode in regex-based redirects.

Testing improvements:

  • Added feature tests for simple and regex Unicode path redirects in both the database and file redirect drivers, verifying correct behavior for both encoded and raw Unicode paths. (DatabaseRedirectTest.php, FileRedirectTest.php) [1] [2]
  • Added a Unicode regex pattern to the regex detection test to ensure proper detection and handling of Unicode patterns. (RegexDetectionTest.php)

@Alt-Charlie Alt-Charlie left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @andreasbergqvist thanks for this and sorry it's taken a while to get to. Unicode support is a welcome addition and the tests are useful. A couple of things possibly worth some attention before we mergeL

1. Only decode the Unicode bytes for regex matching

The regex replacement now runs on the fully decoded URI, so encoded reserved characters change meaning in the redirect target. With /old/(.*) → /new/$1, a request to /old/a%3Fb used to redirect to /new/a%3Fb but now goes to /new/a?b, so part of the path becomes a query string. %23 becomes a fragment in the same way.

Could you decode only the multibyte UTF-8 sequences (the %80 to %FF escapes) and leave the ASCII escapes such as %3F, %23 and %2F encoded? A preg_replace_callback over those sequences does it and your Unicode patterns still match. A test asserting /old/a%3Fb still redirects to /new/a%3Fb would be good to have alongside the Unicode ones.

2. Remove the duplicate lookups

This one isn't entirely down to your change. The existing code already repeats lookups when there's no query string, because $path and $pathOnly end up identical. Adding the decoded variants doubles that, though: for most requests the path has no percent escapes, so the four decoded candidates match the four encoded ones. On the file driver each miss reads and parses every redirect file, so normal page requests now do that up to eight times instead of four.

Could you build the candidate list once, remove duplicates, and loop over it? That fixes the existing duplication as well and the "no need to redirect" check can use the same list. If the encoded forms come first, an exact match on the stored value also wins over a decoded one, as it did before.

Hope that makes sense. Any questions or alternative thoughts please let me know!

@Alt-Charlie Alt-Charlie self-assigned this Oct 8, 2026
Preserve encoded reserved characters during regex replacement and deduplicate redirect lookup candidates.

Co-authored-by: Copilot App <[email protected]>
@Alt-Charlie
Alt-Charlie merged commit f2d9969 into alt-design:main Oct 30, 2026
1 check passed
@Alt-Charlie

Copy link
Copy Markdown
Contributor

Hey @andreasbergqvist

Thanks for addressing those points. Both points are sorted: reserved characters stay encoded in regex targets, and the lookups are deduplicated with the encoded forms first. I’ve run the suite and checked the edge cases from my earlier comment, and everything behaves as before apart from the Unicode redirects, which now work.

Merged and released: https://github.com/alt-design/Alt-Redirect-Addon/releases/tag/v1.9.0

@Alt-Charlie Alt-Charlie linked an issue Oct 30, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Japanese characters in url not matched

2 participants