Repository navigation
syntax: don't lift alternation prefixes that can match more than one way - #1401
Merged
Merged
Conversation
Lifting a common prefix out of an alternation, `XA|XB` to `X(?:A|B)`, changes which match leftmost-first semantics prefers when `X` can match more than one way. For example, `ab*b|ab*c` on `abbc` reported `abbc` instead of `abb`: after `b*` takes `bb`, the lifted form tries both branches before it backtracks into `b*`. Literals, classes and look-around assertions match at most one way, so the lifted prefix now stops at the first element that is anything else. Fixes rust-lang#1400
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1400
lift_common_prefixrewritesXA|XBasX(?:A|B), which only preserves leftmost-first semantics whenXmatches one way. OtherwiseX(?:A|B)prefersX's first choice followed byBoverX's second choice followed byA, which is whatXA|XBpicks.ab*b|ab*conabbcreported0..4instead of0..3, anda*?b|a*?aonabreported0..1instead of0..2.The lifted prefix now stops at the first element that isn't a literal, a class or a look-around assertion, since each of those matches at most one way. Prefixes like
[A-Z]foo|[A-Z]quuxfrom the existing tests, and\wfoo|\wbarfrom fbe2e16, still lift. A shape like\w+foo|\w+barno longer gets a top-level concatenation from this rewrite, so it can lose the reverse inner optimization that fbe2e16 mentions. I haven't run rebar. The check is also conservative: a fixed repetition such as[a-z]{3}matches one way but isn't lifted.The new assertions in
smart_alternationand the twotestdata/regression.tomlcases fail without the change.