Skip to content

Add grammar-based differential fuzzing for CometRegex #5811

Open

Description

@sam-1112

What is the problem the feature request solves?

PR #5415 introduces CometRegex, a conservative plan-time analyzer that allows a restricted subset of literal Java regex patterns to use native Rust rlike execution by default.

The current parity suite uses a fixed corpus. This covers patterns we anticipated, but it may not detect an unexpected combination of otherwise admitted constructs. The safety property is that every pattern admitted by CometRegex must have the same matching behavior in Java's java.util.regex engine and Comet's native Rust regex kernel.

Describe the potential solution

Add a deterministic grammar-based generator that produces both admitted and rejected regex patterns, including combinations of:

  • printable ASCII literals;
  • character classes, ranges, negation, and escaped metacharacters;
  • concatenation and alternation;
  • capturing and non-capturing groups;
  • greedy and counted quantifiers;
  • constructs that CometRegex is expected to reject.

For every generated pattern admitted by CometRegex, compare Java Pattern.matcher(subject).find() with the actual native Rust rlike result.

Generated subjects should include ASCII and non-ASCII strings, empty strings, newlines, control characters, exotic whitespace, supplementary Unicode code points, and NULL where applicable.

The generator should use a reproducible seed and report the seed, pattern, subject, and both results on failure. If the test is too expensive for every pull request, it could run in a nightly workflow.

Additional context

Follow-up from #5415 and #5351.

This tracks the grammar-based fuzzing requested during review of #5415: #5415 (review)

Andy also performed an offline fuzzing campaign during the review, but that process is not currently part of the repository or CI: #5415 (comment)

Activity

  1. sam-1112 commented on Nov 11, 2026

    @sam-1112
    ContributorAuthor

    take

  2. sam-1112 commented on Nov 11, 2026

    @sam-1112
    ContributorAuthor

    I’ll implement this in two PRs:

    1. MVP differential fuzzer: a small grammar generator, admit/reject assertions, fixed seeds and subjects, Java vs. native rlike comparison, and reproducible failure reports.
    2. Coverage expansion and CI: broader grammar and subject coverage, depth/size boundary cases, statistics, and nightly CI integration.

    The first PR will deliver a working end-to-end fuzzer and verify that comparisons actually exercise the native path. The second will build on it to expand coverage and support regular runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions