Skip to content

perf(parser): replace input.substring with matchAsPrefix in GridFilterEngine lexer #1512

Open

Description

@ZhuchkaTriplesix

Summary

In lib/features/workspace/grid_filter_engine.dart (lines 389, 400, 408, 458):

Inside _FilterLexer.tokenize() and _tryMatchPredicate(), logical operators and predicates are checked against a fresh substring copy on every single token iteration:

final rem = input.substring(i);
final andMatch = RegExp(r'^(AND\b|&&)', caseSensitive: false).firstMatch(rem);
...
final rem = input.substring(p);
final isNullMatch = RegExp(r'^IS\s+(NOT\s+)?NULL\b', caseSensitive: false).firstMatch(rem);

Whenever the user types into the data grid filter bar, thousands of transient string slices are allocated, triggering garbage collection and UI stutter.

Architectural Solution: Dedicated Zero-Allocation Micro-Parsers

Decompose _tryMatchPredicate and token matching into dedicated, testable micro-functions operating strictly on (String input, int offset) with zero substring copying:

  1. static int? _matchLogicalOperator(String input, int offset)
    • Matches AND, OR, NOT, &&, ||, ! via precompiled static patterns using matchAsPrefix(input, offset).
  2. static ({int consumed, _PredicateNode node})? _matchNullPredicate(String input, int start, int colIdx)
    • Micro-parser for IS NULL and IS NOT NULL.
  3. static ({int consumed, _PredicateNode node})? _matchInPredicate(String input, int start, int colIdx)
    • Micro-parser for IN (...) and NOT IN (...).
  4. static ({int consumed, _PredicateNode node})? _matchBetweenPredicate(String input, int start, int colIdx)
    • Micro-parser for BETWEEN min AND max.
  5. static ({int consumed, _PredicateNode node})? _matchComparisonPredicate(String input, int start, int colIdx)
    • Micro-parser for relational operators (=, !=, <, <=, >, >=, :, LIKE, ILIKE).
  6. static ({String value, int consumed})? _scanQuotedStringLiteral(String input, int start)
    • Single-pass quote escaper for single ('...) and double ("...) quotes.

Acceptance Criteria

  • Each predicate type and operator has an isolated, self-contained micro-function.
  • No intermediate input.substring(i) calls in the lexer loop.
  • Micro-functions return Dart 3 records with primitive offsets and nodes.
  • 100% test pass rate in test/features/workspace/data_grid_filter_engine_test.dart.

Activity

  1. added
    parserTheme parser epic label: parser
    performanceTheme parser epic label: performance
    data-gridInteractive data grid, cell editor, filtering, groupings
    P3Low priority / Polish & Enhancements
    on Oct 12, 2026
  2. added this to the 0.5.0 milestone on Oct 12, 2026
  3. added a commit that references this issue on Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low priority / Polish & Enhancementsdata-gridInteractive data grid, cell editor, filtering, groupingsparserTheme parser epic label: parserperformanceTheme parser epic label: performance

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions