Skip to content

perf(parser): eliminate dynamic RegExp compilation and substring copying in DestructiveSqlDetector #1511

Open

Description

@ZhuchkaTriplesix

Summary

In lib/core/database/destructive_sql_detector.dart (lines 244–260), stripCommentsAndStrings(String sql) detects PostgreSQL dollar-quotes ($$ or $tag$):

// 3. Dollar quotes in PostgreSQL: $$ or $tag$
if (sql[i] == '$') {
  final match =
      RegExp(r'^$([a-zA-Z0-9_]*)$').matchAsPrefix(sql.substring(i));
  if (match != null) {
    final tag = match.group(0)!;
    i += tag.length;
    final closeIdx = sql.indexOf(tag, i);
...
  1. Dynamic RegExp compilation: On every $ encountered (including parameters like $1, $2), a new RegExp instance is compiled at runtime.
  2. Substring copying: sql.substring(i) copies the entire remainder of the script into a new String buffer. For a multi-megabyte SQL file, this creates severe heap churn and GC pauses.
  3. Char string allocations: The scanner indexes characters via sql[i] rather than unboxed sql.codeUnitAt(i), creating single-character string allocations on every step.

Architectural Solution: Zero-Allocation Micro-Functions

Decompose the monolithic 120-line stripCommentsAndStrings loop into dedicated, inlineable zero-allocation micro-scanner functions. Each function accepts (String sql, int start, int len) and returns the int nextOffset without heap allocations or try-catch:

  1. static int _skipLineComment(String sql, int start, int len)
    • Scans until \n or \r using codeUnitAt(i).
  2. static int _skipBlockComment(String sql, int start, int len)
    • Scans until */ using codeUnitAt(i).
  3. static int _skipDollarQuote(String sql, int start, int len)
    • Uses precompiled static _dollarTag with matchAsPrefix(sql, start).
    • Locates closing tag via sql.indexOf(tag, start + tag.length) without substrings.
  4. static int _skipStringLiteral(String sql, int start, int len, int quoteCode)
    • Fast scanner for single/double quotes with quote escaping ('', \').

Main loop becomes clean and declarative:

while (i < len) {
  final code = sql.codeUnitAt(i);
  if (code == _chDash && i + 1 < len && sql.codeUnitAt(i + 1) == _chDash) {
    i = _skipLineComment(sql, i + 2, len);
    buffer.write(' ');
    continue;
  }
  if (code == _chSlash && i + 1 < len && sql.codeUnitAt(i + 1) == _chAsterisk) {
    i = _skipBlockComment(sql, i + 2, len);
    buffer.write(' ');
    continue;
  }
  if (code == _chDollar) {
    final next = _skipDollarQuote(sql, i, len);
    if (next > i) {
      i = next;
      buffer.write("''");
      continue;
    }
  }
  if (code == _chSingleQuote) {
    i = _skipStringLiteral(sql, i + 1, len, _chSingleQuote);
    buffer.write("''");
    continue;
  }
  buffer.writeCharCode(code);
  i++;
}

Acceptance Criteria

  • Each token/comment/quote kind is encapsulated in a dedicated micro-function.
  • Micro-functions operate strictly with codeUnitAt and primitive offsets, producing zero heap allocations.
  • No dynamic RegExp instances are compiled during scanning.
  • All unit tests in test/core/database/destructive_sql_detector_test.dart pass.

Activity

  1. added
    parserTheme parser epic label: parser
    performanceTheme parser epic label: performance
    coreCore library logic and services
    P2Medium priority / Parity & Refactoring
    on Oct 12, 2026
  2. added this to the 0.5.0 milestone on Oct 12, 2026
  3. added a commit that references this issue on Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priority / Parity & RefactoringcoreCore library logic and servicesparserTheme parser epic label: parserperformanceTheme parser epic label: performance

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions