Skip to content

perf(database): eliminate repeated substring allocations in stripLeadingWhitespaceAndLineComments via micro-scanner #1515

Open

Description

@ZhuchkaTriplesix

Summary

In lib/core/database/sql_limit.dart (lines 24–40):

String stripLeadingWhitespaceAndLineComments(String sql) {
  var s = sql.trimLeft();
  while (true) {
    if (s.isEmpty) return s;
    if (s.startsWith('--')) {
      final nl = s.indexOf('\n');
      if (nl == -1) return '';
      s = s.substring(nl + 1).trimLeft(); // Full copy of the rest of the script!
      continue;
    }
    if (s.startsWith('/*')) {
      s = s.substring(_blockCommentEnd(s, 0)).trimLeft(); // Another full copy!
      continue;
    }
    return s;
  }
}

If a user executes a 10 MB SQL migration script or database dump that starts with 50 header comment lines (copyright, generator metadata, table comments), the while (true) loop copies the entire remaining 10 MB string on every single line, allocating up to 500 MB of heap memory ((N \cdot M)$ complexity) before running the query.

Architectural Solution: Zero-Allocation Micro-Scanner

Replace the loop with an index-based micro-scanner function:

  1. static int _skipLeadingCommentsOffset(String sql)
    • Advances an integer pointer int i through leading whitespace, -- comments, and /* */ block comments using sql.codeUnitAt(i).
    • Returns the index of the first executable SQL statement character.
  2. The outer method performs exactly ONE sql.substring(start) on the final slice:
    String stripLeadingWhitespaceAndLineComments(String sql) {
      final start = _skipLeadingCommentsOffset(sql);
      return start == 0 ? sql : sql.substring(start);
    }

Acceptance Criteria

  • Header comment stripping operates with strict (N)$ memory and CPU bounds.
  • Exactly one substring call is made regardless of how many comment lines precede the script.
  • Full parity with existing SQL limit tests in test/core/database/sql_limit_test.dart.

Activity

  1. added
    editorCode/SQL editor
    performanceTheme parser epic label: performance
    coreCore library logic and services
    P2Medium priority / Parity & Refactoring
    on Oct 12, 2026
  2. added this to the 0.5.0 milestone on Oct 12, 2026
  3. added 2 commits that reference this issue on Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priority / Parity & RefactoringcoreCore library logic and serviceseditorCode/SQL editorperformanceTheme parser epic label: performance

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions