Skip to content

perf(editor): eliminate quadratic line counting and substring allocations in SqlStatementSplitter #1350

Closed

Description

@ZhuchkaTriplesix

Summary

In lib/core/database/sql_statement_splitter.dart:

  1. Quadratic Line Counting ((N \cdot M)$):
    In close(int end) (lines 40–42):

    out.add(SqlStatementSpan(start: s, end: e, line: _lineOf(sql, s)));

    _lineOf(sql, s) (lines 147–153) iterates from 0 to offset counting newline characters (0x0A) on every statement. For a script with $ statements across $ lines, this performs (N \cdot M) \approx O(N^2)$ characters scans. In large SQL dumps or multi-statement migration scripts, this quadratic scanning takes seconds.

  2. Per-Dollar Substring Allocations & Dynamic RegExp:
    In lines 67–76:

    if (c == '\$') {
      final match = RegExp(r'^\\$').matchAsPrefix(sql.substring(i));

    On every $ character encountered (including numbered parameters like $1, $2), sql.substring(i) copies the entire remainder of the script into a new string, and compiles a fresh RegExp instance. For a 5 MB file with dollar quotes or parameters, this creates massive transient allocations and GC pressure.

  3. UI Thread Blocking:
    SqlStatementSplitter.spans is invoked synchronously on the UI thread in GenericSqlWorkspace._runSpans whenever the user executes a statement or script (Ctrl+Enter / F5).

Proposed Solution

  1. Maintain a running currentLine counter during the main character iteration in SqlStatementSplitter.spans, turning line tracking into a strict (N)$ single pass.
  2. Use a static compiled RegExp and pass the search offset start: i to matchAsPrefix(sql, i) instead of allocating sql.substring(i).
  3. Offload statement splitting to an isolate via compute if script length exceeds a threshold (e.g. 64 KB).

Acceptance Criteria

  • Statement splitting runs in linear (N)$ time with no quadratic line counting.
  • No string allocations or dynamic regex creations occur on $ tokens.
  • SqlStatementSpan.line calculations remain 100% accurate across all test scripts.

Activity

  1. added
    editorCode/SQL editor
    performanceTheme parser epic label: performance
    coreCore library logic and services
    backendBackend database driver execution and queries
    P2Medium priority / Parity & Refactoring
    on Oct 10, 2026
  2. added a commit that references this issue on Oct 11, 2026
  3. ZhuchkaTriplesix commented on Oct 11, 2026

    @ZhuchkaTriplesix
    MemberAuthor

    Fixed in #1453 (and #1399) by using single-pass linear line counting, static dollar tag RegExp, zero-string-allocation scanner with codeUnitAt, and background isolate offloading via compute for large scripts (>64KB) in SqlStatementSplitter.spansAsync.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priority / Parity & RefactoringbackendBackend database driver execution and queriescoreCore library logic and serviceseditorCode/SQL editorperformanceTheme parser epic label: performance

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions