Repository navigation
perf(parser): eliminate dynamic RegExp compilation and substring copying in DestructiveSqlDetector #1511
Copy link
Copy link
Open
Labels
P2Medium priority / Parity & RefactoringMedium priority / Parity & RefactoringcoreCore library logic and servicesCore library logic and servicesparserTheme parser epic label: parserTheme parser epic label: parserperformanceTheme parser epic label: performanceTheme parser epic label: performance
Milestone
Description
Summary
In lib/core/database/destructive_sql_detector.dart (lines 244–260), stripCommentsAndStrings(String sql) detects PostgreSQL dollar-quotes ($$ or $tag$):
// 3. Dollar quotes in PostgreSQL: $$ or $tag$
if (sql[i] == '$') {
final match =
RegExp(r'^$([a-zA-Z0-9_]*)$').matchAsPrefix(sql.substring(i));
if (match != null) {
final tag = match.group(0)!;
i += tag.length;
final closeIdx = sql.indexOf(tag, i);
...- Dynamic RegExp compilation: On every
$encountered (including parameters like$1,$2), a newRegExpinstance is compiled at runtime. - Substring copying:
sql.substring(i)copies the entire remainder of the script into a newStringbuffer. For a multi-megabyte SQL file, this creates severe heap churn and GC pauses. - Char string allocations: The scanner indexes characters via
sql[i]rather than unboxedsql.codeUnitAt(i), creating single-character string allocations on every step.
Architectural Solution: Zero-Allocation Micro-Functions
Decompose the monolithic 120-line stripCommentsAndStrings loop into dedicated, inlineable zero-allocation micro-scanner functions. Each function accepts (String sql, int start, int len) and returns the int nextOffset without heap allocations or try-catch:
static int _skipLineComment(String sql, int start, int len)- Scans until
\nor\rusingcodeUnitAt(i).
- Scans until
static int _skipBlockComment(String sql, int start, int len)- Scans until
*/usingcodeUnitAt(i).
- Scans until
static int _skipDollarQuote(String sql, int start, int len)- Uses precompiled static
_dollarTagwithmatchAsPrefix(sql, start). - Locates closing tag via
sql.indexOf(tag, start + tag.length)without substrings.
- Uses precompiled static
static int _skipStringLiteral(String sql, int start, int len, int quoteCode)- Fast scanner for single/double quotes with quote escaping (
'',\').
- Fast scanner for single/double quotes with quote escaping (
Main loop becomes clean and declarative:
while (i < len) {
final code = sql.codeUnitAt(i);
if (code == _chDash && i + 1 < len && sql.codeUnitAt(i + 1) == _chDash) {
i = _skipLineComment(sql, i + 2, len);
buffer.write(' ');
continue;
}
if (code == _chSlash && i + 1 < len && sql.codeUnitAt(i + 1) == _chAsterisk) {
i = _skipBlockComment(sql, i + 2, len);
buffer.write(' ');
continue;
}
if (code == _chDollar) {
final next = _skipDollarQuote(sql, i, len);
if (next > i) {
i = next;
buffer.write("''");
continue;
}
}
if (code == _chSingleQuote) {
i = _skipStringLiteral(sql, i + 1, len, _chSingleQuote);
buffer.write("''");
continue;
}
buffer.writeCharCode(code);
i++;
}Acceptance Criteria
- Each token/comment/quote kind is encapsulated in a dedicated micro-function.
- Micro-functions operate strictly with
codeUnitAtand primitive offsets, producing zero heap allocations. - No dynamic RegExp instances are compiled during scanning.
- All unit tests in
test/core/database/destructive_sql_detector_test.dartpass.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P2Medium priority / Parity & RefactoringMedium priority / Parity & RefactoringcoreCore library logic and servicesCore library logic and servicesparserTheme parser epic label: parserTheme parser epic label: parserperformanceTheme parser epic label: performanceTheme parser epic label: performance
Activity
Merge pull request #1603 from QueryaHub/issue/1511-destructive-sql-de…