Repository navigation
Stream data from Memory functionality - #90
psharma-1909 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ea2ab39. Configure here.
| this._localStream = null; | ||
| this._unpipeStream = function () {}; | ||
| this._streamFactory = options.infileStreamFactory; | ||
| assignLocalInfileOptions(this, options); |
There was a problem hiding this comment.
Execute skips LOCAL INFILE completion
High Severity
Execute now accepts infileStream and a connection infileStreamFactory, but it never copies infileOk from Query. After _streamLocalInfile sends file bytes it returns undefined, so the command emits end before the server OK. That desynchronizes the connection and can release a pooled connection while the stream is still writing.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit ea2ab39. Configure here.




Summary
Aligns singlestore-nodejs with SingleStore Load Data from Local Files:
LOAD DATA LOCAL INFILEalready worked viainfileStreamFactory; this PR adds in-memory streaming, an explicitlocal_infileconnection option, fixes factory inheritance for prepared statements and pool cluster, and documents local-file vs in-memory loads.Changes
Features
infileStream(query option) — Stream bytes from a Node.jsReadableforLOAD DATA LOCAL INFILE. The filename in SQL is ignored. Takes precedence overinfileStreamFactory.local_infile(connection option) — ControlsCLIENT_LOCAL_FILES(default true). Whenfalse, file and stream loads fail with a clear client error; relevant server errors (1148 / 3948 / 4166) are rewritten when appropriate.lib/commands/local_infile.js— Shared helpers for option assignment, disabled errors, and error rewriting.Query/Execute— Shared local-infile handling; connection-level factory used when the query does not set one.PreparedStatementInfo.execute— Passes connectioninfileStreamFactoryintoExecute.PoolCluster.query— Inherits connection-levelinfileStreamFactorywhen omitted on the query.Types
ConnectionOptions:local_infileQueryOptions:infileStreamTests
test/unit/commands/test-local-infile.test.cjs,test/unit/connection/test-connection_config.test.cjs,test/unit/pool-cluster/test-load-infile-factory.test.cjstest/integration/connection/test-load-infile.test.cjs(memory stream,local_infile: false)Documentation
Note
Medium Risk
Changes
LOAD DATA LOCAL INFILEbehavior (new in-memory path and client-side disable), which affects bulk ingest and a historically sensitive protocol feature, though defaults stay permissive and opt-out is explicit.Overview
Adds in-memory bulk load for
LOAD DATA LOCAL INFILEvia a new query optioninfileStream(a NodeReadable). When set, uploaded bytes come from that stream and the filename in SQL is ignored;infileStreamwins overinfileStreamFactory.Introduces
local_infileon the connection (default true), wired toCLIENT_LOCAL_FILES, with string/boolean parsing and interaction withflags. When disabled, loads fail early with a clear connector message, and selected server errors (1148 / 3948 / 4166) are rewritten for LOCAL INFILE SQL via sharedlib/commands/local_infile.js.Query and Execute share the same option assignment and streaming path (connection-level factory when the query omits one). Prepared statement
executeandPoolCluster.querynow propagate connectioninfileStreamFactorywhere it was missing before.TypeScript adds
local_infileandinfileStream; README and site docs document file vs memory loads and fixsinglestore-nodejsrequire examples. Unit and integration tests cover memory loads, disabled infile, config flags, and pool cluster inheritance.Reviewed by Cursor Bugbot for commit ea2ab39. Bugbot is set up for automated code reviews on this repo. Configure here.