Repository navigation
Refactor native binary build process for current platform only - #2421
Merged
Merged
Conversation
Cross-platform builds with build-native never produced a working
binary, and nothing uses them; every CI job builds on its own
platform. Two things break them, both silently at build time:
- pkg compiles bytecode on the host, and V8 on the target rejects it
("V8 rejected the bytecode cache", seen building for win-x64 on
Linux and running under wine);
- native addons are packaged as yarn install compiled them. pkg's
prebuild-install step, which is meant to fetch an addon for the
target, looks for prebuild-install under @yao-pkg/pkg/node_modules,
where it is not installed in this workspace. Even when reachable,
its node-abi 3.57.0 maps Node 22 to ABI 124 instead of 127, so the
download 404s. Failures are logged at debug level only, and pkg
falls back to the host addon ("Bad EXE format" under wine).
So build-native is now a Node script that only builds for the current
platform. OS and ARCH stay as optional arguments, so the CI calls are
unchanged, but now they only assert the platform and a mismatch is an
error. The pkg target is taken from the package's pkg.targets, which
becomes the single place the Node version is declared. It always
passes --no-native-build, so it no longer depends on is_native, which
never matched on Windows ("windows" vs pkg's "win"). That made no
practical difference, given the above: Windows also ended up with the
addon yarn install built.
Root `yarn build-native` used to try every target; it now builds each
package for the current platform. The scripts call the file through
`node` so they do not rely on shebang handling on Windows; scanner's
bin/build-native stays a symlink to this one.
Assisted-by: Claude:claude-opus-5-5
build-native packages native addons (better_sqlite3.node) exactly as yarn install compiled them. If that ran on a different Node major than the pkg target (node22), the build still succeeds, after several minutes, and the binary then dies at runtime with ERR_DLOPEN_FAILED (NODE_MODULE_VERSION mismatch). build-native now checks first. It takes the addons from the .node entries in pkg.assets and fails if the current Node major differs from the target's, or if any addon does not load under process.dlopen. The error names the `yarn rebuild` to run. Packages without native assets, like scanner, skip the check. `yarn build-native` runs it via --check-only before `yarn build`, so it fails in under a second instead of after the TypeScript build. Yarn passes "$@" through without also appending the arguments. The binary test setup calls build-native directly and gets the check as part of the build. Assisted-by: Claude:claude-opus-5-5
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
There are no active review issues for the host-only Node.js native-binary build changes.
No open issues 🎉
What changed in this PR
Replaces the shared native-binary build script with Node.js, limiting CLI and scanner builds to the current platform and adding native-addon compatibility checks before building.
Changes:
- Removes cross-platform builds and selects the host target from each package’s
pkg.targets. - Adds a preflight check for Node-version compatibility and native-addon loading.
- Updates package scripts and binary-test setup to invoke the builder through Node.js.
| File | Description |
|---|---|
packages/scanner/package.json |
Runs preflight checks before building the scanner binary. |
packages/cli/tests/binary/helpers/setup.ts |
Invokes the native builder through Node.js. |
packages/cli/package.json |
Runs preflight checks before building the CLI binary. |
packages/cli/bin/build-native |
Replaces Bash with a host-only Node.js builder and native-addon checks. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Drop the cross-platform build support in build-native which was unused and broken; the build-native script is rewritten from shell to node and only allows building for the current platform. Additionally, add a node ABI preflight check to avoid building binaries that will be broken (mostly useful in dev environments).