Conversation
…un out of fuel Fixes #236. The s3 dispatcher signs each request with SigV4, which includes the SHA-256 of the body. It hashed the body in WASM, at about 120 fuel per byte, while a call's fuel budget (100M) does not grow with the body: the body travels outside the metered JSON. Uploads over about 750 KB ran out of fuel and failed with 500. - host_sha256(data_ptr, data_len, out_ptr) -> i32, behind a new `hash` capability, hashes a range of plugin memory natively and refreshes the call's deadline, as the other blocking host calls do. Ranges are checked before anything is written; -1 for a negative or out-of-bounds range, and the plugin keeps running. - The SDK wraps it as hash::sha256 / hash::sha256_hex, hashing in-process outside WASM so plugin unit tests see the same digests. - The s3 plugin hashes the payload through it and declares `hash`. Also, as the issue noted, a failure was invisible outside dev mode: - Every request that fails inside the gateway is logged at error with its cause; a dispatch failure names its plugin. - A trap reports its cause: out of fuel says so and states the budget, a missed deadline is a Timeout, other traps keep wasmtime's full chain instead of only the backtrace header. Plugin lockfiles pick up the SDK's sha2 dependency; they also recorded the SDK at 0.7.0.
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository: barbacane-dev/barbacane/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #236.
Cause
SigV4 signs the payload hash. The s3 dispatcher computed it in WASM (software SHA-256, about 120 fuel per byte), while a call's fuel budget (100M) does not grow with the body, which travels outside the metered JSON. Uploads over about 750 KB ran out of fuel; outside dev mode the 500 left no trace.
Change
host_sha256(data_ptr, data_len, out_ptr) -> i32, behind a newhashcapability, hashes a range of plugin memory natively. Both ranges are checked before anything is written; a negative or out-of-bounds range returns -1 and the plugin keeps running; input and digest may overlap. It refreshes the call's deadline afterwards, as the other blocking host calls do, since hashing a large input takes wall-clock time the plugin did not spend running.hash::sha256/hash::sha256_hex. In-process outside WASM, so plugin unit tests see the same digests.hash. The canonical request stays small and is still hashed bybarbacane-sigv4.dev_error_response, 17 call sites) is logged aterrorwith its cause, which previously reached only a dev-mode caller. A dispatch failure names its plugin.WasmError::Timeout(defined, previously never produced), and other traps keep wasmtime's full cause chain instead of only the "error while executing at wasm backtrace" header.Compatibility: this
s3.wasmimportshost_sha256, so it needs a gateway of this version or later. An olders3.wasmkeeps working on the new gateway.Tests
Host function, pure (
barbacane-wasm/src/hash.rs, 13): FIPS 180-2 vectors, a millionas, every length from 0 to 257 bytes (across three 64-byte padding boundaries) againstringas an independent oracle, 20 MiB againstring, a range mid-memory with the bytes around it untouched, empty input at the start, middle and end, digest overwriting its input, digest ending exactly at the end of memory; refused: negative pointer, length or output, input one past the end, digest one byte short, memory smaller than a digest, overflowing ends. Every refused case checks memory is unchanged.Through the linker (
barbacane-wasm/tests/host_sha256.rs, 10, WAT modules): digest of a data segment, empty range, the whole memory to its last byte, out of bounds / digest not fitting / each negative argument returning -1 without a trap. Budgets: 12 MiB hashed with a 100k-fuel budget, paired with a control that touches each byte in WASM under the same budget and runs out; a large input under a 30 ms time budget, with the input sized at run time so that hashing it takes at least three budgets on the machine running the test, paired with a control that overruns it and must reportTimeout.Capability (6):
host_sha256refused with no capability,logorverify_signature; accepted withhash;hashgrants nothing else;plugin.tomlacceptshash.SDK (3) and s3 plugin (4 new, 28 total): payload hash equals the body's at 1, 55, 56, 63, 64, 65 bytes, 1 KiB, 1 MiB and 8 MiB; no body and an empty body sign the empty hash; a known vector; the signature changes when the last byte of a 1 MiB body changes.
End to end (
barbacane-test/tests/s3_dispatch.rs, 5): the gateway with the reals3.wasmin front of a wiremock S3 that accepts a PUT only ifx-amz-content-sha256is the SHA-256 of the bytes it received andAuthorizationequals the SigV4 signature recomputed withbarbacane-sigv4. Uploads of 1 B, 64 KB, 700 KB, 800 KB, 1 MiB, 4 MiB, 20 MiB and 24 MiB arrive byte-for-byte and correctly signed; an empty upload is signed over the empty hash; a body exactly at--max-body-sizeis stored; one byte over is a 413 and never reaches S3; a control shows the mock refuses a mismatched hash.Logging (
barbacane-test/tests/plugin_failures.rs, 1): a dispatcher trapping onunreachablegives 500, and the log line namesplugin 'trapper' dispatch failedand the trap.Mutation-checked
plugin 's3' dispatch failed: WASM execution trapped: out of fuel: the call used all 100000000 instructions of its budgethost_sha256dev_error_responseThe first deadline test used a 1 ms budget and flaked under the full workspace run; it now sizes its input to the machine, and passed in isolation and in two full workspace runs. During development, one run of
s3_dispatchfailed right after swappings3.wasmbuilds, with the name not captured; it did not recur in 11 later runs, three of them as three concurrent copies of the suite (15 gateways).Checks
cargo fmt, both CI clippy steps,cargo test --workspace --exclude barbacane-test(18 suites), the s3 plugin's fmt/clippy/tests, SDKcargo check --target wasm32-unknown-unknown, and thes3_dispatch,plugin_failures,plugins,compilation,proxy,validationintegration suites with--test-threads=2.