Repository navigation
feat(client): Add an Objectstore client for batched uploads - #133
Conversation
Port the Objectstore upload client from the Sentry CLI (getsentry/toolkit#2062) into `@sentry/api/client`, so hosts share it. `findExistingObjects` and `putObjects` group HEADs and small inserts into multipart batch requests (1000 ops, 25 MB, 4 in flight) and PUT objects over 1 MB individually (5 in flight). One request per object trips the Sentry proxy's per-request rate limit. The client uses only web-standard APIs, like the rest of the entry: hosts pass each upload's bytes through `read()` and may inject `fetch`. Failures throw `ObjectstoreError`, an `ApiError` carrying the URL.
BYK
left a comment
BYK
left a comment
There was a problem hiding this comment.
We should also add a lint rule to the CLI repo to enforce not adding API-specific stuff there and point them to this repo.
| ); | ||
| } | ||
|
|
||
| const delimiter = encoder.encode(`--${boundary}`); |
There was a problem hiding this comment.
OMG, don't we have a native multipart parser in web standards? I'm happy to use an "if node use this, if web use this" thing if they both have separate native versions.
There was a problem hiding this comment.
I checked. Neither the web platform nor Node has a native parser that works for this protocol. The only one is Response#formData() (undici in Node, the same API in browsers). It exposes just each part's name, filename and value, and drops any other part headers. Objectstore puts each operation's result in per-part headers: x-sn-batch-operation-index and x-sn-batch-operation-status on responses (see objectstore-server/src/batch.rs). Requests have the same problem: FormData can't set the per-part x-sn-batch-operation-kind/-key/x-sn-expiration headers the server reads. With formData() we'd lose the status and index of every operation, so the hand-rolled byte-level encoder and parser stays. It matches the Rust client's many.rs. I'm leaving this as is unless you'd rather change the server protocol.
There was a problem hiding this comment.
Can you at least file an issue to the relevant repo (guessing this is Relay? Or objectstore (https://github.com/getsentry/objectstore)) to make this multi-part compliant with a good solution proposal. I'm so annoyed right now.
There was a problem hiding this comment.
Filed getsentry/objectstore#769. It proposes an opt-in FormData-native batch format: a leading JSON operations manifest in requests, and in responses one JSON "<index>" result part per operation plus a "<index>.body" file part for GET payloads. The current header format stays for the Rust and Python clients. It also asks for a quick standalone fix, because the server's unquoted name=part makes Node's Response.formData() throw on today's responses. Once something like that ships, this client can drop its hand-rolled encoder and parser.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Use p-limit, as the CLI does, instead of a local limiter. The client entry already has runtime dependencies (http-cache-semantics), and p-limit uses only web-standard APIs. The client bundle keeps it external, and the packed-package tests link every runtime dependency.
Read the uploads of a batch one at a time, so the batches in flight hold at most 4 files open instead of up to 1000 each (EMFILE). Percent-encode each `/`-separated key segment in object URLs with the path-segment set the Rust client uses (url crate), so keys with `%`, `?`, `#`, spaces, or non-ASCII PUT the same object that batch operations address.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit bdc75d7. Configure here.
p-limit keeps starting queued tasks after one rejects, so a failed batch or PUT (e.g. a 429) was followed by more requests and file reads. Clear the queues of the limiters in the same operation when a task fails, before p-limit can start the next one.
Match the url crate's SPECIAL_PATH_SEGMENT set, which the Rust client uses for http(s) object URLs: add `\` (fetch parses it as `/`, storing the object under another key) and `^`.
`main` has failed Build and Build (Node LTS) since a47f626. **What broke:** the a47f626 regen of `openapi-derefed.json` added `feature_id` to the autofix POST response and made it required. That matches Sentry: `GroupAutofixEndpoint` (`src/sentry/seer/endpoints/group_ai_autofix.py`) always returns `{run_id, sentry_run_id, feature_id}`, and the API docs example includes `"feature_id": "autofix"`. The schema is correct. The test fixtures still built the old two-field response, which caused: - `pnpm typecheck`: `test/typecheck.ts(33,7): error TS2741: Property 'feature_id' is missing` - `pnpm test`: the two "runtime validator entry points" smoke tests (Valibot and Zod) rejected the fixture. **Fix:** add `feature_id: "autofix"` to both fixtures. No test or check is skipped or loosened; the negative `run_id: "42"` cases are unchanged. **Verification:** I ran the Build workflow steps locally on Node 22.23 from a clean checkout of `main` plus this change. Node LTS wasn't available locally, so CI covers that job. - `pnpm install --frozen-lockfile`: pass - `pnpm build`: pass - `pnpm typecheck`: pass. Without the change it reports 1 TS error. - `pnpm test`: 51 files and 635 tests pass. Without the change, `smoke.test.ts` has 2 failures. - `npm pack --dry-run`: pass Unblocks CI on #133. <!-- junior-request-attribution:start --> via @BYK. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:agent%3A790a7f162e08c84f43da8a5304ce4980 --> -- [View Session](https://junior-prod.sentry.dev/conversations/agent%3A790a7f162e08c84f43da8a5304ce4980) [[Sentry]](https://sentry.sentry.io/explore/conversations/agent%3A790a7f162e08c84f43da8a5304ce4980/?project=4510944073809921) <!-- junior-session-footer:end --> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Burak Yigit Kaya <[email protected]>
Bumps @sentry/api to 1.44.0, which ships the batched Objectstore client from getsentry/sentry-api-schema#133, and replaces the CLI's own protocol code with it. src/lib/objectstore.ts is now a thin adapter: it reads files from disk when their request starts, sends requests through customFetch so custom CAs still apply, and rethrows ObjectstoreError as the CLI's ApiError. Batching and limits are unchanged: 4 concurrent batches, 25 MB per batch, and 5 concurrent PUTs for files over 1 MB. Protocol tests now live in @sentry/api; the CLI tests cover the adapter. The bump also regenerates the alert and event skill references, which pick up field descriptions from the newer schema. Co-Authored-By: Jamie Quadri <[email protected]> Co-Authored-By: Burak Yigit Kaya <[email protected]>
Moves the Objectstore upload client from the Sentry CLI into
@sentry/api/client, as requested in review on getsentry/toolkit#2062. That PR fixessentry snapshots uploadhitting HTTP 429. The new CLI sent one request per image, and the Sentry proxy in front of Objectstore rate-limits per request. The old Rust CLI batched requests (Session::many()), and this client does the same.The logic matches toolkit at 5459297:
findExistingObjectssends HEADs in multipart batches to/v1/objects:batch/{usecase}/{scope}/, up to 1000 operations per batch and 4 batches in flight.putObjectsbatches inserts up to 1 MB each, within 1000 operations and 25 MB per batch, with 4 batches in flight. It PUTs larger objects one by one, up to 5 at once, in parallel with the batches.API (
@sentry/api/client)Choices
@sentry/api/clientis already the web-standard request layer that hosts inject things into. Putting the client there meanstsconfig.browser.jsonandportability.test.tscheck it automatically.node:imports and noBuffer. Multipart encoding and parsing work on bytes (TextEncoder,Uint8Array), and boundaries come fromcrypto.randomUUID(). The host supplies file contents throughread(), which runs only when that upload's request starts. The CLI will pass() => readFile(path)and the size fromstat.fetch. It defaults toglobalThis.fetch. The CLI will pass its custom-CA fetch.ObjectstoreErrorextends this package'sApiError, soinstanceof SentryApiErrorstill matches. It addsurl, which is the failing object's URL or the request URL, and the CLI's own error type needs it.p-limit. Once a task fails, queued tasks don't start.putObjectandbuildObjectUrlstay internal because the CLI only uses the two functions above.Verification
test/api-client/objectstore.test.ts(13 tests, ported from toolkit) covers batch shape, key encoding, concurrency (3 batches in flight for 2,500 keys and for 60 MB), per-operation, request, and protocol errors, and individual PUTs. All 13 pass, and so does the wholetest/api-clientsuite (370 tests).distagainst a localghcr.io/getsentry/objectstore:nightly. 1,200 objects took 2 batched HEAD requests and 3 upload requests (2 batches plus 1 PUT for a 2 MB object). Non-ASCII keys round-tripped, and the bytes read back matched.pnpm buildandtsc -p tsconfig.browser.jsonpass.pnpm typecheckandpnpm testfail onmainwithout this change, and they fail the same way with it.test/typecheck.tsand twosmoke.test.tscases expect afeature_idfield that was added to the schema in the last generated spec. Main's Build has been red since a47f626.Toolkit will switch
packages/clito this client after the next@sentry/apirelease. Toolkit currently pins 1.26.0, so it needs a catalog bump at the same time.via @BYK.
--
View Session [Sentry]