Skip to content

feat: Support gzip compression in server, Rust and Python clients - #693

Open
sentry-junior[bot] wants to merge 9 commits into
mainfrom
feat/server-gzip-compression
Open

sentry-junior[bot] wants to merge 9 commits into
mainfrom
feat/server-gzip-compression

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Adds gzip compression alongside Zstd in the server, Rust client, and Python client. Both clients now support compressed uploads and transparent download decompression. Zstd remains the default.

  • GCS payload downloads send Accept-Encoding: gzip to prevent decompressive transcoding, which otherwise returns plaintext and ignores Range. This preserves stored bytes for compressed downloads and range reads. The backend HTTP client already disables automatic decompression.
  • Both clients decode concatenated gzip members, including payloads assembled by multipart uploads.
  • The Rust client compresses all upload body types through a generic helper that avoids extra reader and stream boxes.
  • Rust batch sizing uses a conservative gzip size bound because streaming compression leaves the final size unknown. The bound uses zlib's compressBound formula without the zlib wrapper, plus gzip framing. Precompressed uploads use their exact size.
  • Python shares compression handling across streaming uploads, resumable-upload spooling, and downloads.
  • Python rejects unsupported compression values up front, including on Usecase. Previously, an unknown usecase encoding caused Zstd compression with an incorrect encoding label.
  • Adding Compression::Gzip to the shared Rust enum breaks downstream exhaustive matches. Callers must handle the new variant or use a wildcard arm.

Closes FS-575

@linear-code

linear-code Bot commented Oct 20, 2026

Copy link
Copy Markdown

FS-575

@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.37828% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.01%. Comparing base (2612bee) to head (0246e2c).

Files with missing lines Patch % Lines
...ents/python/src/objectstore_client/_compression.py 92.70% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #693      +/-   ##
==========================================
+ Coverage   91.90%   92.01%   +0.11%     
==========================================
  Files         117      118       +1     
  Lines       24131    24321     +190     
==========================================
+ Hits        22178    22380     +202     
+ Misses       1953     1941      -12     
Components Coverage Δ
Rust Backend 94.94% <100.00%> (+0.02%) ⬆️
Rust Client 84.29% <100.00%> (+1.42%) ⬆️
Python Client 93.88% <93.57%> (+0.15%) ⬆️

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sentry-junior
sentry-junior Bot force-pushed the feat/server-gzip-compression branch from 9f66941 to 74adec7 Compare October 20, 2026 13:10
@sentry-junior sentry-junior Bot changed the title feat: Support gzip compression in server and Rust client feat: Support gzip compression in server, Rust and Python clients Oct 20, 2026
Comment thread clients/python/src/objectstore_client/_compression.py Outdated
Comment thread clients/python/src/objectstore_client/_compression.py Outdated
Comment thread clients/rust/src/many.rs Outdated
Comment thread clients/rust/src/many.rs Outdated
Adds Compression::Gzip to the shared types, so the server accepts and
stores Content-Encoding: gzip. GCS (and S3-compatible) downloads now
send Accept-Encoding: gzip to opt out of GCS decompressive transcoding,
which would otherwise return decompressed bytes and ignore Range.

The Rust client can compress, decompress, and size-bound gzip uploads.

Refs FS-575
S3 and SeaweedFS serve objects as stored regardless of Accept-Encoding,
so only the GCS backend needs to opt out of transcoding.
Uploads can be compressed with gzip on the fly or spooled for resumable
uploads, precompressed gzip payloads are accepted, and gzip downloads
are transparently decompressed, including multi-member payloads from
multipart uploads. Unsupported algorithms are rejected up front,
including on Usecase.

Refs FS-575
- py-client: Replace the zstd-specific reader wrapper and the gzip reader
  with a single _CompressionReader driven by a compress/flush compressor.
- py-client: Document why the gzip compressor sets wbits.
- rust-client: Name the parts of the gzip size bound and drop its test.
@jan-auer
jan-auer force-pushed the feat/server-gzip-compression branch from d72e30a to 810ab32 Compare October 22, 2026 12:27
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Semver Impact of This PR

⚪ None (no version bump detected)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


Python Client

  • Split the signing key from the token policy: SecretKey(kid, secret_key) now only holds the EdDSA key, and TokenGenerator(key, expiry_seconds=60, permissions=None) wraps it with the token defaults and is what you pass to Client(token=...). The deprecated sign_for_scope() alias is removed. by sentry-junior in #721
    • Replace SecretKey(kid, secret_key, expiry_seconds, permissions) or TokenGenerator(kid, secret_key, ...) with TokenGenerator(SecretKey(kid, secret_key), expiry_seconds, permissions).
      # Before
      client = Client(url, token=SecretKey(kid, secret_key, expiry_seconds=300))
      
      # After
      key = SecretKey(kid, secret_key)
      client = Client(url, token=TokenGenerator(key, expiry_seconds=300))
    • Replace .sign_for_scope(...) with .token_for_scope(...).
  • Remove the deprecated compression argument from Session.put() by sentry-junior in #718
    • Replace session.put(..., compression=...) with session.put(..., compress=...).
  • Remove the deprecated propagate_traces argument from Client by sentry-junior in #714
    • Drop propagate_traces=... from Client(...) calls. Sentry's StdlibIntegration, enabled by default, already propagates traces on Objectstore requests.
  • Add a response_content_disposition option ("inline" or "attachment") to Session.object_url() and Session.presigned_object_url(). Pre-signed URLs include it in their signature. by jan-auer in #696

Rust Client

  • Remove the deprecated Session::mint_token() by sentry-junior in #717
    • Replace session.mint_token() with session.get_token().
  • Remove the deprecated TokenGenerator::sign() by sentry-junior in #716
    • Replace generator.sign(&scope) with generator.create_token(&scope).sign().
  • Remove the deprecated PutBuilder::compression() by sentry-junior in #715
    • Replace .compression(...) with .compress(...) on put builders.
  • Usecase::new now returns a Result, and the panicking constructor is now Usecase::from_static by sentry-junior in #710
    • Replace Usecase::new("name") with Usecase::from_static("name").
    • Replace Usecase::try_new(name) with Usecase::new(name).
  • Mark Error as #[non_exhaustive] so new error variants can be added without a breaking change by lcian in #706
    • Add a wildcard arm (_ => ...) to exhaustive match expressions on objectstore_client::Error.
  • Remove the multipart upload API (Session::initiate_multipart_upload(), Session::resume_multipart_upload()) and the multipart feature flag by lcian in #704
    • Use resumable uploads instead.
    • Remove match arms for the removed Error::InvalidPartNumber, Error::InvalidUploadId and Error::MultipartComplete variants.

Other

  • feat: Support gzip compression in server, Rust and Python clients by sentry-junior[bot] in #693

🤖 This preview updates automatically when you update the PR.

@sentry-junior sentry-junior Bot changed the title feat: Support gzip compression in server, Rust and Python clients feat(rs-client)!: Support gzip compression Oct 22, 2026
@jan-auer
jan-auer marked this pull request as ready for review October 22, 2026 15:34
@jan-auer
jan-auer requested a review from a team as a code owner October 22, 2026 15:34
@lcian lcian changed the title feat(rs-client)!: Support gzip compression feat: Support gzip compression in server, Rust and Python clients Oct 22, 2026

@lcian lcian left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Junior, please answer my comment, and also check if we need to do anything in our Objectstore proxy endpoint in the sentry repo after this ships.

Leaving it to Jan to decide whether we want to feature flag this in the Rust client.

Comment thread Cargo.toml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants