Repository navigation
Release the input buffer when SChunk(data=...) fails; test the prefilter writes without the GIL - #777
Open
FabianIsensee wants to merge 1 commit into
Open
FabianIsensee wants to merge 1 commit into
FabianIsensee wants to merge 1 commit into
Conversation
The squash of Blosc#765 resolved the conflict in SChunk.__init__() from main's side, which releases the input buffer only when c-blosc2 returns an error. When _store_buffer() raises (a chunk larger than BLOSC2_MAX_BUFFERSIZE), the buffer stayed exported. The loop is back in a try/finally. test_data_writes_reach_prefilter() also runs with set_releasegil(True) now: since Blosc#766 the writes keep the GIL for Python prefilters. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This was referenced Nov 6, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi Francesc,
a follow-up to #765, from the look at the merged version you asked for. The squash resolved the conflicts with #766 as intended (
_store_buffer()decides on the GIL with_can_compress_without_gil(), and errors go through_raise_callback_error()), except inSChunk(data=...): there the resolution kept main's loop, which releases the input buffer only when c-blosc2 returns an error. When_store_buffer()raises, for a chunk larger thanBLOSC2_MAX_BUFFERSIZE, the buffer stays exported for good:The loop is back in a
try/finally, as on the PR branch.test_data_writes_reach_prefilterran with the GIL held only, because Python prefilters crashed withset_releasegil(True)before #766. Now that #766 keeps the GIL for them, it runs with the GIL released too, which checks that_store_buffer()follows that rule.Tests: there is no test for the leak itself, as it needs a 2 GiB buffer; the snippet above raises the BufferError on main and resizes the bytearray with this PR. Full suite: 11001 passed; the 2 b2view http tests fail without aiohttp, as on main.
Merge order: none needed. It merges cleanly with each of the open #769-#775, including #772, which changes
_store_buffer()itself.I used Claude Code (Anthropic) to check the merged version, write the change and run the tests; I reviewed the result.
Best,
Fabian
🤖 Generated with Claude Code