Repository navigation

Faster NDArray reads, writes, fills, serialization and expressions; fixes for crashes and wrong results - #744

Closed

FabianIsensee wants to merge 19 commits into

Blosc:mainfrom

FabianIsensee:perf-and-fixes

Closed

Faster NDArray reads, writes, fills, serialization and expressions; fixes for crashes and wrong results#744
FabianIsensee wants to merge 19 commits into
Blosc:mainfrom
FabianIsensee:perf-and-fixes

Commits on Oct 28, 2026
- Skip the index store in mark_indexes_stale for unindexed arrays
  
  FabianIsensee
  and
  claude
  committed
  7590301
- Release the GIL in NDArray slice reads
  
  FabianIsensee
  and
  claude
  committed
  499933b
- Make GIL-free slice reads safe around fork, exit and lazy expressions
  
  FabianIsensee
  and
  claude
  committed
  078cd62
Commits on Oct 29, 2026
- Tidy the mark_indexes_stale pre-check and its tests
  
  FabianIsensee
  and
  claude
  committed
  03db12a
Commits on Oct 30, 2026
- Merge the open PRs #742 and #743 as the base for the performance work
  
  FabianIsensee
  and
  claude
  committed
  7a076a0
- Fix crashes with frame-backed operands in miniexpr and matmul
  
  FabianIsensee
  and
  claude
  committed
  441dc57
- Make the general NDArray.__getitem__ path accept the keys NumPy accepts
  
  FabianIsensee
  and
  claude
  committed
  359144a
- Resolve basic keys of NDArray.__getitem__/__setitem__ in Cython
  
  FabianIsensee
  and
  claude
  committed
  e8f72af
- Lower the per-call overhead of creating arrays and of small writes
  
  FabianIsensee
  and
  claude
  committed
  6c47400
- Fix SChunk.update_special(), failed data writes and decompress_chunk()
  
  FabianIsensee
  and
  claude
  committed
  b30eefa
- Write scalars and NDArray values chunk by chunk in NDArray.__setitem__
  
  FabianIsensee
  and
  claude
  committed
  88d3ffc
- Cast whole chunks with reused buffers in asarray/astype
  
  FabianIsensee
  and
  claude
  committed
  35078a6
- Write cframes straight into the returned bytes
  
  FabianIsensee
  and
  claude
  committed
  554a307
- Speed up lazy-expression getitem, std/var and chunked reductions
  
  FabianIsensee
  and
  claude
  committed
  b28fb0e
- Keep expressions that miniexpr gets wrong on the numexpr path
  
  FabianIsensee
  and
  claude
  committed
  55660b4
- Clip the last chunk when collecting the True positions of a bool mask
  
  FabianIsensee
  and
  claude
  committed
  c6609db
- Start blosc2.nthreads from BLOSC_NTHREADS and keep NUMEXPR_NUM_THREADS
  
  FabianIsensee
  and
  claude
  committed
  cf1e115
- Add release notes for the NDArray performance work and fixes
  
  FabianIsensee
  and
  claude
  committed
  7510159
- TEMP: build against the companion c-blosc2 PR branch
  
  FabianIsensee
  and
  claude
  committed
  f8e910f

Navigation Menu

Repository navigation

Faster NDArray reads, writes, fills, serialization and expressions; fixes for crashes and wrong results - #744

Faster NDArray reads, writes, fills, serialization and expressions; fixes for crashes and wrong results#744
FabianIsensee wants to merge 19 commits into
Blosc:mainfrom
FabianIsensee:perf-and-fixes

Commits on Oct 28, 2026

Skip the index store in mark_indexes_stale for unindexed arrays

Release the GIL in NDArray slice reads

Make GIL-free slice reads safe around fork, exit and lazy expressions

Commits on Oct 29, 2026

Tidy the mark_indexes_stale pre-check and its tests

Commits on Oct 30, 2026

Merge the open PRs #742 and #743 as the base for the performance work

Fix crashes with frame-backed operands in miniexpr and matmul

Make the general NDArray.getitem path accept the keys NumPy accepts

Resolve basic keys of NDArray.getitem/setitem in Cython

Lower the per-call overhead of creating arrays and of small writes

Fix SChunk.update_special(), failed data writes and decompress_chunk()

Write scalars and NDArray values chunk by chunk in NDArray.setitem

Cast whole chunks with reused buffers in asarray/astype

Write cframes straight into the returned bytes

Speed up lazy-expression getitem, std/var and chunked reductions

Keep expressions that miniexpr gets wrong on the numexpr path

Clip the last chunk when collecting the True positions of a bool mask

Start blosc2.nthreads from BLOSC_NTHREADS and keep NUMEXPR_NUM_THREADS

Add release notes for the NDArray performance work and fixes

TEMP: build against the companion c-blosc2 PR branch