Skip to content

Reshape uses virtual view instead of deep copy - #90

Open
konstibob wants to merge 1 commit into
zarr-developers:mainfrom
konstibob:reshape_codec
Open

Reshape uses virtual view instead of deep copy#90
konstibob wants to merge 1 commit into
zarr-developers:mainfrom
konstibob:reshape_codec

Conversation

@konstibob

Copy link
Copy Markdown
Contributor

added mechanism similar to https://github.com/numpy/numpy/blob/main/numpy/_core/src/multiarray/shape.c

to construct a virtual view instead of copying when possible, only adjusting the shape and stride parameters in the indexCal Parameter of a ma2.Array

@konstibob konstibob changed the title uses virtual view instead of deep copy Reshape uses virtual view instead of deep copy Aug 21, 2026
Comment on lines +90 to +112
*
* <p>The two obvious candidates are both wrong here. {@link Array#reshape} always allocates and
* copies. {@link Array#reshapeNoCopy} hands the raw backing store to the new shape and discards
* the input's strides and offset, so it silently reorders the elements of any view &mdash; such as
* the output of the {@code transpose} codec, or the strided section that {@code Array.write}
* passes in for every chunk of a multi-chunk write.
*
* <p>{@link Array#get1DJavaArray} instead returns the backing store itself when the input already
* walks it in lexicographical order (ma2 tracks this as {@code Index.fastIterator}), and a C-order
* copy when it does not. The result therefore shares its storage with the input whenever the
* elements are already laid out in {@code ravel} order, and is a correct copy otherwise. Either
* way {@code ravel(B) == ravel(A)} holds, and the reshaped array is itself in lexicographical
* order, so a following codec gets the cheap path too.
*
* <p>This is conservative compared to NumPy's {@code _attempt_nocopy_reshape}, which also keeps a
* view when splitting the axes of a strided array, and when the axes being merged happen to be
* internally contiguous ({@code stride[k] == shape[k+1] * stride[k+1]}). Matching that would need
* an {@link ucar.ma2.Index} with custom strides and a non-zero offset, which ma2's public API
* cannot build safely: {@code new Index(shape, stride)} forces {@code offset = 0} and leaves the
* internal {@code fastIterator} flag set, so the resulting array would later hand out its whole
* backing store as if it were the data. Decode is unaffected either way, because it always
* receives a freshly allocated array; on encode the extra copy is limited to multi-chunk writes,
* where the caller passes a strided section.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
*
* <p>The two obvious candidates are both wrong here. {@link Array#reshape} always allocates and
* copies. {@link Array#reshapeNoCopy} hands the raw backing store to the new shape and discards
* the input's strides and offset, so it silently reorders the elements of any view &mdash; such as
* the output of the {@code transpose} codec, or the strided section that {@code Array.write}
* passes in for every chunk of a multi-chunk write.
*
* <p>{@link Array#get1DJavaArray} instead returns the backing store itself when the input already
* walks it in lexicographical order (ma2 tracks this as {@code Index.fastIterator}), and a C-order
* copy when it does not. The result therefore shares its storage with the input whenever the
* elements are already laid out in {@code ravel} order, and is a correct copy otherwise. Either
* way {@code ravel(B) == ravel(A)} holds, and the reshaped array is itself in lexicographical
* order, so a following codec gets the cheap path too.
*
* <p>This is conservative compared to NumPy's {@code _attempt_nocopy_reshape}, which also keeps a
* view when splitting the axes of a strided array, and when the axes being merged happen to be
* internally contiguous ({@code stride[k] == shape[k+1] * stride[k+1]}). Matching that would need
* an {@link ucar.ma2.Index} with custom strides and a non-zero offset, which ma2's public API
* cannot build safely: {@code new Index(shape, stride)} forces {@code offset = 0} and leaves the
* internal {@code fastIterator} flag set, so the resulting array would later hand out its whole
* backing store as if it were the data. Decode is unaffected either way, because it always
* receives a freshly allocated array; on encode the extra copy is limited to multi-chunk writes,
* where the caller passes a strided section.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants