Skip to content

add SofaStream.verify - #138

Open
artpelling wants to merge 12 commits into
developfrom
verify-stream
Open

add SofaStream.verify#138
artpelling wants to merge 12 commits into
developfrom
verify-stream

Conversation

@artpelling

@artpelling artpelling commented Jun 19, 2026

Copy link
Copy Markdown
Member

SofaStream has no verify method. For verification of conventions, one would need to read the entire file into memory which defeats the purpose of SofaStream. I've added a verify method to solve that. The method reads everything but the actual numerical data from the (potentially too large for memory) .sofa into a Sofa in-memory object and calls Sofa.verify.

Changes proposed in this pull request:

  • added SofaStream.verify
  • changed np.array to np.asarray in utils.py to avoid always loading everything into memory
  • added tests

Disclaimer:

I partially used an LLM for the NetCDF4 reading stuff. Local tests seem to work well

@artpelling artpelling moved this from Backlog to Require review in Weekly Planning Jun 19, 2026
@artpelling
artpelling requested review from a team, ahms5, f-brinkmann and hoyer-a June 19, 2026 14:52
@artpelling artpelling added this to the v1.2.3 milestone Jun 19, 2026
@artpelling artpelling changed the title add verify to SofaStream add SofaStream.verify Jun 19, 2026

@f-brinkmann f-brinkmann left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the effort. I think this is great to have. I suggested a few small changes and to think about a good way to efficiently and comprehesively test the added functionality.

Oh and please already add this to the HISTORY. We do not have a test to enforce this yet but should already start this new best practice.

Comment thread sofar/sofastream.py Outdated
Comment thread sofar/sofastream.py
Comment thread sofar/sofastream.py Outdated
Comment thread tests/test_sofastream.py Outdated

@hoyer-a hoyer-a left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, code looks good and this is a great addition.
I only have a conceptual question, since this would differ from how SofaStream is being used so far.

Comment thread sofar/sofastream.py Outdated
@mberz mberz moved this from Require review to Agenda in Weekly Planning Jul 17, 2026
@mberz mberz modified the milestones: v1.2.3, v1.3.0 Jul 17, 2026
@mberz
mberz changed the base branch from main to develop July 17, 2026 12:17
@mberz mberz moved this from Agenda to Implementation in progress in Weekly Planning Jul 17, 2026
@mberz

mberz commented Jul 17, 2026

Copy link
Copy Markdown
Member

Since this PR proposes a new feature, I've marked it for milestone 1.3.0, not as a patch (bugfix) release

@artpelling

Copy link
Copy Markdown
Member Author

I've implemented all requested changes.

Since this PR proposes a new feature, I've marked it for milestone 1.3.0, not as a patch (bugfix) release

Sure. May I ask if you would consider making a beta release to PyPI with this feature? I want to depend on it in irdl but I cannot publish a new release unless these changes somehow make it onto the index. If a proper release is available, beta releases should be ignored during a standard installation process unless explicitly requested, so I could explicitly depend on it and the normal user would not notice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Implementation in progress

Development

Successfully merging this pull request may close these issues.

4 participants