Repository navigation
perf(linalg): reduce Gram buffer copies -- 49.9% less peak memory - #1301
Merged
Merged
Conversation
Use a Fortran output buffer and return its symmetric transpose. Reuse symmetrify and let BLAS handle input conversion. Save 103,680,096 peak allocated bytes for a 3600-by-8 design. Preserve supplied-output behavior and fallback types. Co-Authored-By: Aiden <[email protected]>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1301 +/- ##
=======================================
Coverage 62.15% 62.15%
=======================================
Files 206 206
Lines 21955 21955
Branches 2564 2564
=======================================
Hits 13646 13646
Misses 7682 7682
Partials 627 627
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Member
|
Thanks Dex, some significant memory improvements here. |
Member
|
Thanks Dex, again significant memory improvements! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
tdot_blaswhen building dense Gram matrices.tracemalloc, starting after input creation.What & Why
The old code builds a C-order output, then BLAS copies it to Fortran order.
Returning a C-order result adds another full matrix copy.
The change starts with a Fortran-order buffer and reuses the shared symmetry helper.
Its transpose has the same symmetric values and C-order layout, so returning it avoids another allocation.
Validation
Peak allocated memory for one Gram computation, with five repeats per situation:
The complete allocation workload also checks small arrays, empty dimensions, strided inputs, supplied buffers, and fallback types.
Three runs produced these statistics:
The 15 observations repeat the same five input seeds across three runs.
The complete workload retains the small validation results, so its allocation totals differ slightly from individual calls.
All normalized output bytes, layouts, mutations, aliases, and exceptions matched the original implementation.
Native validation: 2 passed, with one existing plotting deprecation warning.
To reproduce a single-call allocation measurement on each revision, run:
Inputs exist before tracing starts. Measurements use Linux, Python 3.12.13, NumPy 2.5.4, SciPy 1.18.1 BLAS, and one BLAS thread.
Peak allocation is measured with
tracemalloc; it is not process resident memory.Why this is safe and easy to maintain
The small change keeps the existing shared symmetry helper and adds no net code lines.
It preserves numerical results, output layout, and supplied-buffer behavior.
The full-rank and product-kernel tests cover the production callers.
The helper keeps its existing native and NumPy implementations.
AI assistance
This PR was prepared with AI assistance from Aiden. Candidates were tracked with Weco Observe.
Weco share link: https://dashboard.weco.ai/share/UZdek2QWIzbIY-gluzhnKnbGDUj_cb6n