Skip to content

perf(linalg): reduce Gram buffer copies -- 49.9% less peak memory - #1301

Merged
lawrennd merged 1 commit into
SheffieldML:mainfrom
dexhunter:perf/tdot-gram-memory
Nov 16, 2026
Merged

lawrennd merged 1 commit into
SheffieldML:mainfrom
dexhunter:perf/tdot-gram-memory

Conversation

@dexhunter

Copy link
Copy Markdown
Contributor

TL;DR

  • Reduce peak memory in tdot_blas when building dense Gram matrices.
  • Use one Fortran-order output buffer and return its symmetric transpose without copying it.
  • A 3,600-by-8 call uses 207,591,048 → 103,910,952 bytes, saving 103,680,096 bytes (49.9%).
  • Measure peak allocated bytes with tracemalloc, starting after input creation.
  • Both required native tests pass; Ruff lint and format checks pass.

What & Why

The old code builds a C-order output, then BLAS copies it to Fortran order.
Returning a C-order result adds another full matrix copy.

The change starts with a Fortran-order buffer and reuses the shared symmetry helper.
Its transpose has the same symmetric values and C-order layout, so returning it avoids another allocation.

Validation

Peak allocated memory for one Gram computation, with five repeats per situation:

Situation Before After Change
1,800 observations, 8 features, float64 51,955,848 bytes 26,035,752 bytes 25,920,096 fewer bytes (49.9%)
3,600 observations, 8 features, float64 207,591,048 bytes 103,910,952 bytes 103,680,096 fewer bytes (49.9%)
1,800 observations, 8 features, float32 12,960,280 bytes 12,960,280 bytes Unchanged

The complete allocation workload also checks small arrays, empty dimensions, strided inputs, supplied buffers, and fallback types.
Three runs produced these statistics:

Statistic Value
Old -> new (mean) 198.0 MiB -> 99.1 MiB (49.9% less peak memory)
95% CI of the new/old ratio (bootstrap, 10,000 resamples) 0.501–0.501 (49.9–49.9% less peak memory)
Mean ± SD of the change per input -98.9 MiB ± 0 bytes
95% CI of the mean change (t-interval) -98.9 MiB to -98.9 MiB
Number of fixed inputs (n) 15 (3 runs)

The 15 observations repeat the same five input seeds across three runs.
The complete workload retains the small validation results, so its allocation totals differ slightly from individual calls.
All normalized output bytes, layouts, mutations, aliases, and exceptions matched the original implementation.

Native validation: 2 passed, with one existing plotting deprecation warning.

python3 -m pytest GPy/testing/test_util.py::TestUtil::test_checkFullRank GPy/testing/test_kernel.py::TestKernelGradientContinuous::test_Prod1 -q --tb=line
ruff check GPy
ruff format --check GPy

To reproduce a single-call allocation measurement on each revision, run:

OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 python3 - <<'PYTHON'
import gc
import tracemalloc
import numpy as np
from GPy.util.linalg import tdot_blas

mat = np.ascontiguousarray(np.random.RandomState(0).randn(3600, 8))
gc.collect()
tracemalloc.start()
out = tdot_blas(mat)
print(tracemalloc.get_traced_memory()[1], "peak bytes")
tracemalloc.stop()
assert out.shape == (3600, 3600)
PYTHON

Inputs exist before tracing starts. Measurements use Linux, Python 3.12.13, NumPy 2.5.4, SciPy 1.18.1 BLAS, and one BLAS thread.
Peak allocation is measured with tracemalloc; it is not process resident memory.

Why this is safe and easy to maintain

The small change keeps the existing shared symmetry helper and adds no net code lines.
It preserves numerical results, output layout, and supplied-buffer behavior.
The full-rank and product-kernel tests cover the production callers.
The helper keeps its existing native and NumPy implementations.

AI assistance

This PR was prepared with AI assistance from Aiden. Candidates were tracked with Weco Observe.

Weco share link: https://dashboard.weco.ai/share/UZdek2QWIzbIY-gluzhnKnbGDUj_cb6n

Use a Fortran output buffer and return its symmetric transpose.
Reuse symmetrify and let BLAS handle input conversion.
Save 103,680,096 peak allocated bytes for a 3600-by-8 design.
Preserve supplied-output behavior and fallback types.

Co-Authored-By: Aiden <[email protected]>
@codecov

codecov Bot commented Nov 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.15%. Comparing base (b616c84) to head (06a7d8e).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1301   +/-   ##
=======================================
  Coverage   62.15%   62.15%           
=======================================
  Files         206      206           
  Lines       21955    21955           
  Branches     2564     2564           
=======================================
  Hits        13646    13646           
  Misses       7682     7682           
  Partials      627      627           
Flag Coverage Δ
unittests 62.15% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@lawrennd lawrennd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lawrennd

Copy link
Copy Markdown
Member

Thanks Dex, some significant memory improvements here.

@lawrennd
lawrennd merged commit d55a4d9 into SheffieldML:main Nov 16, 2026
19 checks passed

@lawrennd lawrennd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lawrennd

Copy link
Copy Markdown
Member

Thanks Dex, again significant memory improvements!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants