Skip to content

perf(markov): normalize tmatrix_cov in place -- 49% less peak memory - #347

Merged
clonker merged 1 commit into
deeptime-ml:mainfrom
dexhunter:perf/tmatrix-cov-in-place-normalize
Nov 13, 2026
Merged

clonker merged 1 commit into
deeptime-ml:mainfrom
dexhunter:perf/tmatrix-cov-in-place-normalize

Conversation

@dexhunter

Copy link
Copy Markdown
Contributor

TL;DR

  • This PR lowers the peak memory of tmatrix_cov in markov/tools/estimation/dense/covariance.py when it returns the full covariance tensor.
  • It divides the tensor by the norm in place, so the function holds one (M, M, M) array, not two.
  • Benchmark: tracemalloc peak of one call on a 200 x 200 count matrix, 128.33 MB -> 65.33 MB, 49.1% less.
  • python -m pytest tests/markov/tools/estimation/impl/dense/covariance_test.py::TestCovariance::test_tmatrix_cov -q --tb=short passes (1 passed).

What & Why

Before, the full-tensor branch built the non-normalized tensor Z and then ran cov = Z / norm[:, None, None]. That division made a second (M, M, M) array while Z was still alive. For M = 200 each array is 64 MB, so the call needed about 128 MB.

Now the code runs Z /= norm[:, None, None] and returns Z. Z is a new array that only this function owns, and it always has a float dtype because alpha = C + 1.0. The division is the same operation on the same values, so the output stays identical. The count matrix C is not changed.

Results

Situation Before After Change
Full tensor, M = 10 27.5 kB 27.5 kB no change
Full tensor, M = 50 2.08 MB 1.16 MB 44% less
Full tensor, M = 100 16.08 MB 8.37 MB 48% less
Full tensor, M = 200 128.33 MB 65.09 MB 49% less
Full tensor, M = 300 432.73 MB 218.30 MB 50% less
One row (k=7), M = 200 0.64 MB 0.64 MB no change

Each row is the tracemalloc peak of one call on a random dense float64 count matrix. The best time of five calls for M = 200 went from 14.4 ms to 10.2 ms. The one-row call uses dirichlet_covariance, which this PR does not touch.

Why this is safe and easy to maintain

The change is two lines in one function and adds no new code path. The returned tensor has the same values, shape and dtype as before. The test test_tmatrix_cov in tests/markov/tools/estimation/impl/dense/covariance_test.py compares the full tensor and a single row with reference values.

Validation

  • Native tests: the command in the TL;DR gives 1 passed. The full file covariance_test.py also passes (4 passed).
  • Benchmark: one full-tensor call on a 200 x 200 float64 count matrix, with the result kept alive. Three repeated measurements gave 65.33 MB each, against 128.33 MB for the unchanged code.
  • The benchmark also compared every element of the tensor with the Dirichlet closed form to 1e-10. It checked integer, float32, Fortran-ordered, sparse and non-square counts against the unchanged code too.
  • Reproduce one row of the table:
import numpy as np, tracemalloc
from deeptime.markov.tools.estimation import tmatrix_cov
C = np.random.rand(200, 200)
tracemalloc.start()
cov = tmatrix_cov(C)
print(tracemalloc.get_traced_memory()[1] / 1e6, "MB")

Checklist

  • Make sure to include one or more tests for your change (the existing test_tmatrix_cov already covers this function; no new test added)
  • Add yourself to AUTHORS

AI assistance

This PR was prepared with AI assistance from Aiden. Candidates were tracked with Weco Observe.

Weco share link: https://dashboard.weco.ai/share/UBiNlGOILh2wSMbn76AW3iUVKTEl5RkR

tmatrix_cov built the non-normalized (M, M, M) tensor Z and then made a
second (M, M, M) array with cov = Z / norm. Divide Z by the norm in
place and return it, so only one tensor is alive at the peak.

tracemalloc peak for a 200 x 200 count matrix: 128.33 MB -> 65.33 MB
(49.1% less). Values, shape and dtype of the result are unchanged.

Co-Authored-By: Aiden <aiden@weco.ai>
@codecov

codecov Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.03%. Comparing base (b1b7006) to head (bcf405d).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #347   +/-   ##
=======================================
  Coverage   94.03%   94.03%           
=======================================
  Files         141      141           
  Lines       11199    11199           
=======================================
  Hits        10531    10531           
  Misses        668      668           
Flag Coverage Δ
decomposition 48.85% <0.00%> (ø)
markov 69.18% <100.00%> (ø)
misc 65.70% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@clonker
clonker merged commit 38ec7a8 into deeptime-ml:main Nov 13, 2026
76 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants