Skip to content

perf(ims): sum each row in a local variable in lusol - #3146

Open
jdhughes-dev wants to merge 2 commits into
MODFLOW-ORG:developfrom
jdhughes-dev:ilu-lusol-accumulate
Open

jdhughes-dev wants to merge 2 commits into
MODFLOW-ORG:developfrom
jdhughes-dev:ilu-lusol-accumulate

Conversation

@jdhughes-dev

@jdhughes-dev jdhughes-dev commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

lusol, which applies the ILUT and MILUT preconditioners, summed each row of the forward and backward solves in x(i), which the compiler stored and reloaded for every term. Each row is now summed in a local variable, which makes the linear solve 39 to 45 percent faster per inner iteration with COMPLEXITY COMPLEX, with results unchanged to the bit.

Checklist of items for pull request

  • Replaced section above with description of pull request
  • Referenced issue or pull request Faster application of the ILU preconditioners #3145
  • Added new test or modified an existing test
  • Formatted new and modified Fortran source files with fprettify
  • Added a release note to doc/ReleaseNotes/items/ with a plain-language description of the bug fix, change, or feature; required for changes that may affect users
  • Removed checklist items not relevant to this pull request

For additional information see instructions for contributing and instructions for developing.

@jdhughes-dev

Copy link
Copy Markdown
Contributor Author

Reviewer Summary

  • Change: the forward and backward solves in lusol (src/Utilities/Libraries/sparskit2/ilut.f90) sum each row in a local t and store x(i) once. Because x(i) can alias x(jlu(k)), the old code stored and reloaded x(i) for every term. The order of operations is unchanged, and the in-place call documented in the routine (x and y the same array) still works, since y(i) is read before x(i) is written.
  • Bit-identical results: in an autotest subset of 18 test files, 15 of which use COMPLEX or PRECONDITIONER_LEVELS (94 tests passed), every model using ILUT or MILUT was rerun with the fix(ims): retry the ILU0 factorization after a bad pivot #3144 binary: 49 completed runs wrote 99 binary head, concentration, or budget files, and all are identical to the byte.
  • Timing (from Faster application of the ILU preconditioners #3145): linear solve time per inner iteration with COMPLEXITY COMPLEX, BICGSTAB, equal inner iteration counts, macOS arm64, gfortran 13: 7.24 to 4.40 ms for 120,000 cells and 31.2 to 17.1 ms for 480,000 cells. Run to convergence, the 120,000-cell model takes 2.14 s of linear solve time instead of 3.30 s, with the same iteration counts and heads.
  • Unit tests: lusol_apply_exact factors a nonsymmetric tridiagonal matrix exactly with ILUT and MILUT and checks that lusol returns the solution; lusol_bit_identical factors a nonsymmetric 4 by 4 grid with fill dropped and requires lusol to match a copy of the original algorithm to the bit, also when called in place. Leaving out a term or the local sum fails both tests; reversing either sum, which only changes roundoff, fails lusol_bit_identical.
  • Parallel runs: the default PETSc preconditioner is block Jacobi with the IMS preconditioner on each block, which applies ILUT and MILUT with lusol, so parallel runs get the same speedup.
  • ILU0 and MILU0: not changed here; the ILU0 part of Faster application of the ILU preconditioners #3145 changes results at roundoff level and will be a separate pull request.

@mjr-deltares mjr-deltares left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

lusol, which applies the ILUT and MILUT preconditioners, accumulated
each row of the forward and backward solves in x(i). Because x(i) can
alias x(jlu(k)), the compiler stored and reloaded x(i) for every term.
Each row is now summed in a local variable and stored once. The order of
operations is unchanged, so results are bit-identical, and the routine
can still be called in place.

Adds unit tests that lusol solves a tridiagonal system that ILUT and
MILUT factor exactly, and that it matches the original algorithm to the
bit for an inexact factorization, including in place.

Refs MODFLOW-ORG#3145
A bare underscore in PRECONDITIONER_LEVELS stops the LaTeX release
notes build.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants