Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/build-and-test/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ pip install -e ".[dev]" ; echo "exit: $?"
run alone, `pip install -e ".[test]"` is enough and much smaller.

An editable install puts the Python sources on the path, so an edit under
`src/blochsim` takes effect on the next import. The two C++ extensions are
`src/blochsim` takes effect on the next import. The C++ extensions are
compiled artifacts and do not: re-run the install after editing any `.cpp`,
`.hpp` or `.cu`.

Expand Down
2 changes: 1 addition & 1 deletion .claude/skills/cut-a-release/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ touches `pyproject.toml`, `CMakeLists.txt`, the `.cpp` sources, or the workflow.

## What gets built, and why there is so little of it

Both kernels are plain CPython extensions against the **stable ABI** from 3.10
The CPU kernels are plain CPython extensions against the **stable ABI** from 3.10
on. They call no PyTorch API and link no PyTorch library. So:

- one `cp310-abi3` wheel per platform serves every supported interpreter —
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ carried out under them.
| `src/blochsim/sequence/` | The description an acquisition is assembled from — events, operators, builders — and the dispatch that turns one into a kernel launch. |
| `src/blochsim/_model/` | What a signal model *is*: the physics, the simulator that orders its events, and the binding that resolves a protocol's structure once and rebinds its values per call. |
| `src/blochsim/_epg_cpu.cpp`, `_perk_cpu.cpp` | The CPU kernels. Every path the GPU has exists here too, and the two agree to float32 round-off. |
| `src/blochsim/_trains_cpu.cpp`, `trains/` | Refocusing-train design: the prescribed-signal inverse of a CPMG train, in double precision on the host only, over NumPy with no PyTorch import. It shares the real-subspace operators of `_real_subspace.hpp` with `_epg_cpu.cpp`. |
| `src/blochsim/_epg_kernels.hpp`, `_pools_kernels.hpp`, `_perk_kernels.hpp` | The GPU kernels, written over the tiles of `_tile.hpp`. `_kernels.hpp` is the table a launch reads: each kernel's parameters, and which of them size the block. |
| `src/blochsim/_gpu.cu`, `_gpu_kernel.cu.in`, `_gpu_host.cpp`, `_gpu_launch.py` | The same kernels compiled ahead of time for the card (`_gpu`), and for the host (`_gpu_host`), which is how the suite checks them without one; and the launcher both answer to. |
| `src/blochsim/sequence/_epg_gpu.py`, `_pools_gpu.py`, `estimators/_perk_gpu.py` | What a GPU launch is given: the tiling and the arguments. |
Expand Down
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@

### Added

- **Refocusing trains from a prescribed echo signal.** `blochsim.trains` is
new: `prescribed_flips` inverts the phase graph of a CPMG train one echo at
a time, with or without relaxation; `busse_flips` designs the
variable-flip-angle trains of long 3D fast spin echo readouts (Busse 2006,
2008) from a lowest, a centre and a highest angle; `busse_centre_echo`
places the centre echo for a contrast-equivalent echo time, trying every
centre in one batch; `effective_te` gives that echo time for any train.
They run in a new double-precision C++ extension (`_trains_cpu`) on the
host, take and return NumPy arrays, and solve a batch of trains across
threads.
- **Coarse-to-fine dictionary search.** `DictionaryMatcher(coarse=k)` places
the atoms on the lattice their parameters were simulated on, scores every
k-th value along each parameter, then the `2k + 1` values around the winner
Expand All @@ -28,6 +38,14 @@

### Changed

- **`import blochsim` no longer imports PyTorch.** It imports
`blochsim.trains` alone; the simulators, estimators and design tools are
imported the first time any of them is asked for, by attribute, by
`from blochsim import ...` or by submodule. `import blochsim.trains` costs
what NumPy costs.
- **A forked child starts its own worker pool.** The CPU kernels' pool is
tied to the process that created it, so a process forked after a threaded
call no longer waits on workers it does not have.
- **Every GPU, a bounded share of it, by default.** `execution="auto"` is the
default of a simulator, an estimator's `map` and a `ModelOperator`. A call
too small to repay a launch, or a simulator call carrying a gradient, runs
Expand Down
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ set(BLOCHSIM_CUDA_PACKAGE "" CACHE STRING
if(NOT BLOCHSIM_CUDA_PACKAGE)
blochsim_add_kernel(_epg_cpu src/blochsim/_epg_cpu.cpp)
blochsim_add_kernel(_perk_cpu src/blochsim/_perk_cpu.cpp)
blochsim_add_kernel(_trains_cpu src/blochsim/_trains_cpu.cpp)
endif()

# The GPU kernels compiled for the host, one program at a time. Nothing in the
Expand Down
2 changes: 2 additions & 0 deletions docs/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ the per-voxel nonlinear least-squares fit, are bartorch's.
| {doc}`sequence` | `blochsim` | Operators, sequence descriptions, RF definitions |
| {doc}`estimators` | `blochsim.estimators` | Dictionary matching, lookup tables, PERK |
| {doc}`optim` | `blochsim.optim` | Sequence parameter optimization |
| {doc}`trains` | `blochsim.trains` | Refocusing trains from a prescribed echo signal |

```{toctree}
:hidden:
Expand All @@ -22,4 +23,5 @@ simulators
sequence
estimators
optim
trains
```
25 changes: 25 additions & 0 deletions docs/api/trains.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Refocusing trains

```{eval-rst}
.. currentmodule:: blochsim.trains
```

Refocusing flip angles designed from the echo signal they are to give. The
functions take and return NumPy arrays in degrees and ms, run on the host in
double precision, and do not import PyTorch: `import blochsim.trains` costs
what NumPy costs.

{func}`prescribed_flips` inverts the extended phase graph of a CPMG train one
echo at a time: the flip angle of each pulse is the smallest whose echo has the
prescribed amplitude, with or without relaxation. {func}`busse_flips` builds
on it the variable-flip-angle trains of long 3D fast spin echo readouts, and
{func}`busse_centre_echo` places their centre echo for a contrast-equivalent
echo time, which {func}`effective_te` computes for any train. A batch of
trains is solved in one call, across threads.

| Object | Description |
| --- | --- |
| {obj}`~blochsim.trains.prescribed_flips` | Flip angles whose echoes take prescribed amplitudes, and the echoes they give. |
| {obj}`~blochsim.trains.effective_te` | Contrast-equivalent echo time of every echo of a train. |
| {obj}`~blochsim.trains.busse_flips` | Variable flip angles with a prescribed lowest, centre and highest angle. |
| {obj}`~blochsim.trains.busse_centre_echo` | Centre echo whose contrast-equivalent echo time is nearest a target. |
3 changes: 2 additions & 1 deletion docs/developer-guide/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Besides the environment and PyTorch build of the
{doc}`user guide <../user-guide/index>`, you need:

- **A C++17 compiler and CMake.** The CPU kernels (`src/blochsim/_epg_cpu.cpp`,
`_perk_cpu.cpp`) are compiled at install, and there is no Python fallback.
`_perk_cpu.cpp`, `_trains_cpu.cpp`) are compiled at install, and there is no Python fallback.

::::{tab-set}

Expand Down Expand Up @@ -76,6 +76,7 @@ python -c "import blochsim._epg_cpu as k; print(k.__file__)"
| `src/blochsim/sequence/` | Events, operators and builders, and the dispatch to the kernels. `_parameters.py` is the parameter ABI the Python dispatch, the C++ extension and the GPU kernels share. |
| `src/blochsim/_model/` | The signal model: physics, the simulator that orders events, and the binding that resolves a protocol's structure once. |
| `src/blochsim/_epg_cpu.cpp`, `_perk_cpu.cpp` | The CPU kernels. Every GPU path exists here too; the two agree to float32 round-off. |
| `src/blochsim/_trains_cpu.cpp`, `trains/` | The train-design kernels, in double precision on the host only, and their NumPy interface. |
| `src/blochsim/_epg_kernels.hpp`, `_pools_kernels.hpp` | The GPU kernels. |
| `src/blochsim/_operator.py`, `_subspace.py` | The model operator and the temporal basis a reconstruction composes with. |
| `src/blochsim/simulators/`, `estimators/`, `optim/` | Shipped sequences, and what is built on them. |
Expand Down
6 changes: 6 additions & 0 deletions docs/developer-guide/internals/kernels.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,12 @@ specialization and the pool models -- and the two agree to float32 round-off.
On Linux the same CUDA kernels are also compiled for the host (`_gpu_host`),
which is how the suite checks them without a card.

The train-design kernels (`_trains_cpu.cpp`, behind `blochsim.trains`) are a
third extension on the same worker pool. They invert a CPMG train for a
prescribed echo signal in double precision, on the host only, and share the
real-subspace shift and rotation with the simulation kernels
(`_real_subspace.hpp`), which use them at float.

### Layouts and compiles

The EPG kernels on a card are written for their layouts (`_layout.hpp`). A
Expand Down
4 changes: 2 additions & 2 deletions scripts/check_wheel.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,8 @@ def main() -> int:
if root is None:
raise SystemExit("blochsim is not installed")
kernels = extensions(root, "_*_cpu.*")
if len(kernels) != 2:
raise SystemExit(f"expected two kernels in {root}, found {kernels}")
if len(kernels) != 3:
raise SystemExit(f"expected three kernels in {root}, found {kernels}")
for path in kernels:
load(path, "blochsim", LARGEST_REASONABLE_BYTES)
# A build from source keeps the card's module beside the package.
Expand Down
110 changes: 96 additions & 14 deletions src/blochsim/__init__.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,102 @@
"""Main BlochSim API."""
"""Main BlochSim API.

__all__ = ["ModelOperator", "Simulator", "Subspace"]
The simulators, the estimators and the design tools are built on PyTorch,
and importing it takes far longer than anything :mod:`blochsim.trains` does.
So ``import blochsim`` imports :mod:`blochsim.trains` alone, and the rest of
the package the first time anything else is asked of it.
"""

from . import estimators # noqa
from . import optim # noqa
from . import sequence # noqa
from . import simulators # noqa
from __future__ import annotations

from ._model import Simulator # noqa
from ._operator import ModelOperator # noqa
from ._subspace import Subspace # noqa
from .sequence import * # noqa
import threading
from importlib import import_module
from importlib.util import find_spec
from typing import TYPE_CHECKING, Any

__all__.extend(sequence.__all__)
from . import trains # noqa: F401

from . import _functional # noqa
from ._functional import * # noqa
if TYPE_CHECKING:
from . import ( # noqa: F401
_functional,
estimators,
optim,
sequence,
simulators,
)
from ._functional import * # noqa: F403
from ._model import Simulator # noqa: F401
from ._operator import ModelOperator # noqa: F401
from ._subspace import Subspace # noqa: F401
from .sequence import * # noqa: F403

__all__.extend(_functional.__all__)
#: Set on the thread that is importing the rest of the package, while it does.
_loading = threading.local()
_loaded = False


def _load() -> None:
"""Import everything built on PyTorch, and name it here.

No lock is held: the import system serializes each module's import and
detects a cycle between threads, which a lock of this module's own would
hide from it. Two threads arriving together both run this, to the same
end.
"""
global _loaded
_loading.here = True
try:
from . import _functional, estimators, optim, sequence, simulators
from ._model import Simulator
from ._operator import ModelOperator
from ._subspace import Subspace

namespace = globals()
namespace.update(
estimators=estimators,
optim=optim,
sequence=sequence,
simulators=simulators,
_functional=_functional,
Simulator=Simulator,
ModelOperator=ModelOperator,
Subspace=Subspace,
)
names = ["ModelOperator", "Simulator", "Subspace"]
for module in (sequence, _functional):
for name in module.__all__:
namespace[name] = getattr(module, name)
names.extend(module.__all__)
namespace["__all__"] = names
_loaded = True
finally:
_loading.here = False


def __getattr__(name: str) -> Any:
missing = AttributeError(f"module {__name__!r} has no attribute {name!r}")
if name.startswith("__") and name != "__all__":
raise missing
# A submodule that imports a sibling while the package loads is asking
# the import system, and an AttributeError is what sends it there.
if _loaded or getattr(_loading, "here", False):
raise missing
if name != "__all__":
# A submodule is the import system's to import.
if find_spec(f"{__name__}.{name}") is not None:
return import_module(f"{__name__}.{name}")
# Everything the package exports is public. A private name that is
# no submodule is an optional module that was not built, asked for
# by ``from blochsim import``: it is missing, and loading the
# package from inside another module's import would not find it.
if name.startswith("_"):
raise missing
_load()
try:
return globals()[name]
except KeyError:
raise missing from None


def __dir__() -> list[str]:
_load()
return sorted(globals())
44 changes: 9 additions & 35 deletions src/blochsim/_epg_cpu.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,9 @@
#endif

#include "_ifunc.hpp"
#include "_real_subspace.hpp"
#include "_threads.hpp"

// MSVC rejects the GNU spelling of both of these outright, and the kernel is
// built by whichever compiler the platform ships.
#if defined(_MSC_VER)
#define BLOCHSIM_ALWAYS_INLINE __forceinline
#define BLOCHSIM_RESTRICT __restrict
#else
#define BLOCHSIM_ALWAYS_INLINE __attribute__((always_inline)) inline
#define BLOCHSIM_RESTRICT __restrict__
#endif

namespace {

constexpr std::uint8_t PRE_SHIFT = 1;
Expand Down Expand Up @@ -3010,20 +3001,10 @@ __attribute__((target_clones("default", "sse4.2", "avx2", "avx512f")))
inline void shift_real(
std::vector<float>& plus, std::vector<float>& minus, const std::size_t states
) {
// No states is impossible -- every entry point rejects a state count
// below one -- but not provable here, and the loops below count down
// from ``states - 1``.
if (states == 0) {
return;
}
for (std::size_t state = 0; state + 1 < states; ++state) {
minus[state] = minus[state + 1];
}
minus[states - 1] = 0.0F;
for (std::size_t state = states - 1; state > 0; --state) {
plus[state] = plus[state - 1];
}
plus[0] = -minus[0];
blochsim::real_subspace::shift(
plus.data(), minus.data(), states,
blochsim::real_subspace::ACROSS_AXIS<float>
);
}

// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -3096,17 +3077,10 @@ void simulate_real_range(
const float alpha = view.flip[event] * b1;
const float cosine = std::cos(alpha);
const float sine = std::sin(alpha);
const float cosine_half_sq = 0.5F * (1.0F + cosine);
const float sine_half_sq = 0.5F * (1.0F - cosine);
const float half_sine = 0.5F * sine;
for (std::size_t state = 0; state < states; ++state) {
const float p = plus[state];
const float m = minus[state];
const float z = longitudinal[state];
plus[state] = cosine_half_sq * p + sine_half_sq * m - sine * z;
minus[state] = sine_half_sq * p + cosine_half_sq * m + sine * z;
longitudinal[state] = half_sine * p - half_sine * m + cosine * z;
}
blochsim::real_subspace::rotate(
plus.data(), minus.data(), longitudinal.data(), states,
cosine, sine
);
}
} else if (buffers.kind[event] == 2 && (action & RECORD) != 0) {
const std::int64_t index =
Expand Down
Loading
Loading