GML part M5: layers and architectures share one method per Q/P pair, Bool choice and triangle

CHANGELOG.md

@@ -55,6 +55,8 @@ breaking release).
 
 - **`LowerTriangular` and `UpperTriangular` are renamed `StrictlyLowerTriangular` and `StrictlyUpperTriangular`**, as in GeometricOptimizers 0.9, and are re-exported under the new names. A file saved with the old names no longer loads through the registry. The rigid-body tutorial registers the two old names for its committed files.
 
+- **`ResNetLayer` encodes its bias as a `Symbol` type parameter, the two volume-preserving feedforward layers are one struct, and two fields and a keyword are renamed.** `ResNetLayer`'s third type parameter is `:bias` or `:no_bias`, as in the volume-preserving feedforward layers, where it was `true` or `false`; its keyword `use_bias::Bool` stays. `VolumePreservingFeedForwardLayer{M, N, bias, AT, Triangle}` is now the one struct of the two volume-preserving feedforward layers, with `Triangle` `:lower` or `:upper`, where it was their abstract supertype; `VolumePreservingLowerLayer{M, N, bias, AT}` and `VolumePreservingUpperLayer{M, N, bias, AT}` are its aliases and keep their constructors. Their `initialparameters` takes the bias initializer as the keyword `init_bias`, as `ResNetLayer` and the gradient layers do, where it took `init_bias!`. `GSympNet` stores its activation in the field `activation`, as `LASympNet` does, where it was `act`. `StandardTransformerIntegrator`'s field `upsacling_activation` is spelled `upscaling_activation`. `GradientLayer` stores `second_dim` as an `Int`, and the six SympNet layer constructors (`GradientLayerQ`/`P`, `ActivationLayerQ`/`P`, `LinearLayerQ`/`P`) convert the dimension to an `Int`, so `GradientLayerQ(Int32(4), Int32(8), tanh) === GradientLayerQ(4, 8, tanh)` and `ActivationLayerQ(Int32(4), tanh) === ActivationLayerQ(4, tanh)`.
+
 - **The `[compat]` bounds move to the versions the training API needs**: GeometricOptimizers `"0.9"`, AbstractNeuralNetworks `"0.9.2"`, NeuralNetworkParameters `"0.5"`, SymbolicNeuralNetworks `"0.9"`, GeometricBase `"0.15"`, GeometricEquations `"0.21.5"` and GeometricSolutions `"0.6.6"`. HDF5 leaves `[compat]` and `[weakdeps]`.
 
 ### Fixed
@@ -81,8 +83,16 @@ breaking release).
 
 - **A `DataLoader`'s two output fields are concrete, its minibatch index set is concrete and draws every sample, and a loader of one sample batches.** `output_dim` and `output_time_steps` were `Union{Int, Nothing}` whatever the loader held, which made a kernel launch read a value of no single type (issue #211); the copies of a solution and of an ensemble both go through one helper, `data_tensor`, which builds a 1-based tensor, and the batch indices are one host-built `2 × n` matrix copied to the backend once, with no `@views` (issue #212, which asked for `@views`). A `DataLoader` of one sample drew no minibatch (known issue K8) and a loader of matrices never drew its last sample (known issue K12); both now draw every pair, and the count of a trajectory is `cld(n · n_params, batch_size)`, so a seeded run draws the minibatches it drew before, except the ones the last sample joins. The `JET.report_opt` line of `test/quality/jet.jl` for a launcher that reads a `Union{Int, Nothing}` field is a `@test` rather than a `@test_broken`: the batch half of issue #325 is fixed here. This closes known issues B10, C17, K8 and K12, which leave `KNOWN_ISSUES.md`. B10's `DataLoader(::EnsembleSolution{T, T1, Vector{ST}})` for a `dataser` of `(t, q, q̇)` is reachable — a rigid-body `odeensemble` produces one, and the tutorial and three scripts call it — and `test/data_loader/data_loader.jl` tests it, so the method stays.
 
+- **`LinearSymplecticTransformer` keeps the activation it is given.** Its constructor built `new{typeof(tanh)}`, so `LinearSymplecticTransformer(4, 3; activation = sin)` raised a `MethodError`; the gradient layers of its chain now carry `sin`. The test is `test/architectures/architecture_structure.jl`.
+
+- **`PositionalEncoding` allocates its encoding on the backend of its input.** The layer built a host `Matrix` and added it to its input, which a GPU kernel cannot read. It now fills `similar(x, eltype(x), dim, seq_length)` by one broadcast over the indices, so the layer runs on a `JLArray` with scalar indexing disallowed, and the matrix it adds is a `JLArray` there; `positional_encoding(T, dim, seq_length)` fills a host `Matrix` the same way, and the layer's output matches that of `4c4a8320` at 16 ULPs of the largest magnitude (`test/layers/layer_pins.jl`). The "CPU only" warnings of the `PositionalEncoding` docstring and of the `positional_encoding` keyword of `Transformer` are gone. This closes known issues B11 and K5, which leave `KNOWN_ISSUES.md`. The test is `test/layers/positional_encoding.jl`.
+
 - **`assign_output_estimate` is a crop of the last `prediction_window` columns, and its gradient is Zygote's `getindex` adjoint.** Its kernel, `augment_zeros` and the `rrule` are gone: the adjoint is zero on the cropped columns, runs on a device with scalar indexing off, and gives a `Float32` cotangent for a `Float64` seed on `Float32` data, where `augment_zeros` returned the seed's type. The docstring's formula starts at column `T − pw + 1`, which is what the code does.
 
+- **The decoder of an `AutoEncoder` stores the decoder's block count in `n_decoder_blocks`, where it stored the encoder's.** `decoder` passed `arch.n_encoder_blocks` to `UnknownDecoder` and `UnknownSymplecticDecoder`, so an `AutoEncoder` with two encoder blocks and three decoder blocks had the decoder architecture `UnknownDecoder(6, 2, 2)`; it is now `UnknownDecoder(6, 2, 3)`. The decoder's model and parameters already used the decoder's count, so no output changes, and the stored field changes only where the two counts differ. This affects every `AutoEncoder` that is not a `SymplecticAutoencoder`, `PSDArch` included. This closes known issue K24, which leaves `KNOWN_ISSUES.md`. The test is `test/architectures/architecture_pins.jl`, testset "encoder and decoder types of each autoencoder family".
+
+- **A `(q, p)` input whose `p` has the wrong size raises the layer's own `ErrorException("Dimension mismatch.")`.** The forward methods of `ActivationLayer`, `GradientLayer` and `LinearLayer` checked the size of `q` and not the size of `p`, so a `p` of the wrong size raised whichever error its first operation produced: a `DimensionMismatch` from a matrix product or a broadcast, or an `AssertionError` from the `SymmetricMatrix` product. Each method now checks `q` and `p` together. No valid input changes. This closes known issue K23, which leaves `KNOWN_ISSUES.md`. The test is `test/layers/layer_structure.jl`, testset "a SympNet layer raises on a half of the wrong size".
+
 ### Added
 
 - **`KernelAbstractions.get_backend` is defined for `PoissonTensor`.** GeometricOptimizers 0.9 asks every operand of a checked product for its backend, so `𝕁 * Y` with a `StiefelManifold` raised `ArgumentError: Implement KernelAbstractions.get_backend(::PoissonTensor...)` without it.
@@ -143,6 +153,8 @@ breaking release).
   | `tensor_mat_skew_sym_assign` pullback (C15) | Float32 | — | 8365.5 → 8315.0 |
   | `tensor_mat_skew_sym_assign` pullback (C15) | Float64 | — | 8387.2 → 8354.8 |
 
+- **A pair of layers or architectures that differed in one choice shares one method, and the outputs do not change.** One function, `_shift(Val(C), z, f)`, adds `f` of one half of a `(q, p)` input to the other half, so `src/layers/sympnets.jl` has three forward methods on a `(q, p)` input where it had six, `src/layers/symplectic_attention.jl` two where it had four, and `src/layers/linear_symplectic_attention.jl` one where it had two. Each `Bool` keyword of `VolumePreservingAttention`, the two volume-preserving feedforward layers, `SymplecticAttentionQ`/`SymplecticAttentionP` and `SymplecticAutoencoder` picks its type parameter in one expression. The two volume-preserving feedforward layers have one `initialparameters` where they had four. `MultiHeadAttention` has one `initialparameters` for both weight kinds, and it draws in the same order as before: head by head, and in each head the query, the key and the value projection, so no seeded value moves. Its forward pass still finds each head by its name `head_i`, but takes the names from a tuple that is built once for each number of heads, where it built `Symbol("head_", i)` for every head on every pass; at `4c4a8320`, 1000 passes of a two-head layer allocated 384 000 B more than the same products with constant keys (Julia 1.13.2). `LASympNet` has one `Chain` method where it had four; one `_gradient_layers` builds the alternating gradient layers of `GSympNet`, `LinearSymplecticTransformer` and `SymplecticTransformer`, and replaces the two methods of `_make_block_for_initialization`; the `DataLoader` constructors of `LASympNet`, `GSympNet` and `ClassificationTransformer` call the dimension constructors; and one `encoder` and one `decoder` serve every `AutoEncoder` that is not a `SymplecticAutoencoder`. Every type argument under `src/layers/` is spelled `::Type{T}`, and the aliases `LinearLayerQ` and `LinearLayerP` no longer name an unused third parameter, which Julia drops from an alias anyway. The outputs of 27 layer cases, 13 architecture cases and the encoders and decoders of two autoencoders, each in `Float32` and `Float64`, are compared at 16 ULPs of the largest magnitude of the pinned vector with the outputs that `4c4a8320` gives for the same seeds and inputs, in `test/layers/layer_pins.jl` and `test/architectures/architecture_pins.jl`; `test/architectures/forward_inference.jl` checks that the forward pass of every exported architecture with a chain infers, `StandardTransformerIntegrator(32)` with its default of 32 heads among them. This closes known issue C19, which leaves `KNOWN_ISSUES.md`.
+
 - **The kernels that zero their output are gone, and `tensor_mat_skew_sym_assign`'s pullback allocates without zeroing.** After the deletions in the entry above, the only zeroed buffers left under `src/kernels/kernel_ad_routines/` were the two of `tensor_mat_skew_sym_assign`'s pullback. Both kernels assign every element once over `ndrange = size`, so both now use `similar` or `KernelAbstractions.allocate`, and no accumulating rule remains. This closes known issue C15, which leaves `KNOWN_ISSUES.md`.
 
 ### Fixed

KNOWN_ISSUES.md

@@ -2,28 +2,6 @@
 
 What is known to be wrong in GeometricMachineLearning and is not fixed.
 
-## B. Known defects
-
-### B11 · `PositionalEncoding` is CPU-only, and the failure is a compile error rather than a slowdown.
-
-- **location:** —
-- **kind:** defect
-- **found:** 2026-09-18
-- **evidence:**
-  `positional_encoding` builds its matrix with `Matrix{T}(undef, …)`, so the layer's
-  `x .+ P` adds a host array to whatever it is given. Every other layer allocates through the
-  backend — `KernelAbstractions.allocate`, or `similar(x, …)` in the forward pass. Broadcasting a
-  host `Matrix` against a device array does not fall back to the CPU; it fails to compile, because
-  the host array cannot be read from a kernel.
-
-  **No test can catch this, because the suite has no GPU test.** That is what makes it worth an
-  entry rather than a docstring alone: a green matrix says nothing about it. Both docstrings — the
-  layer's and the `positional_encoding` keyword of [`Transformer`](@ref) — carry the warning.
-
-  Closing it means giving the builder the backend, so that it allocates through
-  `KernelAbstractions.allocate` and fills with a kernel instead of a loop, and it cannot be verified
-  here without a GPU job to run it under.
-
 ## C. Follow-ups and cleanups
 
 ### C6 · Three generated MNIST PDFs are in this branch's history
@@ -89,20 +67,6 @@ What is known to be wrong in GeometricMachineLearning and is not fixed.
   deliberately, and the resolution is upstream declaring them `public`. Each check's reason is
   written beside it in the test file.
 
-### C19 · `_make_block_for_initialization` has two methods
-
-- **location:** `src/architectures/symplectic_transformer.jl`
-- **kind:** dead code
-- **found:** 2026-09-19
-- **evidence:**
-  — `src/architectures/symplectic_transformer.jl`
-  and `src/architectures/linear_symplectic_transformer.jl` — that differ only in which fields they read
-  from their argument: one reads `arch.transformer_dim` and `arch.sympnet_activation`, the other reads
-  `arch.dim` and `arch.activation`. The bodies are otherwise identical. This stood as a `TODO` in the
-  source; a comment must stand on its own, so it is recorded here instead. Closing it means agreeing
-  on one set of field names across the two architectures, or passing the dimension and the activation
-  in rather than reading them off `arch`.
-
 ### C20 · There is no Knet.jl example for the Hamiltonian neural network.
 
 - **location:** `TODO.md`
@@ -261,6 +225,82 @@ What is known to be wrong in GeometricMachineLearning and is not fixed.
   `src/data_loader/data_loader.jl:205` states the one-type signature. Closing it means widening the
   three signatures to `NamedTupleOfArrays{T, (:q, :p)}`, and the docstring with them.
 
+### K21 · `LagrangianNeuralNetwork` stores its activation in the field `act`, where every other architecture with one activation names it `activation`.
+
+- **location:** `src/architectures/lagrangian_neural_network.jl:5`
+- **kind:** found late
+- **found:** 2026-10-20, by the critics of part M5; the field is the same on `origin/main` at `4c4a8320`
+- **evidence:**
+  ```
+  $ grep -rn " act::\|\.act\b" src/architectures/
+  src/architectures/lagrangian_neural_network.jl:5:    act::AT
+  src/architectures/lagrangian_neural_network.jl:16:        [Dense(nn.width, nn.width, nn.act) for _ in 1:nn.nhidden]
+  src/architectures/lagrangian_neural_network.jl:20:        Dense(nn.dimin, nn.width, nn.act),
+  ```
+
+  The constructor's keyword is `activation`; only the field keeps the short name. The fix renames
+  the field, which changes a public field name.
+
+### K22 · A comment in `symplectic_attention.jl` names a test file under `test/parameters/`, which does not exist.
+
+- **location:** `src/layers/symplectic_attention.jl:131`
+- **kind:** docs
+- **found:** 2026-10-20, by the critics of part M5; the comment is the same on `origin/main` at `4c4a8320`
+- **evidence:**
+  The comment above the symmetric forward method of `SymplecticAttention` names
+  `test/parameters/symplectic_attention_network_parameters_gradient.jl`. The file is
+  `test/layers/symplectic_attention_network_parameters_gradient.jl`, and `test/parameters/` does not
+  exist. The fix changes the path in the comment.
+
+### K25 · The docstrings of `LinearSymplecticTransformer` and `SymplecticTransformer` say that `n_sympnet = 1` gives one `GradientLayerQ` and one `GradientLayerP`.
+
+- **location:** `src/architectures/linear_symplectic_transformer.jl:22`
+- **kind:** docs
+- **found:** 2026-10-21, by the branch verifier of part M5; the docstrings and the chains are the same on `origin/main` at `4c4a8320`
+- **evidence:**
+  Each transformer unit puts the same `n_sympnet` gradient layers after its `Q` attention layer and
+  after its `P` attention layer, and the alternation starts again in each block. With
+  `n_sympnet = 1` and the default `init_upper = true`, both gradient layers are `GradientLayerQ`:
+
+  ```julia
+  [typeof(l) for l in Chain(LinearSymplecticTransformer(4, 3; n_sympnet = 1)).layers]
+  # LinearSymplecticAttentionQ, GradientLayerQ, LinearSymplecticAttentionP, GradientLayerQ
+  [typeof(l) for l in Chain(SymplecticTransformer(4; n_sympnet = 1)).layers]
+  # SymplecticAttentionQ, GradientLayerQ, SymplecticAttentionP, GradientLayerQ
+  ```
+
+  The same sentence is at `src/architectures/symplectic_transformer.jl:35`. The count
+  `2n_sympnet` holds for one unit, `L = 1`. The fix states the layers of one unit as the chain
+  builds them, or changes the chain, which changes the network.
+
+### K26 · The docstring of `SymplecticAutoencoder` says that `decoder_init_q = true` makes the first layer of each decoder block a ``p``-type layer, where it is a `GradientLayerQ`.
+
+- **location:** `src/architectures/symplectic_autoencoder.jl:36`
+- **kind:** docs
+- **found:** 2026-10-21, by the branch verifier of part M5; the docstring is the same on `origin/main` at `4c4a8320`
+- **evidence:**
+  `decoder_init_q = true` selects `:DecoderInitQ`, whose `decoder_layers_from_iteration` passes
+  `isodd`, so layer 1 of each decoder block is a `GradientLayerQ`:
+
+  ```julia
+  ae = NeuralNetwork(SymplecticAutoencoder(8, 4; decoder_init_q = true), CPU(), Float64)
+  first(decoder(ae).model.layers) isa GradientLayerQ    # true
+  ```
+
+  The fix writes ``q`` for ``p`` in the docstring.
+
+### K27 · The docstring of `UnknownDecoder` names its third argument `n_encoder_blocks` and links to itself for an example.
+
+- **location:** `src/architectures/autoencoder.jl:102`
+- **kind:** docs
+- **found:** 2026-10-21, by the branch verifier of part M5; the docstring is the same on `origin/main` at `4c4a8320`
+- **evidence:**
+  The signature line reads `UnknownDecoder(full_dim, reduced_dim, n_encoder_blocks)`, and the third
+  argument fills the field `n_decoder_blocks` (`src/architectures/autoencoder.jl:113`). The last
+  line says that an example "can be constructed analogously to [`UnknownDecoder`](@ref)", which
+  links the docstring to itself; the example is in the docstring of `UnknownEncoder`. The fix names
+  the argument `n_decoder_blocks` and links `UnknownEncoder`.
+
 ## Upstream
 
 ### K2 · Revise prints EMFILE errors in the test log
@@ -301,16 +341,6 @@ What is known to be wrong in GeometricMachineLearning and is not fixed.
 
 ## Documentation
 
-### K5 · The `PositionalEncoding` docstring names `test/metal/`, which does not exist
-
-- **location:** `src/layers/positional_encoding.jl:75`
-- **kind:** docs
-- **found:** 2026-10-02
-- **evidence:**
-  The docstring says "the GPU tests in `test/metal/` do not cover this layer". The Metal tests
-  are in `test/devices/metal.jl`, and `origin/main` at 97bf0d8 has no `test/metal/` directory.
-  The fix is a change of that path under `src/`.
-
 ### K18 · The tensor-sampling formula in the `DataLoader` manual is one short
 
 - **location:** `docs/src/data_loader/data_loader.md:143`