SymbolicPullback(arch) gives the gradient of the per-sample sum, not of HNNLoss #343

SymbolicPullback(arch) for a HamiltonianArchitecture returns the batch loss HNNLoss(arch) as its value, but its gradient is the gradient of a different function: the sum of the per-sample losses.

HNNLoss is norm(F(X) - Y) / norm(Y) over the whole minibatch, which is not a sum over samples. src/pullbacks/symbolic_hnn_pullback.jl builds the gradient with build_nn_function(...; reduce = +), on the assumption, stated in the comment there, that "the loss of a batch is the sum of the losses of its samples".

Reproducer (Float64, one minibatch of 5 samples):

using GeometricMachineLearning
using GeometricMachineLearning: _get_contents
import Random, Zygote
Random.seed!(9)
arch = StandardHamiltonianArchitecture(2, 4, 1, GeometricMachineLearning.SigmoidActivation())
nn = NeuralNetwork(arch, CPU(), Float64)
x, y = randn(2, 5), randn(2, 5)
loss = HNNLoss(arch)
vz, bz = ZygotePullback(loss)(nn.params, nn.model, (x, y))
vs, bs = SymbolicPullback(arch)(nn.params, nn.model, (x, y))
gz = _get_contents(bz(one(vz)))
gs = _get_contents(bs(one(vs)))
persample = only(Zygote.gradient(ps -> sum(loss(ps, x[:, i:i], y[:, i:i]) for i in 1:5), nn.params))

Results:

• vz == vs == 1.0083246344361512: the values agree.
• dL/dW1[1]: central finite difference 0.002695929479656911, Zygote 0.0026959294110209093, symbolic 0.004388379726963325.
• maximum(abs, persample.L1.W - gs.L1.W) is 5.2e-17: the symbolic gradient is the gradient of the per-sample sum.

The same numbers come from main (cfe4aea, SymbolicNeuralNetworks 0.8.1) and from the branch of #342 (SymbolicNeuralNetworks 0.9.0), so this is older than #342. test/architectures/hamiltonian_neural_network_tests.jl checks only the keys of the symbolic gradient, so the suite does not see it.

Two ways out, one decision: make the symbolic loss the batch loss (no reduce = +, if build_nn_function can express a batch norm), or define HNNLoss as a sum or mean over samples so that the two agree. #342 adds a @test_broken that compares the symbolic gradient with Zygote's and refers to this issue.