docs: record that the reconstruction noise map is the unconstrained uncertainty - #472
Conversation
…ncertainty
reconstruction_covariance_matrix is [F + reg_coeff*H]^-1, the posterior
covariance of the positive-negative (unconstrained) Warren & Dye solve. But
Settings.use_positive_only_solver defaults to True, so the reconstruction is
normally NNLS. Constraining s >= 0 truncates the posterior, so this overstates
per-pixel uncertainty -- more so near the s = 0 boundary, and not meaningfully
at all for pixels pinned at exactly zero.
Documentation only; no behaviour change. The maths is left alone deliberately,
because measurement showed the practical effect is small at the operating point
and because the obvious "fix" is wrong.
Measured on real ray-traced fits (Isothermal + shear,
RectangularBilinearAdaptDensity, Constant regularization), varying the
regularization coefficient and locating the Bayesian-evidence optimum that a
model-fit actually converges on:
at the evidence-optimal coefficient (lambda* = 10 in every case tested)
r_eff 0.05: 96.6% of mesh pinned, noise overstated x1.263 median, flux 0.0%
r_eff 0.10: 87.1% pinned, x1.055 median, flux -1.3%
r_eff 0.30: 42.3% pinned, x1.007 median, flux 0.0%
well below the evidence optimum (under-regularized): up to x2.8 median and
x10 on individual pixels, with source flux through a S/N >= 5 cut moving by
as much as -49%.
The Bayesian evidence selects away from the regime where this matters, so for
most fits the bias is small and one-directional (conservative).
Also recorded: restricting the covariance to the solver's free set is NOT the
correction. That treats the active set as known and so understates. The two
bracket the true truncated-Gaussian posterior, and swapping one bound for the
other would not be more correct.
Caveat on the measurement, not repeated in the docstring: the lens mass was
fixed at truth, so a real fit with a free and imperfect mass model may need a
lower coefficient to absorb residuals -- which is the regime where the gap
opens. Tracked in PyAutoMind
draft/bug/autoarray/reconstruction_noise_map_solver_mismatch.md.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
Correction — one of the "defects left open" in this PR's description is not a defectThis PR's body closes by naming two things left open, one of which was:
That is intended behaviour, confirmed by the author. The control-flow description was accurate — The same claim appears in the "Out of scope" section of #468. It is wrong there too. Corrected in The other item named there stands unchanged: the covariance ignores Only residue worth anything, and it is cosmetic: the coupling is undocumented. Neither No code change; this PR's diff is unaffected. Generated by Claude Code |
2 participants
Documentation only — no behaviour change, one docstring, +28 lines.
Follow-up to #469. That PR fixed how the covariance is computed; this one records what it means.
The gap
reconstruction_covariance_matrixis[F + reg_coeff*H]^-1— the posterior covariance of the positive-negative (unconstrained) Warren & Dye (2003) eq. 12 solve. ButSettings.use_positive_only_solverdefaults toTrue, so the reconstruction is normally NNLS. Constrainings >= 0truncates the posterior, and a truncated Gaussian's covariance is not the untruncated one.So
reconstruction_noise_mapoverstates per-pixel uncertainty — more near thes = 0boundary, and not meaningfully at all for pixels the solver pinned at exactly zero.Why documentation and not a code fix
Two measured reasons.
1. The Bayesian evidence selects away from the regime where it matters. The regularization coefficient is a free parameter (
LogUniform(1e-6, 1e6)), and a pixelized fit maximises the Bayesian evidence when choosing it. Locating that optimum on real ray-traced fits (Isothermal+ shear,RectangularBilinearAdaptDensity,Constant):λ* = 10 in every case, well inside the scanned grid. Well below the evidence optimum — an under-regularized fit — the factor grows to ~2.8 median, ~10× on individual pixels, and source flux through an
S/N >= 5cut moves by as much as −49%. But that is a regime the evidence penalises.2. The obvious fix is wrong. Restricting the covariance to the solver's free set treats the active set as known, so it understates. The two quantities bracket the true truncated-Gaussian posterior — swapping one bound for the other would not be more correct. A real fix means computing the truncated posterior, which is not worth it for a bias this size.
So the honest move is to say what the number is, and let anyone quoting per-pixel error bars on a very compact source know it is good to a few tens of percent rather than exact.
Caveat on the measurement
Deliberately kept out of the docstring but recorded here and in the tracking prompt: the lens mass was fixed at truth. A real fit has it free, and a poor mass model may need a lower coefficient to absorb residuals — exactly the regime where the gap opens. Also: Nautilus samples a posterior over λ, so some mass sits below λ*. And
RectangularBilinearAdaptDensityonly; Delaunay untested.Tracked in
PyAutoMind/draft/bug/autoarray/reconstruction_noise_map_solver_mismatch.md, which also holds two related defects left open here: the noise map ignoringzeroed_ids_to_keepunderuse_edge_zeroed_pixels, and that setting being silently ignored when the positive-only solver is off.Tests
test_autoarray/inversion/— 346 passed. No code changed.🤖 Generated with Claude Code
https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
Generated by Claude Code