Finding: adapters reporting no distribution recalibrate materially worse, even on simple miscalibration

[TMHSDigital]

This is a recorded finding, not a defect. Nothing here is broken and
there is no bug to fix. It documents measured behaviour and states what
remains open.

What was measured

An adapter that reports only a scalar probability for the selected label, with
no distribution over the other options, recalibrates materially worse than
one reporting a full distribution, and it does so even when the underlying
miscalibration is simple and exactly the kind a single temperature should fix.

The figures are in METHODOLOGY under "Adapters that report no distribution".
Two consequences are recorded there:

• Such an adapter is excluded from multiclass Brier entirely, because there is
  no distribution to score.
• Its recalibration is worse on the same miscalibration, because a temperature
  fitted against one scalar per row has less to work with than one fitted
  against the full vector.

Why it happens

Temperature scaling operates on logits across the option set. Given only the
top probability, the fit has to reconstruct a correction from a projection of
the thing it wants to correct. Information that was discarded before the fit
cannot be recovered by the fit.

The Noul path is the clean case rather than a problem: a Noul returns one
probability by design, with no distribution behind it, so nothing was
discarded. It is the cleanest calibration target the wire format offers and
METHODOLOGY says so.

What remains open

Whether anything can be done about it, or whether it is simply a cost of the
transport.

Specific unknowns:

• Is there a correction that works better than temperature on a scalar-only
  column? A monotone fit such as isotonic regression needs no distribution and
  might do better, at the price of needing more rows and being able to overfit.
• How much worse is it, as a function of n? "Materially worse" is currently
  qualitative. If the penalty vanishes at 2,000 rows it is a small-sample
  artifact; if it persists, it is a property of the transport and belongs in
  the adapter documentation as a reason to prefer one that reports a
  distribution.
• Should the report say this at the point of use? A user running a scalar-only
  adapter and seeing a poor recalibration verdict currently has no way to know
  that some of the poorness is the transport rather than the model.

The last one may be worth doing regardless of how the others resolve.

[TMHSDigital]

Two of the three open questions are answered:

• How much worse, as a function of n? Answered in docs(methodology): the no-distribution penalty is a bias, and more rows show it plainer #99. The top-line form's leftover ECE is flat at every size from 500 to 20,000 rows, while the floor falls, so the penalty is a property of the transport, not a small-sample artifact. METHODOLOGY has the table, and scripts/distribution_penalty.py reproduces it.
• Should the report say this at the point of use? It already did: the arm's Distribution caveat, plus a note in the recalibration section.

Still open: whether a monotone fit such as isotonic regression does better on a scalar-only column. That's a new correction method with its own rows-needed and overfitting trade-offs, so it needs a design before code.