[davidwalter2]

Summary

The pixel multiplicity correction and its 49 variations weight a muon by its own pixel-hit flag: w0 = p0_data / p0_MC without a valid pixel hit, w1 = (1 - p0_data) / (1 - p0_MC) with one. Only a few percent of the muons have no valid pixel hit, and those carry a weight far from one, so the variation templates are dominated by the binomial fluctuation of the number of such muons in each bin.

This PR adds weights without that noise. Muons without a valid pixel hit differ from the others by their momentum resolution only. The probability f0 of no valid pixel hit can therefore be given as a function of the generated pt and the reco/gen curvature ratio of a muon, and the muon weighted by the expectation of the flag weight:

w = (1 - f0) w1 + f0 w0

This is the same weight on average for any process and selection.

Changes

• muon_calibration.hpp: PixelMultiplicityProbability (lookup of f0), PixelMultiplicityResponseHelper, PixelMultiplicityResponseUncertaintyHelper, and matchedGenMuonPt / qopRatioRecoGen for the inputs. A muon without a matched generated muon gets the average probability, for which the nominal weight is one.
• muon_calibration.py: make_pixel_multiplicity_helpers(filename, reverse_variations, weights). PixelMultiplicityColumns and define_pixel_multiplicity_cols[_dimuon] give each helper its input columns.
• mz_dilepton.py books the response of the two muons with and without a valid pixel hit for the simulated samples (hPixelResponseTrig, hPixelResponseNonTrig: eta, generated pt, charge, reco/gen curvature ratio, pixel class).
• wremnants/postprocessing/pixel_multiplicity_utils.py and scripts/corrections/make_pixel_corrections.py: the probability map hZeroPixelProb from these histograms, stored in the correction file next to the pixel hit multiplicities. Per eta bin, the response of the muons with a valid pixel hit is a standardized shape whose width depends on the generated pt, and the response of the others is its convolution with a Gaussian; both are fitted with a binned Poisson likelihood. The script now takes the output file name (-o) and can take the multiplicities from a second histmaker output (--inputFilePixelHits).
• W, W-like and dilepton histmakers: --pixelMultiplicityFile and --pixelMultiplicityWeights auto|response|flag|both. both uses the response-based weights and books the flag-based variations in addition, as nominal_pixelMultiplicity{Syst,Stat}Flag.
• setupRabbit.py --pixelMultiplicityWeights flag takes the flag-based variations of an output made with both. Nuisance names and groups are the same in either case.

Default behaviour

Unchanged with the correction file that is in wremnants-data now: it has no probability map, so auto falls back to the flag weights and logs a warning. On 20 Z files the dilepton output with the defaults agrees to 3e-15 in all 519 histograms with one made before this change; the only additions are the two response histograms.

Usage

# 1. response histograms and pixel hit multiplicities (the map only needs the Z simulation)
python scripts/histmakers/mz_dilepton.py -o <dir> ...
# 2. correction file with the probability map
python scripts/corrections/make_pixel_corrections.py -i <dilepton output> -o pixelcorr.pkl.lz4
# 3. any of the W, W-like and dilepton histmakers
... --pixelMultiplicityFile pixelcorr.pkl.lz4 [--pixelMultiplicityWeights both]
# 4. fit inputs; add the option for the flag-based variations of an output made with "both"
python scripts/rabbit/setupRabbit.py -i <output> ... [--pixelMultiplicityWeights flag]

Validation

With a map from the full Z simulation, the response of muons without a valid pixel hit is wider than that of the others by 1.1% (|eta| < 0.8) to 16% (2.3 < |eta| < 2.4) at a generated pt of 40 GeV. The predicted probability agrees with the simulated fractions with chi2/ndf = 1.10 over all generated-pt and curvature-ratio bins.

Full-statistics W histograms, fully correlated variation, relative to the nominal (the noise is estimated from two independent halves of the simulation):

process	rms, flag	noise, flag	rms, response	noise, response	slope	correlation
W+ → μν	19.3e-4	3.5e-4	19.5e-4	0.83e-4	0.965	0.974
W− → μν	18.4e-4	3.9e-4	18.1e-4	0.85e-4	0.984	0.971
Z → μμ	29.6e-4	13.3e-4	26.8e-4	3.0e-4	0.981	0.888

• The nominal histograms are identical with either weights (the nominal is not reweighted unless --reweightPixelMultiplicity is given).
• Per nuisance, the median of noise² / rms² goes from 0.61 to 0.058; it is above 0.5 for 37 of the 49 nuisances with the flag weights and for none with the response-based ones.
• Dilepton, 300 Z files: the variation of the m(ll) spectrum agrees between the two, slope 1.009 ± 0.018.
• Fits: the W mass moves by +0.1 MeV (signal region) and +0.5 MeV (simultaneous ABCD fit) with unchanged total uncertainty; the impact of the pixel multiplicity group goes from 0.42 to 0.53 MeV and from 0.28 to 0.46 MeV. The dilepton Z mass moves by +0.2 MeV, with an impact of 0.25 MeV in both cases.

This branch was ported from a development branch; on the same events its dilepton, W-like and W outputs with both agree with the ones of that branch to 2e-15 in all histograms (apart from nominal_gen_smeared, which uses random numbers), and make_pixel_corrections.py, setupRabbit.py with either set of variations and the fits run on a 30-file output of all datasets.

Open points

• The correction file with the probability map is not in wremnants-data yet; until then it is passed with --pixelMultiplicityFile. Once it is there the response-based weights become the default.
• Beyond their noise the two sets of W templates differ by 2-3e-4 rms, about 14% of the template rms. One part is a yield offset per eta bin of the flag-based variation, from the statistical error of p0_MC in the correction. The other is a pt-dependent difference of 1-2e-4 that is not understood yet.
• The response histograms are booked for all simulated samples of the dilepton histmaker, though only the Z ones are used for the map.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GdFuNtsTV5EX87okA51kTz

[cmsmwbot]

CI ✅ on a5dd077: all 25 jobs green. Slowest: w-analysis 1h14m, lowpu-w 53m, alphas-z-histmaker 50m. Ready for review when it's taken out of draft.

[bendavid]

How is the map actually built here? It would make sense for this to somehow be unified with the ML reweighting scheme in the ONNX weights.

[bendavid]

And is the resulting weight distribution actually significantly better? (I would assume that events in the tails of the response distribution, ie with reco pT far from the gen pT, still get large weights for the variations.)