Plate-solve the full sensor, with a cached optical calibration - #554
Open
mrosseel wants to merge 2 commits into
Open
Plate-solve the full sensor, with a cached optical calibration#554mrosseel wants to merge 2 commits into
mrosseel wants to merge 2 commits into
Conversation
mrosseel
force-pushed
the
fullframe-solve
branch
from
July 29, 2026 08:08
8dd0bfc to
d7fe556
Compare
This was referenced Jul 29, 2026
mrosseel
force-pushed
the
fullframe-solve
branch
from
July 29, 2026 22:25
d7fe556 to
ebd02a0
Compare
…tion
The camera cropped every exposure to a centred square before solving, throwing
away roughly half the sensor area (imx462: 1920x1080 down to 980x980). Cedar
Solve has no trouble with the whole frame, so that discarded area is free
plate-solving sensitivity.
The camera now publishes two frames per exposure: the 512x512 display frame,
unchanged, and a solve frame covering the whole sensor at native scale. Bayer
sensors are binned 2x2 for the solve frame, which removes the RGGB modulation
the old downscale to 512x512 used to smooth away.
Everything downstream of the solver -- SQM photometry, the preview overlay, the
alignment marker -- is written against the display frame, and SQM in particular
is calibrated to its plate scale. So solves are projected back onto the display
frame in one place (project_solution_to_display), keeping those consumers
unchanged. SolveGeometry owns the mapping, composed from the affine transform of
each pipeline stage.
The solver also stopped rediscovering the optics on every frame. It passed a
fixed 12 +/- 4 degree window and never passed a distortion estimate at all. The
first successful solve of a run now captures the measured FOV and lens
distortion; later solves get a +/-0.5 degree window and the measured distortion
as a starting point, which speeds up pattern matching and improves centroid
matching towards the frame corners -- the part of the image the crop used to
discard. A run of failures discards the calibration and reopens the window.
Full frame is on by default and can be turned off with solver_full_frame. It is
disabled automatically when the camera has no sensor profile, or when
camera_rotation is set to something other than a quarter turn, since rotating a
non-square frame by an arbitrary angle would clip the corners.
The 2x2 Bayer bin is gone. It was there because the crop pipeline's downscale
had been smoothing the RGGB checkerboard away implicitly, and losing that was
assumed to bias centroids toward the green sites. Measured against 12 imx462
frames of real sky, the assumption was wrong and the bin was costing exactly
what this change exists to gain:
solve rate median matches median RMSE solves down to
crop (today) 8/12 15.5 15.4" 257 ms
binned 10/12 20.0 40.5" 174 ms
un-binned 11/12 26.0 41.1" 97 ms
Un-binned finds 30% more matches than binned and solves at 2.6x shorter
exposure than the crop -- against 1.5x for the binned version, so binning was
giving away half the sensitivity gain. Angular residuals are statistically
identical (41.1" vs 40.5"), so the centroid-bias worry does not survive
contact with data; the finer sampling is simply what lets marginal stars clear
the detection threshold.
Full frame does cost accuracy against the crop (15.4" -> 41"), from corner
distortion a single-k model cannot fully absorb. That is a real trade, though
41" is negligible against any eyepiece field, and it buys solves on frames the
crop cannot solve at all.
mrosseel
force-pushed
the
fullframe-solve
branch
from
July 31, 2026 00:08
ebd02a0 to
4e116af
Compare
mrosseel
force-pushed
the
fullframe-solve
branch
from
July 31, 2026 00:14
b88183e to
bfdc632
Compare
/api/camera/raw rendered a uint16 sensor frame with Image.fromarray(arr, mode="L"), which reinterprets the 16-bit buffer as 8-bit. The result is interleaved-byte noise that still looks enough like an image to be believed -- I spent a night's captures on it before checking the histogram (median 1, p90 80, 1% at 255) and realising the frames were junk. 16-bit frames now render as mode "I;16" PNGs, preserving every ADU, and a test pins the round trip. Adds /api/camera/rawfull for the whole sensor including the margins the crop discards. Naming follows the exposure sweep's TIFFs: raw is the crop, rawfull is everything. The full frame is ~4 MB, so it is published on demand rather than on every capture -- the endpoint asks, the camera fulfils the request once and clears it. Between them these make the frames the pipeline actually works on retrievable from a running device, which is what debugging a solve failure in the field needs and what nothing currently offers.
mrosseel
force-pushed
the
fullframe-solve
branch
from
July 31, 2026 01:18
bfdc632 to
80a18ce
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two changes to the solver, both aimed at the same thing: stop throwing away sky, and stop rediscovering the optics on every frame.
1. Solve the full sensor
The camera cropped every exposure to a centred square before solving — on the imx462, 1920×1080 down to 980×980, so roughly half the sensor area was discarded. Cedar Solve has no trouble with the whole frame, so that area is free plate-solving sensitivity.
The camera now publishes two frames per exposure:
Bayer sensors bin 2×2 for the solve frame, which removes the RGGB modulation the old downscale to 512×512 was implicitly smoothing away.
Everything downstream of the solver is written against the display frame — SQM in particular scales matched centroids onto the raw photometry image and derotates them, a chain that only works starting from display-frame pixels. So a full-frame solve is projected back onto the display frame in one place (
project_solution_to_display), and the projected copy is what the rest of the system sees. Matched stars outside the crop are dropped along with their catalogue counterparts: they contributed to the solve, which is the point, but they have no pixels in the crop to measure.SolveGeometryowns the mapping, composed from the affine transform of each pipeline stage.Gated by
solver_full_frame(default on), and disabled automatically when the camera has no sensor profile or the frame rotation is not a quarter turn — rotating a non-square frame by an arbitrary angle would clip exactly the corners this exists to recover.2. Cache the optical calibration
The solver passed a fixed
fov_estimate=12.0, fov_max_error=4.0and never passed a distortion estimate at all, so Cedar re-derived both on every frame from scratch.The first successful solve of a run now captures the measured FOV and lens distortion; later solves get a ±0.5° window and that distortion as a starting point. This speeds up pattern matching and improves centroid matching toward the frame corners — the part of the image the crop used to discard, and the part where lens distortion actually bites. A run of consecutive failures discards the calibration and reopens the window. Session-scoped; nothing is persisted.
What the field of view actually does
Worth stating, because it is counterintuitive: after the camera's quarter turn the solve frame's horizontal axis is the sensor's short side, which the square crop already spanned. So horizontal FOV grows only modestly (imx462 ~10.2° → ~11.2°) or not at all (imx296, HQ). Total area still roughly doubles — the extra sky arrives vertically. The FOV search window is therefore derived per camera from the known crop field rather than hard-coded.
Cost
Benchmarked over 220 archived sweep frames (imx462 and HQ), exposures 25 ms – 1.1 s, on an x86 dev box:
2.0× the pixels cost −0.02 ms; 2.9× (HQ) cost −0.28 ms — nothing, within noise. Total extract+solve is 0.6–8% of one exposure cycle depending on exposure, so the CPU budget is not the constraint here.
Caveat on that benchmark: the sweep archive stores cropped raw, so the full-sensor arm is the real crop padded with its own background noise to the right pixel count. It measures the pixel cost honestly and the sensitivity benefit not at all — that needs a device run.
Testing
ruffclean,mypyclean on changed files, 1052 unit+smoke tests pass.Not yet run on hardware — the debug camera has no sensor profile, so that exercises the wiring but not the full-frame path itself.