Skip to content

feat(scanmate-scan): port illumination correction, bit-exact - #74

Merged
russoedu merged 2 commits into
mainfrom
feat/python-scan-illumination-correction
Sep 26, 2026
Merged

russoedu merged 2 commits into
mainfrom
feat/python-scan-illumination-correction

Conversation

@russoedu

Copy link
Copy Markdown
Owner

Levelling and contrast: enhance_raster, estimate_contrast_points and
resolve_contrast_points, with the options and the applied-enhancement report. Every page
is compared to the TypeScript as a whole image, sha256 over every byte, not sampled.

The two roundings, and why thirteen random pages don't catch them

Two roundings sit side by side in this slice, and either would be right on most pixels on
its own:

  • the contrast stretch rounds a half up (Math.round)
  • the unsharp mask writes into a Uint8ClampedArray, which rounds a half to even

A generated document never lands on a half, so both survive a whole-image hash of
thirteen random pages
— which is exactly how a port using one rule for both would have
passed. Mutation testing said so, against a test file whose own docstring claimed the hash
covered it. It didn't.

So this adds pages built by hand, as recipes both sides follow. On a uniform page the
background mean is the page, so every ratio is exactly 1 and the stretch lands on exactly
255/6 = 42.5. The TypeScript reads 43; half-to-even would read 42.

The same reasoning applies to the contrast points. A generated page pins none of the
bounds — its percentiles land mid-block among near-identical ratios, where moving one by a
percent changes nothing. Two grey pages with a chosen histogram put a single pixel on
each boundary instead: both percentiles, both clamps, the >= the cumulative count is
tested with, and a ratio of 0.25, whose histogram position is exactly 42.5 — the one
value where the bin rounding is observable at all.

Absent, on purpose

sharpen_raster — it is this same unsharp mask with the blur done by libvips, and
blur_raster is absent from scanmate-ink for the reason that package's README gives.
sharpen here uses the box-blur mask, which is the one the sharpening was tuned with.

Mutation testing

23 mutants, 20 killed. The first pass killed 9 of 23; the fixtures above closed eleven
more. The three survivors are each provably equivalent or unreachable rather than untested,
and TestWhatNoFixtureCanSeparate asserts the fact that makes each safe:

Survivor Why no input separates it
min(high, max(low, v)) ↔ max(low, min(high, v)) equal whenever low <= high, and both call sites pass literals that satisfy it
alpha taken from the despeckled pixels or the raw ones despeckle copies alpha rather than filtering it, so they are equal for every page
the blur's own max(1, …) radius floor the background radius is already floored at 4 before the blur sees it

Each is kept as written because the TypeScript is written that way — a reader comparing the
two files should find the same expression, not a locally-simplified one.

Also here

@mnci/nx-python-pip 0.2.10 (its own commit), which is the fix for this workspace's red
release: its twine executor now skips a project with no new version, so a release stops
uploading every Python package — the unbumped ones from their permanently-stale on-disk
0.0.1. That is what created the junk 0.0.1 releases on PyPI and what spent the
new-project rate limit that 429'd [email protected].

npm install dropped the four nested @emnapi entries under the wasm32-wasi resolver
bindings again (fourth time), which breaks npm ci; restored, and npm ci re-verified.

Verification

lint typecheck test build green for scanmate-scan; root ESLint clean; nx run-many -t test green across all 15 projects; parity:check reports the goldens fresh
against a real TypeScript build. 43 tests in the new slice.

🤖 Generated with Claude Code

russoedu and others added 2 commits September 26, 2026 11:14
…unbumped packages

Its twine executor now reads the version data `nx release publish` hands every
`nx-release-publish` task and skips a project with no new version, as
`@nx/js:release-publish` already did. That is what has been failing this
workspace's release: only the changed package was meant to publish, but every
Python project was uploaded, and the ones nx had not versioned built from their
on-disk `0.0.1` - permanently stale, since `release.git.commit` is false.

For the four already on PyPI that was silent noise (`--skip-existing` warns and
exits 0, which is why `scanmate-ink`, `-align`, `-merge` and `-seal` each carry
a junk `0.0.1` release, two of them a `0.0.2` as well). For the three PyPI has
never seen it was the actual failure: each redundant upload tried to CREATE a
project at `0.0.1` and spent a slot in PyPI's new-project rate limit, so the one
genuine release - `[email protected]` - died on 429.

`npm install` dropped the four nested `@emnapi` entries under the wasm32-wasi
resolver bindings again, which breaks `npm ci`; restored from the previous
lockfile, and `npm ci` re-verified.

Co-Authored-By: Claude Opus 5 <[email protected]>
Levelling and contrast: `enhance_raster`, `estimate_contrast_points` and
`resolve_contrast_points`, with the options and the applied-enhancement report.
Every page is compared to the TypeScript as a whole image, sha256 over every
byte, not sampled.

Two roundings sit side by side here and either would be right on most pixels on
its own: the contrast stretch rounds a half UP (`Math.round`), while the unsharp
mask writes into a `Uint8ClampedArray`, which rounds a half TO EVEN. A generated
document never lands on a half, so both survive a whole-image hash of thirteen
random pages - which is exactly how a port using one rule for both would pass.
So this adds pages built by hand, as recipes both sides follow: a uniform page
whose every ratio is exactly 1 puts the stretch on exactly 255/6 = 42.5, and the
TypeScript reads 43.

The same reasoning applies to the contrast points. A generated page pins none of
the bounds - its percentiles land mid-block among near-identical ratios, where
moving one by a percent changes nothing. Two grey pages with a CHOSEN histogram
put a single pixel on each boundary instead: both percentiles, both clamps, the
`>=` the cumulative count is tested with, and a ratio of 0.25, whose histogram
position is exactly 42.5 - the one value where the bin rounding is observable.

`sharpen_raster` is absent: it is this same unsharp mask with the blur done by
libvips, and `blur_raster` is absent from `scanmate-ink` for the reason that
package's README gives. `sharpen` here uses the box-blur mask, which is the one
the sharpening was tuned with.

Mutation tested: 23 mutants, 20 killed. The three survivors are each provably
equivalent or unreachable rather than untested, and `TestWhatNoFixtureCanSeparate`
asserts the fact that makes each safe - the clamp bounds are ordered, despeckling
copies alpha rather than filtering it, and the background radius is floored at 4
before the blur's own floor of 1 could ever apply. They are kept as written
because the TypeScript is written that way.

Co-Authored-By: Claude Opus 5 <[email protected]>
@russoedu
russoedu merged commit 09750de into main Sep 26, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant