Companion code for Reproducing a published pupil-diameter model, and running it in the browser on CondadosAI.
The model and the dataset are not ours. They are the work of Vijul Shah, Ko Watanabe, Brian B. Moser and Andreas Dengel (DFKI / RPTU Kaiserslautern-Landau), published as EyeDentify and PupilSense and released openly — code, weights, data. Nothing is trained here and no architecture is proposed.
What this repository adds is everything downstream of their checkpoint: an independent run over the full dataset, the error analysis, the scale experiments, the ONNX conversion and the quantisation study behind it, and the reference implementation of the input transform that the article's in-browser lab reuses.
| MAE over 424,000 crops, all 51 participants | 0.1041 mm (left eye), 0.1546 mm (right) |
| floor for predicting the dataset mean | 0.2369 mm / 0.2557 mm |
| after removing one constant per person | 0.0837 mm / 0.0878 mm |
| participants with >80% of their error as pure offset | 10 of 51 (left), 41 of 51 (right) |
| frame rescaling alone moves the answer by | 0.13–0.56 mm across eight clips |
| int8 export costs | −0.08% on one eye, +15.4% on the other |
| JavaScript input transform agrees with Python to | 1.4 × 10⁻⁴ mm |
The headline is the third row. The error is mostly a per-person offset rather than scatter, because nothing in the pipeline knows how large anything is: the network is handed a fixed 32×16 pixel window and asked for millimetres. Those offsets are fitted on the ground truth, so that line describes where the error lives — it is not an accuracy anyone can reach without a reference instrument.
uv sync --extra export
uv run pupil fetch-weights # both checkpoints, digest-verifiedThe dataset is on Kaggle and needs your own API token in ~/.kaggle/. The full archive is
31.9 GB because it also carries EyeDentify++, the super-resolved variants; only
eyedentify/eyedentify/{left,right}_eyes/ is used here, about 550 MB of 32×16 PNGs plus
one session_data.csv per session.
kaggle datasets download vijuls/PupilDiameterDatasets
uv run pupil prepare-data --archive PupilDiameterDatasets.zip --dest data/eyedentify
uv run pupil predict --data data/eyedentify --eye left_eye --out output/pred_left.csv
uv run pupil predict --data data/eyedentify --eye right_eye --out output/pred_right.csv
uv run pupil evaluate output/pred_left.csv output/pred_right.csv
uv run pupil export --data data/eyedentify # fp32, fp16 and int8, with a reportThe experiments behind the article's other tables:
uv run python scripts/quantisation_cost.py --data data/eyedentify --n 20000
uv run python scripts/scale_sensitivity.py --video data/samples/<clip>.webm
uv run python scripts/scale_reference.py --clips data/samples
uv run python scripts/verify_js_transform.py --crops data/eyedentify/left_eyes/1Everything the article publishes is in output/.
A stock torchvision ResNet-18 with its 1000-way classifier still attached, followed by
Linear(1000, 1). It is fed a 32×16 pixel eye crop, bicubic-resampled to 64×32, scaled to
[0, 1] with no ImageNet normalisation, and zero-padded to 224×224 inside forward — so
96% of what the backbone processes is black. All three are upstream's choices, reproduced
here because the checkpoint is meaningless without them, and strict=True on the state
dict is what proves the reproduction is faithful.
The paper reports a five-fold cross-validated MAE, where each fold's model is scored only
on participants it never saw. The released checkpoint is a single model whose training
split is not documented, so an error over all 51 participants is not the paper's
quantity, and this repository does not present it as one. pupil evaluate writes a
per-participant table for that reason, and summary.json carries baseline_mae_mm — the
error from ignoring the image and always predicting the mean — because that is the floor
any real result has to clear.
web/pil-bicubic.mjs reimplements PIL's bicubic resampler. Canvas drawImage uses a
different filter, and the checkpoints were trained on PIL's output, so a browser port that
scales through canvas feeds the network pixels it never saw. The detail that matters: PIL
writes the intermediate between its horizontal and vertical passes back into an 8-bit
buffer, and keeping that intermediate in floating point — the more accurate thing to do —
gives different pixels. scripts/verify_js_transform.py scores the two paths against each
other in millimetres.
The Apache-2.0 licence here covers the code in src/, scripts/, tests/ and web/,
and nothing else. The weights and the dataset are downloaded at run time from their
owners and are not redistributed. Their terms are stricter than ours:
| Asset | Source | Licence |
|---|---|---|
| PupilSense ResNet-18/50 checkpoints | vijulshah/pupilsense Space repo |
CC BY-NC 4.0 — non-commercial |
| EyeDentify / EyeDentify++ datasets | Kaggle vijuls/PupilDiameterDatasets |
CC BY-NC 4.0 — non-commercial |
| MediaPipe Face Landmarker | Google MediaPipe | Apache-2.0 |
Both the weights and the data are non-commercial. Anything built on them inherits that restriction; an Apache-2.0 file at the root of this repository does not lift it. The dataset also carries human-subject data — 51 consenting participants, published as eye crops rather than faces — and that consent was given for research, so treat it accordingly.
Cite the original work, not this repository:
@article{shah2024eyedentify,
title = {EyeDentify: A Dataset for Pupil Diameter Estimation based on Webcam Images},
author = {Shah, Vijul and Watanabe, Ko and Moser, Brian B. and Dengel, Andreas},
journal = {arXiv preprint arXiv:2407.11204},
year = {2024}
}
@inproceedings{shah2025pupilsense,
title = {PupilSense: A Novel Application for Webcam-Based Pupil Diameter Estimation},
author = {Shah, Vijul and Watanabe, Ko and Moser, Brian B. and Dengel, Andreas},
booktitle = {Proceedings of the 2025 Symposium on Eye Tracking Research and Applications (ETRA)},
year = {2025},
doi = {10.1145/3715669.3723125}
}