Iris Node

Real-time face recognition from a webcam. FastAPI + InsightFace (buffalo_l, ArcFace 512-d embeddings) on the backend, a single-page HUD on the frontend.

Runs on CPU. Every number in this document was measured on an Intel i7-1065G7 (4 cores, 15 W laptop chip) — no GPU involved.

---

Quick start

Requires Python 3.10 or newer and a webcam. Developed and measured on Python 3.14 / Windows; the code itself is platform independent.

Windows

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe -m uvicorn main:app --port 8000

Linux / macOS

python3 -m venv .venv
./.venv/bin/python -m pip install -r requirements.txt
./.venv/bin/python -m uvicorn main:app --port 8000

Installation pulls in insightface and its dependencies (opencv, scipy, scikit-image) — roughly 500 MB, a few minutes on a normal connection.

Open http://localhost:8000/ and sign in:

username   admin
password   admin123

The first launch downloads the buffalo_l models (~280 MB) into ~/.insightface/models/ and seeds that account into an empty database. That download happens once and needs an internet connection; startup takes about a minute the first time and a few seconds afterwards.

These are demo credentials and they are printed above, so treat them as public. They exist so a fresh clone is usable straight away. Override them before the server is reachable by anything but your own machine:

$env:IRIS_ADMIN_USER     = "your-name"
$env:IRIS_ADMIN_PASSWORD = "something-long"

Or set IRIS_SEED_ADMIN=0 to skip seeding entirely, in which case the first page shown is an account-creation screen instead. Seeding only ever happens when the users table is empty, so it never overwrites an account you changed.

The frontend is served by the backend itself, so the webcam runs in a secure context (localhost) with no CORS or file:// problems. The server binds to 127.0.0.1 and is not reachable from the rest of the network by default.

---

What it does

Page	Purpose
/	Live view: webcam, recognition overlay, telemetry
/admin	Subject registry: guided enrollment, archive
/analyze	Diagnostics: why a face was or wasn't recognised
/login	Access control

---

Design notes

The interesting part of this project is not that it recognises faces — a hundred lines of InsightFace do that. It is that it does so continuously, on a CPU, without falling apart. Most of the decisions below exist because a measurement contradicted the obvious approach.

Detection and embedding are separate

FaceAnalysis.get() always runs both. Measured cost per 640×480 frame:

det_size	detection	embedding (per face)	6 faces total
640	143.6 ms	103.5 ms	717 ms
320	41.1 ms	96.7 ms	617 ms
256	22.3 ms	99.0 ms	624 ms

The model is simply detection + N_faces × ~100 ms. Predicted 621 ms at det_size=320 with six faces, against 617 ms measured.

Two things follow. Lowering det_size barely helps when several faces are in frame, because the recogniser dominates. And a face's identity does not change between frames, so recomputing it twice a second is pure waste.

Tracking pays for the recogniser once

Faces are tracked across frames; the embedding is computed when a face first appears and then re-verified occasionally. Steady-state cost drops from "detection + N × 100 ms" to just "detection".

Measured against a live server: 530 ms on the first frame, 47 ms on the ones after. An 11× improvement, and the single change that makes continuous recognition viable on this hardware.

Inference saturates every core

A subtlety that is easy to get wrong. Wall-clock time is not CPU time:

detection (det_size 320)   38 ms wall   148 ms CPU   → 3.88 cores
embedding, one face       101 ms wall   393 ms CPU   → 3.90 cores

ONNX Runtime parallelises across all four physical cores for every single inference. A budget computed from wall-clock time underestimates the real load by roughly 4×. With 4000 core-ms available per second, four cameras running detection three times a second cost 1776 core-ms/s — about 44% of the machine.

The quality gate must be size-normalised

Laplacian variance is the standard sharpness metric, and applying it to the raw crop is wrong: the value tracks crop size rather than sharpness.

face size	raw Laplacian	normalised to 112×112
295 px	10.3	163.8
148 px	70.9	146.9
75 px	520.1	178.0

The raw figure varies 50× with distance. Against a fixed threshold, a large sharp face reads as "blurred" and gets rejected — so the closer you stand to the camera, the less likely you are to be recognised. Resizing the crop to 112×112 first (the resolution ArcFace works at anyway) keeps the metric between 145 and 178 regardless of distance. Genuine motion blur on the same face measures 91 / 43 / 19 / 9, which is what makes a threshold of 55 meaningful.

Minimum face size, measured rather than assumed

The plausible reasoning — "ArcFace works on 112×112 crops, so anything under 60 px is interpolation" — turns out to be wrong. Same face, progressively downscaled, compared against its own full-resolution embedding and against five impostors from the same photo:

face size	correct match	best impostor	margin
109 px	0.987	0.203	0.784
55 px	0.985	0.191	0.794
33 px	0.904	0.200	0.704
16 px	0.667	0.132	0.535

Self-similarity degrades, but impostor similarity does not rise: small faces lose sharpness of identity without drifting toward other people. The floor sits at 32 px, chosen because below that the correct match falls under 0.9 and the margin would narrow with a larger gallery.

Rejection by margin

The original approach — argmax(similarity) >= 0.45 — cannot tell a confident match from a coin flip. A score of 0.46 with the runner-up at 0.45 and a score of 0.46 with the runner-up at 0.20 are treated identically.

The index therefore also requires a minimum gap between the best and second candidate. Given a query exactly halfway between two enrolled subjects:

plain threshold   → "alice, 69.4% confidence"   (runner-up also at 69.4%)
with margin check → unknown

The failure mode this prevents — identity swaps growing as the archive grows — is how these systems usually degrade in practice.

Temporal voting

Assigning identity from a single embedding is unstable: one frame with the head turned and the name disappears, then returns, then disappears. Instead each recognition contributes a decaying vote, and the displayed identity is the one with the most accumulated agreement. Over 20 frames with blurred and dark ones interleaved: zero identity changes.

Votes alone are not enough. A covered face often fails the quality gate and therefore produces no vote at all — neither for nor against — so the name would stay on screen simply because nothing contradicted it. Identity expires two seconds after the last positive confirmation.

Occlusion behaviour, measured:

covered	detected	similarity	outcome
nothing	yes	0.970	recognised
lower half	yes	0.525	recognised
upper 55%	yes	0.485	recognised
85%	yes	0.034	rejected by quality gate
everything	yes	0.144	rejected

ArcFace tolerates half a face remarkably well. That robustness is a feature, but it means the expiry rule is what does the work when someone covers up.

Brightness: the naive method wins

Low light does not stop the detector — it still finds every face at a mean luminance of 7/255 — but it degrades the embedding. Comparing correction methods on progressively darkened frames:

method	similarity @ 0.18	@ 0.06	cost
linear gain ×2	0.952	0.879	0.18 ms
gamma 2.2	0.914	0.866	0.26 ms
CLAHE	0.932	0.824	3.47 ms
histogram equalisation	0.900	0.050	1.12 ms

Plain multiplication — an ordinary brightness slider — beats CLAHE and costs 20× less. Histogram equalisation is actively destructive on very dark input: it amplifies sensor noise into structure and the identity is lost.

The slider applies the same gain to the displayed video (CSS filter, free) and to the frame the server analyses, so what you see is what the system sees.

Client-side tracking

Server round-trips are 40–100 ms. No amount of extrapolation makes that feel real-time, because between responses the position is being guessed. The browser therefore tracks each face itself, matching a small greyscale patch against the surrounding area.

The first version ran that every animation frame on a 176×132 buffer and cost 29.8 ms per frame, saturating the main thread — toBlob callbacks and fetch promises queued behind rendering, and round-trip time climbed to 1361 ms while the server was still answering in 99 ms. The tracker had become the bottleneck it was meant to remove.

It now runs decoupled from drawing, at ~30 Hz on a 112×84 buffer, with an adaptive budget: if a pass costs more than 10 ms the interval widens, and above 34 ms the tracker switches itself off and falls back to velocity extrapolation. Matching itself costs 0.32 ms per face with a maximum error of 1 px.

Pose estimation

Yaw and pitch come from the five keypoints and are used to guide enrollment.

Measuring the nose offset in image coordinates is wrong: tilting the head rotates the eye line, so a perfectly frontal face reads as turned. Rotating one image by ±20° moved the estimate by 0.445. Projecting onto an orthonormal basis built on the eye axis cuts that to 0.292 — better, not perfect, because the keypoint detector itself shifts under rotation. Tolerances are wide accordingly.

Pitch needed a second correction: the nose tip sits closer to the mouth than to the eyes, so a level head reads positive. Measured across frontal faces the neutral point is 0.23, not 0. Without that offset, guided enrollment kept asking people looking straight ahead to raise their chin.

---

Architecture

iris/
  auth.py              session auth, scrypt password hashing (stdlib only)
  enroll.py            guided multi-pose enrollment
  pipeline.py          multi-source orchestration, shared inference worker
  net/
    discovery.py       ONVIF WS-Discovery, subnet sweep, device fingerprinting
    onvif.py           minimal ONVIF SOAP client
  recognition/
    index.py           vector index, margin rejection, threshold calibration
  video/
    source.py          RTSP / file / synthetic camera sources
    gating.py          motion gate, quality gate, pose estimation
    tracker.py         IoU tracking, temporal voting
main.py                API and page routes
db.py                  SQLite persistence
static/                four pages: live, registry, diagnostics, login

Persistence

SQLite. Embeddings are stored as float32 BLOBs; the in-memory index is an L2-normalised N×512 matrix, so cosine similarity is a single matrix-vector product.

An early version grouped scores by subject with a Python loop over every row and took 5.2 ms on 2400 embeddings — seventeen times the cost of the actual arithmetic. Vectorising the aggregation (lexsort + bincount) brought it to 0.68 ms, verified against a naive reference implementation over 200 random queries with zero disagreements.

Security

Proportionate to a home or small-site deployment, not to an enterprise:

• Session cookie, HttpOnly and SameSite=Strict. The former means an XSS cannot read the session; the latter is the CSRF defence, and costs nothing on a same-origin app.
• Passwords hashed with hashlib.scrypt, n=2¹⁵. No binary dependency, and no reason to add one. Note that scrypt needs 128 × n × r bytes — exactly 32 MiB here, which is OpenSSL's default ceiling, so maxmem must be passed explicitly or the call fails.
• Login timing is equalised: the hash is verified even for unknown usernames, so response time does not reveal which accounts exist.
• CORS restricted to localhost. It used to be ["*"], which let any site open in the browser call DELETE /admin/persons/{id}.
• All names from the database reach the DOM through textContent. With innerHTML, a subject registered as <img src=x onerror=...> would execute on every page load.
• 12 MB payload ceiling. Without one, a large POST is materialised in RAM before any validation runs.
• Bound to 127.0.0.1 by default.

---

Configuration

$env:IRIS_THRESHOLD      = "0.45"   # cosine similarity floor
$env:IRIS_MIN_MARGIN     = "0.10"   # minimum gap to the runner-up
$env:IRIS_DET_SIZE       = "256"    # detector working resolution
$env:IRIS_GPU            = "1"      # CUDAExecutionProvider (needs onnxruntime-gpu)
$env:IRIS_ADMIN_USER     = "admin"      # seeded account, first run only
$env:IRIS_ADMIN_PASSWORD = "admin123"
$env:IRIS_SEED_ADMIN     = "0"          # disable seeding, use the setup screen
$env:IRIS_MAX_IMAGE_BYTES = "12582912"  # upload ceiling

/api/calibration derives a threshold from the enrolled data by comparing within-subject and between-subject similarity distributions. If the two overlap, no threshold separates them well, and the honest answer is that the enrollment photos need to be better — more useful than a knob to turn.

---

API

Method	Endpoint	Purpose
POST	/api/auth/setup · /login · /logout	Access control
POST	/api/recognize · /api/recognize-raw	Recognise a frame (JSON or raw JPEG)
POST	/api/analyze	Full per-face diagnostic breakdown
POST	/admin/enroll/start · /frame · /commit	Guided enrollment
POST	/admin/register	Single-shot registration
GET	/admin/persons	Subject list
DELETE	/admin/persons/{id}	Delete subject (cascades to embeddings)
GET	/api/health · /api/calibration	Status and calibration

Interactive docs at /docs.

---

Enrollment

One photo per subject is the biggest weakness a system like this can have. Guided enrollment walks through five poses — frontal, right, left, chin up, chin down — and captures automatically once pose and quality hold for three consecutive frames. Asking someone to click at the right moment reliably captures half a second late, with the pose already lost.

The coverage report matters more than the shot count: it compares the captured embeddings against each other and says plainly when they are too similar. Five near-identical photos do not make the archive more robust than one.

---

Known limits

• Not an anti-spoofing system. A photograph held up to the camera is recognised as the person. Detecting that is a separate problem, not solved here.
• Faces below ~32 px do not produce reliable embeddings. That is an optics problem, not a software one.
• Network cameras: the ONVIF and RTSP discovery code exists and correctly distinguishes cameras from other devices — it was validated on a real subnet where the promising-looking hosts turned out to be two Canon printers and a VoIP phone — but the ingestion path has only been exercised against synthetic and file sources.
• Sustained load on a 15 W laptop chip causes thermal throttling. The same operation measured anywhere between 40 ms and 145 ms depending on how warm the machine was.
• Biometric data. Face embeddings are personal data. In the EU, processing them to identify people falls under GDPR Article 9 and requires a lawful basis before any real-world use. data/iris.db should not be committed.