Skip to content

Extract per-device state out of RequestHandler (#11)#74

Open
ff225 wants to merge 2 commits into
mainfrom
fix/sciot-011-device-state-manager
Open

Extract per-device state out of RequestHandler (#11)#74
ff225 wants to merge 2 commits into
mainfrom
fix/sciot-011-device-state-manager

Conversation

@ff225

@ff225 ff225 commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Closes #11. First of three steps toward #45; unblocked now that #9 has landed.

Why

RequestHandler kept every piece of runtime state in class-level dictionaries — device_profiles, variance_detector, offloading_states, _layer_sizes_cache — mutated from both the ASGI thread and the background I/O thread with no locking.

What changed

New src/server/communication/device_state.py. DeviceStateManager owns the device profiles, the EMA tables, the variance detectors, the adaptive-risk state and the layer-sizes cache, all behind one RLock.

Per-device variance detectors. The detector used to be a single class-level instance, so heterogeneous devices reporting the same layer indices polluted each other's variance statistics and skewed offloading decisions. Each device now has its own.

Thread-safety. The debug-JSON snapshot is now taken under the lock before being handed to the background writer; it previously serialized a dict that the ASGI thread could be mutating.

Backwards compatibility. RequestHandler.device_profiles remains a live view of the manager's mapping, since http_server.py:233 and the dashboard read it directly as a plain dict. No caller outside request_handler.py changes.

Drive-by cleanup. The inference-cycle telemetry path recomputed layer_sizes via _load_stats() — data already in scope — behind a hasattr() guard that was always true.

uv.lock

The first commit regenerates uv.lock, which was unparsable by uv 0.11+ and blocked every uv command, including CI's:

Dependency `google-cloud-firestore` has missing `source` field
but has more than one matching package

The package is forked across two versions by platform marker; the analysis extra listed it without a version/source to disambiguate. Regenerated with uv lock (224 packages resolved). Included here because nothing else could be verified without it.

Testing

244 passed, 1 skipped (fast suite + HTTP end-to-end). device_state.py at 100% coverage; new tests/unit/test_device_state.py covers the EMA maths, per-device isolation, snapshot copying, and an 8-thread concurrency test. ruff clean on all touched files.

Notes for the reviewer

ff225 added 2 commits July 17, 2026 14:34
The committed lock listed `google-cloud-firestore` in the `analysis` extra
without a version/source, while the package is forked across two versions by
platform marker. uv 0.11+ refuses to parse it:

    Dependency `google-cloud-firestore` has missing `source` field
    but has more than one matching package

Every uv command failed on this, so the environment could not be synced at all.
Regenerated with `uv lock` (224 packages resolved).
RequestHandler kept every piece of runtime state in class-level dicts mutated
from both the ASGI thread and the background I/O thread, with no locking.

Introduce DeviceStateManager, which owns the device profiles, the EMA tables,
the variance detectors, the adaptive-risk state and the layer-sizes cache
behind a single RLock. RequestHandler.device_profiles stays as a live view of
the manager's mapping, since http_server and the dashboard read it directly.

The variance detector is now scoped per device: it used to be a single
class-level instance, so heterogeneous devices reporting the same layer indices
polluted each other's variance statistics and skewed offloading decisions.

Also drop a redundant _load_stats() call in the inference-cycle telemetry path
that recomputed layer sizes already available in scope, behind a hasattr()
guard that was always true.

First step of #45; unblocked now that #9 has landed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Review and reduce class-level mutable state in RequestHandler

1 participant