Repository navigation
perf: Pi infrastructure tuning (Backfill Place, atomic power upsert, healthchecks, test isolation, idle-session reaper) - #51
Merged
Conversation
Get Power Statistics read the day's aggregate on one connection and wrote it on another, so a concurrent writer could lose an update and the read->write upgrade failed immediately with 'database is locked'. PowerDailyTable .upsert_reading now does the read-modify-write in one BEGIN IMMEDIATE transaction (merge_reading is a pure function). Both 5-minute flows (Get Power Statistics, Check Watchdog) now use plain functions instead of Prefect tasks, log the happy path at DEBUG and drop the per-run INFO relay hook, cutting ~41 Prefect DB rows per run to ~8. Failure notifications and the watchdog staleness logic are unchanged. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…actions in Backfill Place Backfill Place held one write transaction for its whole ~15 minute run, and each lookup scanned the entire location_unified view (~1.2 s per row). Every morning that made Get Power Statistics and Geocode Places fail with 'database is locked'. Lookups now use indexed timestamp-range queries on the base tables (database/location/nearest_place.py, ~3 ms; identical results to the old query on 598 real lookups), run on a read-only connection, and the results are written in 500-row transactions, so the write lock is held for milliseconds. A full run now takes about a second. Rows are only updated if their place is still NULL. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The retroactive noise flow ran hourly although described as daily, and its tier-1 query full-scanned the 389k-row location_overland table (4.6 s). Ingest already flags noise in real time, so run it once a day at 04:00 local and add idx_overland_accuracy (a plain index: the threshold is an editable setting passed as a bound parameter, which a partial index cannot serve). Tier-1 query: 4,564 ms -> 3 ms. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The developer profile points PREFECT_API_URL at the production server, so tests that ran flows through the Prefect engine wrote real flow runs into it (dozens of failed 'Backfill FX' / 'Backfill GBP' runs). conftest.py now points Prefect at a closed local port before it is imported, so engine use fails fast. test_get_fx_up_to_date.py and test_backfill_gbp.py call .fn() with the run logger and task objects patched, and assert the retry policy and failure/success hooks as configuration. No production code changed; the full suite is about twice as fast. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ect URL - GET /health on the API (async, no DB, no auth; blocked on the public hostnames by the existing path allow-list). - Docker healthchecks for the API, dashboard (/healthz), nginx (/nginx-health) and the Prefect worker (runner health endpoint via PREFECT_RUNNER_SERVER_ENABLE); dashboard and worker wait for a healthy API. - ingest, worker and dashboard reach Prefect at http://prefect-server:4200/api over the Docker network instead of the Tailscale hostname; PREFECT_UI_URL keeps browser-facing links working. - Proposed prefect-server.service with a healthcheck (docs/drafts). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Proposed replacement for scripts/check_system_health.py, kept in docs/drafts because root's cron runs the live script straight from this path. Adds mount, Docker-health, restart-count, per-container OOM, systemd-service, backup-age and Prefect DB-size checks; restarts unhealthy containers with a budget of 3 per 6 hours; replaces the permanently-tripped 'swap > 60%' alert with swap-in rate; matches container names exactly (the old substring match treated 'travelnet' as running whenever 'travelnet-nginx' was); restarts prefect-server through systemd; and has a --dry-run mode. server/tests/test_health_script.py tests the live script once it has the new checks, otherwise this draft. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Runs the real script end to end with stub curl/docker/sudo on PATH (each test first asserts the stubs are what the script will resolve, so nothing can reboot). The script itself is gitignored, so the module skips when it is absent (fresh checkout, CI). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
scripts/reap_idle_dev_sessions.py finds Claude Code sessions (matched on argv[0]), treats a session as active if its project transcript was written or its process tree used CPU within the idle window (default 3 h), and stops idle ones (SIGTERM tree, SIGKILL after a grace period) with one Pushcut notification. Never touches its own process chain, guards against PID reuse, needs two observations before stopping, caps stops per run, and has --dry-run. systemd user service/timer drafts in docs/drafts. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Findings, measurements, decisions and the change log from the 2026-10-07 infrastructure and performance audit of the Pi. This public copy omits security-hardening findings, internal hostnames and addresses, and personal details; those are kept in a private copy. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Deploying travelnet with
|
| Latest commit: |
dc86a02
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://145a8525.travelnet.pages.dev |
| Branch Preview URL: | https://perf-pi-infrastructure.travelnet.pages.dev |
docs/ is a Jekyll site (docs.travelnet.dev). The infrastructure audit and the docs/drafts/ files are working notes meant for the repo, not pages for the website, so exclude them from the build. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Deploying travelnet-docs with
|
| Latest commit: |
dc86a02
|
| Status: | 🚫 Build failed. |
The first live run stopped the session it was being developed in after 3 h without a transcript write while the user was away. The app resumes a stopped session from its transcript, so nothing was lost, but a conversation that is merely waiting on the user should survive longer. The tests now state their window explicitly instead of relying on the default, and a new test pins the 6 h default. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An infrastructure and performance pass over the Pi that runs TravelNet. The common thread: stop long database write locks, cut per-run overhead of the frequent flows, make failures visible, and keep dev work from starving production.
Changes
fix(power)BEGIN IMMEDIATEtransaction; the 5-minute flows use plain functions, log the happy path at DEBUG and drop the per-run INFO relayperf(places)location_unified, lookups on a read-only connection, writes in 500-row transactionsperf(location)horizontal_accuracyis indexedtestconftest.pypoints Prefect at a closed port; the FX and GBP flow tests call.fn()feat(ops)GET /health; Docker healthchecks for the API, dashboard, nginx and Prefect worker; services reach Prefect over the Docker networkfeat(ops)(draft)docs/drafts, with--dry-runtest(ops)feat(ops)docs/drafts)docsDeploy notes
docker compose up -dapplies the compose healthchecks and the internal Prefect URL.nginx.confgains a/nginx-healthlocation on port 80.idx_overland_accuracy) is created automatically on API start./healthz(Dashboard repo).Testing
Covered by the unit tests added in each commit (the suite grows by ~120 tests, including mutation-checked tests for the new safety properties). CI runs the suite on this branch.
🤖 Generated with Claude Code