Repository navigation
feat(observability): Prometheus /metrics and ws.session wide event (Phase 13) - #23
Merged
Merged
Conversation
…ide event (Phase 13) Active games/connections, agent turn latency/error/timeout counters per model, moves, and model-swap counts (DECISIONS.md §14) -- hooked into the existing single centralizing spot for each (AgentController._metrics, Referee._commit/_take_turn, RoomService.start/_finish_room), not new call sites. Also wires the ws.session wide event that docs/admin-and- observability.md has listed since Phase 1 but nothing ever built. The model-swap-aware turn *scheduler* DECISIONS §13 also describes remains explicitly out of scope: this only counts swaps, it doesn't yet try to minimise them. Also fixes a real onboarding bug found while adding this phase's one new dependency: docs/development/setup.md's documented `uv sync` silently drops the workspace-root dev tooling group (prek/zizmor/smokeshow) from the shared venv. Claude-Session: https://claude.ai/code/session_01UbiPa1rKCyqhqufgwJWkeW
- metrics.py: set explicit histogram buckets covering realistic LLM latency (default buckets topped out at 10s, LLM_TIMEOUT_SECONDS defaults to 120s) - controller.py: record real elapsed time on a hard LLM failure instead of hardcoding latency_ms=0 - fake.py: note why FakeLLMClient.base_url is a shared constant, so a future test doesn't assert an absolute swap-count value against it Claude-Session: https://claude.ai/code/session_01UbiPa1rKCyqhqufgwJWkeW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/metrics, unauthenticated (matches/health), dedicatedCollectorRegistry.botpit_active_games/botpit_active_ws_connections(gauges),botpit_agent_turn_latency_seconds/_errors_totalper model,botpit_agent_turn_timeouts_total(real agent seats only),botpit_moves_total,botpit_model_swaps_totalper base_url.AgentController._metrics,Referee._commit/_take_turn,RoomService.start/_finish_room) rather than new call sites.ws.sessionwide event (docs/admin-and-observability.mdhas listed this kind since Phase 1; nothing ever built it) plus the connection gauge, in the websocket route.docs/development/setup.md's documenteduv synccommand silently drops the workspace-rootdev/github-actionsdependency groups (prek/zizmor/smokeshow) from the shared venv -- discovered when adding this phase's one new dependency (prometheus-client). Fixed touv sync --all-groups.agent.turn→ the realgame.turnevent kind).Test plan
prek run --all-filesclean (ruff, mypy, ty, biome)/metricsagainst the real built Docker image (not just TestClient)decide()call, a realReferee.run()agent-seat timeout,ACTIVE_GAMES/MOVES_TOTALover a real start/finish/move cycle,ws.sessionwide event + connection gauge over a real websocket connectionClaude-Session: https://claude.ai/code/session_01UbiPa1rKCyqhqufgwJWkeW