Skip to content

feat(serve): queue frames per session on each connection - #183

Merged
Moikapy merged 64 commits into
mainfrom
claude/open-issues-prs-4nvvn3
Oct 8, 2026
Merged

Moikapy merged 64 commits into
mainfrom
claude/open-issues-prs-4nvvn3

Conversation

@Moikapy

@Moikapy Moikapy commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Part of #144, section B: "per-connection frame handling lets different sessions on one connection run concurrently". This fixes Ossuary sessions queuing behind each other on its one WebSocket.

Before and after

  • Before: attach_client chained every frame on a connection onto one queue. A long prompt.submit on session A held up session B's submit, and even health and session.list, until A finished.
  • After: frames are queued by params.session_id.
    • Frames for one session keep their arrival order and reply order.
    • Different sessions, and frames that name no session (health, session.list, session.create, which share one lane), no longer wait behind other sessions' runs.
    • prompt.abort still bypasses every queue, and queued submits are still registered for cancellation when they arrive.
    • A lane is dropped once it goes idle, so the map doesn't grow with every session ever used.

Consequence: replies from different sessions can now interleave on one socket, so clients must match replies by JSON-RPC id. Ossuary already does this (the pending map in apps/ossuary/electron/gateway-session.ts).

Safety of concurrent runs: the per-session SessionManager in prompts.ts already runs different sessions concurrently across connections, and each run's events go through its own on_event, tagged with its session_id (#136). The doc line saying runs are "serialized so fan-out stays tagged" was stale, and this PR replaces it.

Tests

  • New: sessions A and B share one connection, and A's model call hangs. B's submit and a health frame both complete before A, and A completes once released. The reply order is b, health, a.
  • Changed: the "binary prompt.abort bypasses the queue; a health frame that mentions abort does not" test now gives its health frame the same session_id, so the frame still shares the submit's queue and the test still checks that text mentioning prompt.abort isn't treated as an abort.
  • Mutation check: a single queue for all frames (the old behavior) makes the new test fail, and per-frame lanes with no same-session ordering make the abort-ordering test fail.

Checks

  • tsc --noEmit: clean.
  • vitest: 708/712 pass. The four failures happen only in this container and fail identically on clean main: serve_transport "::1" (no IPv6), plus three fetch/SSRF tests that need DNS (getaddrinfo ENOTFOUND example.com).
  • vitepress build docs: passes. docs/architecture/serve.md and the CHANGELOG are updated.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava


Generated by Claude Code

claude added 30 commits October 2, 2026 17:03
Ingest decision-model and Ollama research, add the decision-models
concept page, note ollama.com cloud auth and stale defaults on the
providers page, and link the fast lane from action-terminal mode.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
Ingest a read of Hermes upstream at bed0d535 and record how it mixes
models (fallback chain plus per-task auxiliary models), where embeddings
live (memory plugins), and how its 14 community Jev plugins work. Add
comparison rows and point decision models and plugin hooks at the
revised #148/#149 scope.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
llama3.2 (3B) is weak at tool calling; qwen3:8b is the common local
pick for agents. Switch the README, docs, persona example and the setup
wizard default to qwen3:8b, keeping llama3.2 as the low-memory note.

The Ollama provider already sends a Bearer header when api_key or
api_key_env resolves, so Ollama cloud works today. Document it
(LICH_BASE_URL=https://ollama.com, LICH_API_KEY_ENV=OLLAMA_API_KEY) and
replace "unused by ollama" with "optional for ollama".

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…press

Optional `models` block names provider chains per role. `chat` sets the
main loop's failover order; `compress` routes context compression and
falls back to the chat chain on failure. ProviderRouter.for_role shares
built clients. Without `models`, behavior is unchanged. The TUI labels
the first chat-role provider.

Part of #149.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…hook

Plugin entries may be { path, settings?, models? } beside a bare path.
Hooks and plugin tools receive the frozen settings and models.chat(role,
...), which refuses roles the entry was not granted. A new
before_llm_call hook may return a note that is capped, sent as a
trailing system message on that one main-loop call, and never saved to
history; throwing hooks fail open.

Part of #149.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
A plugin-owned client for Ollama's local /v1/systemone endpoint asks one
choice question per enemy action (and target) before each LLM turn.
Shadow mode logs one JSONL line per decision; act mode queues orders
through game_bridge's enemy_actions tool and meteor veto when every
answer clears the threshold, else the LLM decides. Requests are checked
against Ollama's limits before sending; failures fall back and are
logged. bench.mjs replays saved snapshots for latency, coverage and
agreement with the LLM.

Part of #148.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…ed; no tools on the last turn

- A tools_enabled list (top-level or gateway) now applies to plugin tools,
  including the gatekeeper's git_commit. The commander persona lists its
  three game_bridge tools explicitly.
- A before_tool_call hook that throws blocks the call
  (blocked_by_plugin: hook_error) instead of allowing it.
- Tool calls on the max_turns turn are not executed; they are closed with
  a turn_budget_exhausted result so history stays valid.

Part of #133.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…pts persist them

Also document the last-turn skip in docs/architecture/agent-loop.md.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
claude added 25 commits October 4, 2026 05:42
…old reply

On abort the loop's `final` is the last assistant so far (often the
previous turn's reply) and `messages` ends with the cancelled user line.
Chat and the TUI now show only the cancel notice and keep
history_after_abort(messages), which drops trailing user lines. The
config temp file is created with the existing mode. Tests cover a second
Ctrl+C (exit 130) and the chat turn after a cancel.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
reply_after_abort(outcome) returns `final` only when it comes after the
last user line, i.e. this run wrote it. Chat prints it and the TUI shows
it before the cancel notice; an earlier turn's reply is still skipped.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
Roadmap map lists #133 (P0), #134 (P1), #117 (P2), #144, #148 and #149
with their wiki pages; the kanban skill names the phase children.
Supersedes the stale #135.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…fig (#117)

Phase 0 + 1 of #117. Discovery is now project .lich/config.json merged
over ~/.lich/config.json (shallow, project wins per key). A project
providers array replaces the global one and drops the global models
unless the project sets its own. The global layer ignores work_dir and
session_dir, and its relative plugin paths resolve against ~/.lich/.
~/.config/lich/config.json is read only when ~/.lich/config.json is
absent, with a hint to move it; nothing writes there. --config still
replaces the whole chain.

Bare lich no longer pins an existing config as --config when it skips
the wizard, so the merge applies there too. first_run tests use an
empty HOME instead of mocking the home lookup.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…n is absent

With work_dir = home the global file is the project file; the legacy
~/.config/lich file was then picked as the base under it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…117)

lich init --global writes the starter config to ~/.lich/config.json
(never overwrites; no work_dir/session_dir). The setup wizard ends with
"Save as the global default?" (default no). Yes writes the answers to
~/.lich/config.json; discovered .lich/plugins entries stay in the
project file since they are project paths. The wizard no longer pins its
written file as --config, so discovery merges project over global.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…s home

With work_dir = home the project file is ~/.lich/config.json, so the
plugins-only project write hit "already exists" and dropped plugins.
Test also asserts the merged TUI plugins.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…json from home

Other projects load that file as the global layer and resolve relative
plugin paths against ~/.lich, so .lich/plugins/x would become
~/.lich/.lich/plugins/x.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
Phase 2 of #117. ~/.lich/profiles/<name>.json, plus an optional
<name>.md used as the system prompt, merges between the global and the
project config (global < profile < project). Selection: --profile, then
LICH_PROFILE, then a project `profile` key, then the default in
~/.lich/config.json. Profile plugin paths resolve against
~/.lich/profiles; work_dir/session_dir in a profile are ignored.

New `lich profile list|show|create|use`: create runs the wizard into the
profile file (never overwrites), use sets `profile` in the global file.
A requested profile skips the first-run wizard; --profile with --config
is an error. File tools now refuse .lich/profiles/.

tui/chat/serve/gateway no longer replace every config error with "no
model configured"; only that case gets the hint.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…t_dir/disk_usage

- work_dir = home: ~/.lich/config.json is the project file, so the
  profile now merges over it instead of under it.
- "profile not found" / "already exists" errors carry no absolute path.
- list_dir and disk_usage check the root with assert_file_tool_access,
  and list_dir skips denied entries, so .lich/profiles is not listed.
- Docs: LICH_PROFILE is ignored with --config.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…children

The home-dir branch now applies only when ~/.lich/config.json exists, so
a legacy ~/.config/lich file stays the base. disk_usage leaves out
entries file tools may not read (.lich/profiles, .lich/config.json).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
New entities/lich-config.md: global < profile < project merge, writers,
file-tool guard, and config profile vs runtime Profile. Guardrails page
and runtime-profile-session updated; index providers line fixed (#165).
SCHEMA gains the `config` tag.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
…awaited shutdown, webhook usage/502 (#144)

Slice 1 of #144 (A1 plus the G-10 and webhook fixes):
- GatewayBus queues conversations on SessionManager instead of its own
  promise chains (one queue implementation).
- max_conversations evicts the least recently used conversation.
- Only telegram's /start (/start, /start@bot, /start <payload>) becomes
  "hello"; /started and other platforms pass through.
- Shutdown awaits adapter stop (bounded by 5 s) before exit.
- bus.reply() returns { text, usage, failed }; the webhook returns the
  run's usage and 502 {"error"} on agent failure.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
Concurrent new chats could each pass the pre-run eviction and all store,
leaving the map over the cap. store_history now trims least recently
used entries after each write. Adds a keep-alive stop regression test.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
Slice 2 of #144 (section A, items 2-3):
- access.ts gains GatewayPolicy (allowlists + gateway toolset), built
  once from config; the runner builds it and hands it to the bus, which
  accepts an injected policy.
- PlatformAdapter declares capabilities { kind: "text", max_reply_chars }.
  telegram 4096, discord 2000 (was a bare literal), twitch 450, webhook
  uncapped; idle adapters keep theirs. The runner logs them at start.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
Frames are now queued by params.session_id instead of one tail per
connection, so a long prompt.submit no longer holds up other sessions or
session-less frames (health, session.list, session.create) on the same
WebSocket. One session's frames keep their order; prompt.abort still
bypasses every queue. Idle lanes are dropped. Docs: the stale "runs are
serialized" note is replaced (events are per-run since #136).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LHQAfXHofy6QkAgmfmJava
@Moikapy Moikapy added the needs-review label Oct 8, 2026 — with Cursor

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #183 — Risk: MEDIUM (per-connection serve queue becomes per-session lanes; 4 files, +102/−11. Runtime scheduling, not docs-only, so not LOW. Auth, bind, token, tool permissions, and public exports are unchanged, so not HIGH.)

Findings: Critical 0 / Warning 0 / Suggestion 1 (src/serve/server.ts:262, pin in-flight session bags so a concurrent session.create cannot LRU-evict them)
Regressions of settled findings: none
Not reviewed: none
Action: escalated to @Moikapy
Log: 2026-10-08T15:50:52Z, escalated, MEDIUM runtime scheduling change; CI still running (typecheck 20/22, ossuary, wiki_lint); auto-merge left off; head commit author is claude, not @Moikapy

@Moikapy — this lets different sessions, and frames with no session_id, run at the same time on one lich serve socket. Same-session order, the prompt.abort bypass, idle-lane deletion, and Ossuary matching replies by JSON-RPC id look right. I did not find a security or correctness break. CI has not finished, and the head commit is authored by claude, so I did not enable auto-merge.

Open in Web View Automation 

Sent by Cursor Automation: PR REVIEW

Comment thread src/serve/server.ts
@Moikapy
Moikapy merged commit 4a0686d into main Oct 8, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants