fix(gateway): one session queue, LRU eviction, telegram-only /start, awaited shutdown, webhook usage/502

CHANGELOG.md

@@ -2,6 +2,15 @@
 
 ## Unreleased
 
+- Gateway (#144): conversations queue on the shared `SessionManager` instead of
+  their own promise chains. Beyond `max_conversations`, the least recently used
+  conversation is evicted (it was the oldest inserted). Only Telegram's `/start`
+  command (`/start`, `/start@bot`, `/start <payload>`) is rewritten to "hello";
+  `/started ...` and other platforms pass through. Shutdown waits for adapters
+  to stop (up to 5 s) before exiting.
+- **Breaking (webhook):** `POST /message` returns the run's token `usage`
+  instead of `null`, and an agent failure is `502 {"error":"agent error: ..."}`
+  instead of `200` with the error in `reply`.
 - Profiles (#117): `~/.lich/profiles/<name>.json` (plus an optional `<name>.md`
   used as the system prompt) is merged between the global and project config.
   Select one with `--profile`, `LICH_PROFILE`, a project `profile` key, or the

docs/getting-started.md

@@ -94,7 +94,7 @@ curl -s -X POST http://localhost:8089/message \
 Observed response shape (the `reply` text is the model's answer; `usage` is `null` on this endpoint):
 
 ```json
-{"reply":"Hello! How can I help you today? I can assist with coding, file management, running commands, web searches, and more — just let me know what you'd like to do.","usage":null}
+{"reply":"Hello! How can I help you today? I can assist with coding, file management, running commands, web searches, and more — just let me know what you'd like to do.","usage":{"prompt_tokens":812,"completion_tokens":41,"total_tokens":853}}
 ```
 
 Stop the gateway with Ctrl+C (SIGINT) or `kill` (SIGTERM); both shut down adapters cleanly.

docs/user-guide/gateway.md

@@ -4,7 +4,7 @@
 
 ## How it works
 
-`lich gateway <platform...>` runs a long-lived process that forwards inbound chat messages to **one shared agent** and routes replies back. Per-conversation memory is keyed `platform:chat_id` (Telegram/Discord chat ids, Twitch channel names, webhook `chat_id` field): each conversation keeps its own bounded history capped at 40 messages (oldest evicted; conversations beyond 200 are evicted oldest-first). Messages for the same conversation are serialized, so overlapping messages never interleave histories; different conversations can run concurrently. Failures become a safe one-line reply: `agent error: <flattened message, 300 chars max>`.
+`lich gateway <platform...>` runs a long-lived process that forwards inbound chat messages to **one shared agent** and routes replies back. Per-conversation memory is keyed `platform:chat_id` (Telegram/Discord chat ids, Twitch channel names, webhook `chat_id` field): each conversation keeps its own bounded history capped at 40 messages (oldest evicted; beyond 200 conversations, the least recently used one is evicted). Messages for the same conversation are serialized, so overlapping messages never interleave histories; different conversations can run concurrently. Failures become a safe one-line reply, `agent error: <flattened message, 300 chars max>`; the webhook returns it as HTTP `502 {"error":"agent error: ..."}`.
 
 ```mermaid
 flowchart LR
@@ -25,7 +25,7 @@ lich gateway telegram discord twitch # no webhook server
 lich gateway                         # defaults to webhook
 ```
 
-The gateway is silent after startup: Telegram/Discord/Twitch respond only in chats, channels, or servers the bot can see or has joined, and the webhook only serves HTTP. Telegram media messages arrive as the placeholder text `media not supported yet`; other non-text events are ignored. Telegram `/start` is answered like a plain "hello".
+The gateway is silent after startup: Telegram/Discord/Twitch respond only in chats, channels, or servers the bot can see or has joined, and the webhook only serves HTTP. Telegram media messages arrive as the placeholder text `media not supported yet`; other non-text events are ignored. Telegram `/start` (also `/start@bot` and `/start <payload>`) is answered like a plain "hello"; other text, and other platforms, are passed through as written.
 
 **Security defaults:** the webhook binds loopback only; Telegram/Discord/Twitch default-deny until you configure allowlists; the gateway agent uses a read-only tool subset (no `terminal`, no file writes) unless you override `gateway.tools_enabled`.
 
@@ -40,7 +40,7 @@ lich gateway webhook
 ```sh
 curl -s -X POST http://127.0.0.1:8089/message \
   -H "content-type: application/json" -d '{"text": "hello"}'
-# -> {"reply":"...","usage":null}
+# -> {"reply":"...","usage":{"prompt_tokens":...,"completion_tokens":...,"total_tokens":...}}
 
 curl -s http://127.0.0.1:8089/health
 # -> {"status":"ok"}
@@ -203,19 +203,18 @@ Request:
 Only `text` is required. Any `platform` field in the body is ignored; the conversation key is always `webhook:<chat_id>`. Success (`200`):
 
 ```json
-{"reply":"Hello! How can I help you today? ...","usage":null}
+{"reply":"Hello! How can I help you today? ...","usage":{"prompt_tokens":812,"completion_tokens":24,"total_tokens":836}}
 ```
 
-`reply` is the agent's final answer; `usage` is always `null` on this endpoint (webhook replies are formatted without usage stats, unlike the chat/TUI footers). Errors:
+`reply` is the agent's final answer; `usage` is the run's token totals. Errors:
 
 | Status | When |
 | --- | --- |
 | `400` | Body missing or has no `text` field: `{"error":"text is required"}`. |
 | `401` | `LICH_GATEWAY_TOKEN` is set and the `x-lich-token` header does not match. |
 | `404` | Anything other than `POST /message` or `GET /health`. |
 | `500` | Internal dispatch failure: `{"error":"internal error"}`. |
-
-Agent-level failures (e.g. every provider failed) return `200` with `reply` set to a sanitized one-line `agent error: ...` string, so callers always get a deliverable text.
+| `502` | The agent run failed (e.g. every provider failed): `{"error":"agent error: ..."}`, a sanitized one-line message. |
 
 ### `GET /health`
 

docs/user-guide/godot.md

@@ -44,10 +44,10 @@ curl -s -X POST http://127.0.0.1:8089/message \
   -d '{"text":"round 1: hero1 at full. goblin is the only living enemy.","chat_id":"run-1"}'
 ```
 
-Success is exactly one JSON object. This endpoint always sends `usage: null` — it does not forward provider token counts:
+Success is exactly one JSON object, with the run's token totals in `usage`:
 
 ```json
-{"reply":"...","usage":null}
+{"reply":"...","usage":{"prompt_tokens":812,"completion_tokens":24,"total_tokens":836}}
 ```
 
 | Status | Body |
@@ -56,10 +56,11 @@ Success is exactly one JSON object. This endpoint always sends `usage: null` —
 | `401` | `{"error":"unauthorized"}` when `LICH_GATEWAY_TOKEN` is set and `x-lich-token` does not match |
 | `404` | `{"error":"not found"}` for any other method or path |
 | `500` | `{"error":"internal error"}` if the handler throws before a response is sent |
+| `502` | `{"error":"agent error: ..."}` when the agent run fails (a sanitized one-line message) |
 
-A failed run is still `200`. `reply` is then a sanitized `agent error: ...` line. There is no streaming, pagination, or cursor. Full platform notes: [webhook API](gateway.md#webhook-api-reference).
+There is no streaming, pagination, or cursor. Full platform notes: [webhook API](gateway.md#webhook-api-reference).
 
-Memory is keyed `platform:chat_id`, capped at 40 messages (oldest dropped) and 200 conversations (oldest dropped). For a roguelike, `chat_id` = the run id gives the commander that process's memory of the run. A new run id starts a fresh history. That history is in memory only — restarting the gateway clears it. Restate facts the digest still needs. Durable notes are a different file, below.
+Memory is keyed `platform:chat_id`, capped at 40 messages (oldest dropped) and 200 conversations (least recently used dropped). For a roguelike, `chat_id` = the run id gives the commander that process's memory of the run. A new run id starts a fresh history. That history is in memory only — restarting the gateway clears it. Restate facts the digest still needs. Durable notes are a different file, below.
 
 ## Wire the example plugin
 
@@ -157,6 +158,6 @@ Plugins run in-process with the agent's privileges (files, network, environment)
 ## Limits
 
 - A long run evicts gateway history past 40 messages. Put habits that still matter in the digest, or in `memory.jsonl` if they must survive a restart.
-- More than 200 concurrent `chat_id`s on one process drops the oldest conversation. Fine for one developer machine; a host of many runs should know the cap.
+- More than 200 concurrent `chat_id`s on one process drops the least recently used conversation. Fine for one developer machine; a host of many runs should know the cap.
 - Combat must finish if the gateway is down, the call times out, or `orders.jsonl` is empty or garbage. The game's fallback is the game's — this repo does not ship one.
 - Same-`chat_id` calls run one after another. Different `chat_id`s run concurrently. The plugin itself makes no concurrency guarantee; one bridge per `chat_id` is the intended pattern.

examples/persona_orchestrator/README.md

@@ -49,7 +49,7 @@ curl -s -X POST http://127.0.0.1:8090/message \
   -d '{"text":"round 1: hero1 at full. goblin is the only living enemy.","chat_id":"npc:commander:run-1"}'
 ```
 
-Success is `{reply, usage}`. `usage` is the run's `usage_total`. The CLI webhook still sends `usage: null`; a Godot client that only reads `reply` needs no change. Missing `text` is `400 {"error":"text is required"}`. A bad token is `401`. Wrong `Content-Type` is `415`. Oversized body is `413`. Unknown or missing `chat_id` is still `200` with `reply` starting `agent error: unknown persona` and `usage: null` — this example does not default `chat_id` to `"default"`, because a persona cannot be inferred.
+Success is `{reply, usage}`. `usage` is the run's `usage_total`. The CLI webhook sends the same shape, and returns `502` on an agent failure; this example keeps `200` with an `agent error:` reply. Missing `text` is `400 {"error":"text is required"}`. A bad token is `401`. Wrong `Content-Type` is `415`. Oversized body is `413`. Unknown or missing `chat_id` is still `200` with `reply` starting `agent error: unknown persona` and `usage: null` — this example does not default `chat_id` to `"default"`, because a persona cannot be inferred.
 
 From a source checkout, with the working directory at the repo root:
 

src/gateway/bus.ts

@@ -1,17 +1,19 @@
 /**
  * GatewayBus: conversation-keyed runner over one shared Agent.
  *
- * Per-conversation history lives in a bounded Map; concurrent messages for
- * the same conversation are serialized through a promise chain so history
- * never interleaves. Agent failures become sanitized reply strings.
+ * Per-conversation history lives in a bounded Map evicted by least recent
+ * use; messages for the same conversation are serialized on the shared
+ * SessionManager so history never interleaves. Agent failures become
+ * sanitized reply strings.
  */
 import type { Agent } from "../agent/agent.js";
 import type { AgentConfig } from "../agent/config.js";
 import { history_after_run_error } from "../agent/loop.js";
 import type { Message } from "../providers/types.js";
+import { create_session_manager, type SessionManager } from "../session/manager.js";
 import { logger } from "../util/log.js";
 import { check_gateway_sender } from "./access.js";
-import { sanitize_agent_error } from "./types.js";
+import { sanitize_agent_error, type GatewayReply } from "./types.js";
 
 export interface GatewayBusOptions {
   history_cap?: number;
@@ -33,7 +35,7 @@ export class GatewayBus {
   private readonly agent_factory: () => Agent;
   private agent: Agent | undefined;
   private readonly histories: Map<string, Message[]> = new Map();
-  private readonly chains: Map<string, Promise<void>> = new Map();
+  private readonly sessions: SessionManager = create_session_manager();
   private readonly history_cap: number;
   private readonly max_conversations: number;
   private stop_logging: (() => void) | undefined;
@@ -48,32 +50,19 @@ export class GatewayBus {
     }
   }
 
-  /** Serializes runs per conversation and resolves to the reply text. */
+  /** Serializes runs per conversation and resolves to the reply text (undefined when empty or denied). */
   async handle(platform: string, chat_id: string, user_id: string, text: string): Promise<string | undefined> {
+    const reply = await this.reply(platform, chat_id, user_id, text);
+    return reply === undefined || reply.text.length === 0 ? undefined : reply.text;
+  }
+
+  /** Like `handle`, with the run's usage and whether it failed; undefined when the sender is denied. */
+  async reply(platform: string, chat_id: string, user_id: string, text: string): Promise<GatewayReply | undefined> {
     if (check_gateway_sender(this.config, platform, chat_id, user_id) === false) {
       return undefined;
     }
     const key = conversation_key(platform, chat_id);
-    const previous = this.chains.get(key) ?? Promise.resolve();
-    const run = previous.then(() => this.run_once(key, platform, chat_id, user_id, text));
-    let tracked: Promise<void> = Promise.resolve();
-    tracked = run.then(
-      () => {
-        this.release_chain(key, tracked);
-      },
-      () => {
-        this.release_chain(key, tracked);
-      },
-    );
-    this.chains.set(key, tracked);
-    return run;
-  }
-
-  /** Drops a settled chain entry unless a newer message re-queued the key. */
-  private release_chain(key: string, tracked: Promise<void>): void {
-    if (this.chains.get(key) === tracked) {
-      this.chains.delete(key);
-    }
+    return this.sessions.enqueue(key, () => this.run_once(key, platform, chat_id, user_id, text));
   }
 
   /** Unsubscribes the debug tool logger (bus owns no other resources). */
@@ -88,21 +77,37 @@ export class GatewayBus {
     chat_id: string,
     user_id: string,
     text: string,
-  ): Promise<string | undefined> {
-    const input = text.startsWith("/start") === true ? "hello" : text;
+  ): Promise<GatewayReply> {
+    const input = is_telegram_start(platform, text) === true ? "hello" : text;
     const history = this.history_for(key);
     const agent = this.ensure_agent();
     try {
       const result = await agent.run({ input, history, label: `gw:${platform}:${chat_id}` });
-      this.histories.set(key, cap_history(result.messages, this.history_cap));
-      return final_reply_text(result.outcome.final?.content);
+      this.store_history(key, cap_history(result.messages, this.history_cap));
+      return { text: result.outcome.final?.content ?? "", usage: result.usage_total };
     } catch (error) {
       logger.error(`gateway bus run failed for ${key} (user ${user_id})`, error);
       const kept = history_after_run_error(error);
       if (kept !== undefined) {
-        this.histories.set(key, cap_history(kept, this.history_cap));
+        this.store_history(key, cap_history(kept, this.history_cap));
       }
-      return sanitize_agent_error(error);
+      return { text: sanitize_agent_error(error), failed: true };
+    }
+  }
+
+  /**
+   * Re-inserts the key so Map order tracks the most recent use, then trims to
+   * the cap: concurrent new chats can each pass `history_for` before any stores.
+   */
+  private store_history(key: string, messages: Message[]): void {
+    this.histories.delete(key);
+    this.histories.set(key, messages);
+    while (this.histories.size > this.max_conversations) {
+      const oldest = this.histories.keys().next();
+      if (oldest.done === true || oldest.value === key) {
+        break;
+      }
+      this.histories.delete(oldest.value);
     }
   }
 
@@ -113,7 +118,7 @@ export class GatewayBus {
     return this.agent;
   }
 
-  /** Oldest-first eviction keeps the conversation map bounded. */
+  /** Least-recently-used eviction keeps the conversation map bounded. */
   private history_for(key: string): Message[] {
     while (this.histories.size >= this.max_conversations && this.histories.has(key) === false) {
       const oldest = this.histories.keys().next();
@@ -141,6 +146,11 @@ function conversation_key(platform: string, chat_id: string): string {
   return `${platform}:${chat_id}`;
 }
 
+/** Telegram's bot-start command (`/start`, `/start@bot`, `/start <payload>`); not `/started` or other platforms. */
+function is_telegram_start(platform: string, text: string): boolean {
+  return platform === "telegram" && /^\/start(@\w+)?(\s|$)/.test(text);
+}
+
 /**
  * Newest `cap` messages, starting at the first user message in the window.
  * When the window has no user message, drop a leading tool-result fragment so
@@ -195,7 +205,3 @@ function assistant_owning_tools(messages: readonly Message[], start: number): nu
   const calls = parent.tool_calls;
   return calls !== undefined && calls.length > 0 ? index : undefined;
 }
-
-function final_reply_text(content: string | undefined): string | undefined {
-  return content === undefined || content.length === 0 ? undefined : content;
-}

src/gateway/runner.ts

@@ -49,7 +49,7 @@ function is_known_platform(platform: string): boolean {
 function build_adapters(config: AgentConfig, bus: GatewayBus, platforms: readonly string[]): PlatformAdapter[] {
   const params: AdapterParams = {
     config,
-    handle_message: (platform, chat_id, user_id, text) => bus.handle(platform, chat_id, user_id, text),
+    handle_message: (platform, chat_id, user_id, text) => bus.reply(platform, chat_id, user_id, text),
     get_agent: () => {
       throw new Error("get_agent is reserved for future use");
     },
@@ -109,16 +109,37 @@ async function start_all_adapters(adapters: readonly PlatformAdapter[]): Promise
   }
 }
 
+/** Upper bound on waiting for adapters to stop, so a hung platform socket cannot block exit. */
+const SHUTDOWN_TIMEOUT_MS = 5000;
+
+/** Stops every adapter and waits for them (bounded), then releases the bus and agent. */
+export async function shutdown_gateway(
+  agent: Agent,
+  bus: GatewayBus,
+  adapters: readonly PlatformAdapter[],
+  timeout_ms = SHUTDOWN_TIMEOUT_MS,
+): Promise<void> {
+  const stops = Promise.allSettled(
+    adapters.map((adapter) =>
+      adapter.stop().catch((error: unknown) => logger.warn(`gateway adapter stop failed: ${adapter.name}`, error)),
+    ),
+  );
+  let timer: ReturnType<typeof setTimeout> | undefined;
+  const timeout = new Promise<void>((resolve) => {
+    timer = setTimeout(() => {
+      logger.warn(`gateway adapters did not stop within ${timeout_ms}ms; exiting anyway`);
+      resolve();
+    }, timeout_ms);
+  });
+  await Promise.race([stops, timeout]);
+  clearTimeout(timer);
+  bus.stop();
+  agent.close();
+}
+
 function install_signal_handlers(agent: Agent, bus: GatewayBus, adapters: readonly PlatformAdapter[]): void {
   const shutdown = (): void => {
-    for (const adapter of adapters) {
-      void adapter
-        .stop()
-        .catch((error: unknown) => logger.warn(`gateway adapter stop failed: ${adapter.name}`, error));
-    }
-    bus.stop();
-    agent.close();
-    process.exit(0);
+    void shutdown_gateway(agent, bus, adapters).finally(() => process.exit(0));
   };
   process.once("SIGINT", shutdown);
   process.once("SIGTERM", shutdown);