Skip to content

Summarize long sessions earlier, at a token ceiling - #104

Merged
nasser1941 merged 1 commit into
mainfrom
feat/compact-ceiling
Oct 10, 2026
Merged

nasser1941 merged 1 commit into
mainfrom
feat/compact-ceiling

Conversation

@nasser1941

Copy link
Copy Markdown
Owner

Closes #95.

In the dahua session the context reached about 84K tokens, with a large window and no summary. Past roughly 75K, Qwen3.6 35B looped, overlooked facts it had printed itself, and lost track of what it had tried. lcode only made room at 85% of the window, about 220K with 256K.

What changes

  • compact_at (default 65536, at least 16K, 0 = off): lcode makes room once the conversation reaches it, even in a larger window: it prunes old tool output first, and summarizes only if that's not enough. The 85% rule stays. The bench gets --compact-at TOKENS.
  • Summaries keep the user's requests word for word:
    • up to the last 8 (long ones cut in the middle), as a JSON list under the summary;
    • without the files and notes lcode attached;
    • carried over through later summaries.
  • /context says where the session makes room: makes room at 64K (compact_at).
  • No summary right after a summary: after one, the ceiling doesn't trigger another until the conversation has grown by a quarter of compact_at; a full window always does. A real test with a 6K ceiling summarized 4 times in a row before this guard: the model re-read the same file until the loop breaker stopped it. That's also why there's a 16K minimum.
  • Removed files are named next to the request. When pruning removes the only copy of a file read or fetched page, lcode says so next to the request the model is working on, and the stub is firmer. In a real test without this, the model answered a question about a pruned CHANGELOG.md from memory, and got it wrong ("status secret, exit code 5, secret_limit"). With it, the model read the file again and answered correctly (0.26.0).
  • Docs: how-it-works (context management), configuration, bench (--compact-at and the measurement), and the CHANGELOG.

Measured (qwen3.6-35b, 128K window, lcode bench --session --rounds 3, 24 tasks in one conversation)

No room made (--compact-at 64k: the conversation peaks at ~40K) --compact-at 32k
Pruned / summarized 0× / 0× 1× / 1×
Tasks solved 21/24 21/24
Time 33 min 27.5 min
Average prompt 22.9K tokens 16.6K tokens
Tool-call errors 15 11
  • What it shows: making room earlier than needed costs no tasks and saves time. So a 64K ceiling only affects sessions long enough to need it, like the dahua one.
  • What it doesn't: these 24 tasks never reach 64K, so the benefit above 64K comes from that session, not from this benchmark.

Tests

  • The ceiling and the limit it computes.
  • No re-summary right after a summary; a full window always summarizes.
  • The 16K floor on compact_at.
  • Requests kept word for word, without attachments, through two summaries.
  • Only the latest 8 requests kept.
  • /context output.
  • The note next to the request.

🤖 Generated with Claude Code

With a large window, lcode only made room at 85% (about 220K tokens with
256K), far past where small models work well. compact_at (default 64K,
at least 16K) makes room once the conversation reaches it, even when the
window is larger; 0 keeps only the 85% rule. Summaries keep the user's
latest requests word for word, /context shows where the session makes
room, and lcode bench --compact-at compares ceilings.

A conversation that was just summarized isn't summarized again until it
grew by a quarter of the ceiling, so a summary close to the ceiling
can't trigger one summary after another. Files whose only copy was
removed to make room are named next to the current request, because
models answered questions about them from memory instead of reading
them again.

Closes #95

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@nasser1941
nasser1941 merged commit 7334d73 into main Oct 10, 2026
12 checks passed
@nasser1941
nasser1941 deleted the feat/compact-ceiling branch October 10, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Summarize long sessions earlier, at a token ceiling

1 participant