Skip to content

Summarize long sessions earlier, at a token ceiling #95

Closed

Description

@nasser1941

Problem

A session ran with a large context window and never summarized, so the prompt grew to about 84K tokens. Past about 75K, Qwen3.6 35B (3B active parameters) degraded in several ways:

  • it looped on one command;
  • it ignored a clearly printed fact from its own test output (grid_layout=3×3, when the user wanted 2×3);
  • it lost track of what had already been tried.

lcode compacts at 85% of the window, which with a 256K window means about 220K tokens: far past where small models work well.

Proposal

  • A token ceiling (setting compact_at, default around 64K tokens) on top of the percentage: lcode prunes, then summarizes, when either is reached.
  • Keep the user's goal: the summary keeps the user's requests verbatim (the most recent ones at least), so the goal survives.
  • Show it: /context shows the ceiling.
  • Measure with lcode bench --session before choosing the default.

Acceptance criteria

  • A configurable ceiling, with a measured default
  • Summaries keep the user's requests
  • Docs

Activity

  1. nasser1941 commented on Oct 10, 2026

    @nasser1941
    OwnerAuthor

    Shipped in 0.27.0 (#104). The measurement is in #104: with qwen3.6-35b, making room earlier costs no tasks (21/24 either way) and saves time (27.5 vs 33 min).

    PyPI build check, with a 16K ceiling and two requests: the first read CHANGELOG.md; the second ("which version added the secret check?") made lcode remove that read to make room. The model read the file again, because it's now named next to the request, and answered correctly (0.26.0). Before that fix, it answered from memory and got it wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions