Summarize long sessions earlier, at a token ceiling

CHANGELOG.md

@@ -6,6 +6,26 @@ All notable changes to lcode are documented here. The format follows
 
 ## [Unreleased]
 
+### Changed
+
+- Long sessions are summarized earlier: once the conversation reaches 64K tokens (`compact_at`),
+  even when the window is larger. Before, a 256K window was summarized only at about 220K, far past
+  where small models work well. lcode still removes old tool output first, and summarizes only if
+  that's not enough. `lcode config set compact_at 0` keeps only the 85% rule, and
+  `lcode bench --compact-at` compares ceilings. Measured with qwen3.6-35b: making room earlier
+  doesn't cost tasks (21 of 24 either way) and saved 17% of the time. See
+  [Context management](https://nasser1941.github.io/lcode/how-it-works/#context-management).
+- Summaries keep your latest requests (up to eight) word for word, so the goal survives the
+  retelling. `/context` shows where the session makes room.
+
+### Fixed
+
+- After lcode removed an earlier file read to make room, the model sometimes answered questions
+  about that file from memory, and got them wrong, instead of reading it again. The files whose
+  only copy went are now named next to the request it's working on, and the note in their place
+  says plainly that the content is gone. A conversation that was just summarized isn't summarized
+  again until it has grown by a quarter of `compact_at`, which must be at least 16K (or 0).
+
 ## [0.26.0] - 2026-10-10
 
 ### Added

docs/bench.md

@@ -51,7 +51,7 @@ Each task normally starts a new conversation. `--session` runs them all in **one
 instead, as a long working session would, so the context fills up and lcode's
 [context management](how-it-works.md#context-management) has to work: the summary shows how often
 it removed old tool output and how often it summarized the conversation. `--no-prune` turns the
-removal off, to compare.
+removal off, to compare, and `--compact-at` sets the ceiling (`--compact-at 0`: only at 85%).
 
 ```bash
 lcode bench --context 32k --session --rounds 3
@@ -74,6 +74,17 @@ at no measurable cost in tasks solved or time. The bench tasks are independent o
 which is where summaries hurt least; in a real session, where later requests build on earlier
 ones, keeping the conversation intact matters more.
 
+With a large window, `compact_at` decides when lcode makes room. In a 128K window, 24 tasks in one
+conversation reach about 40K tokens, so the default ceiling of 64K never comes into play. Making
+room earlier than needed doesn't cost tasks:
+
+| 128K window, 24 tasks | No room made (peak ~40K) | `--compact-at 32k` |
+|---|---|---|
+| Conversation summarized | 0 times | 1 time (old tool output removed once) |
+| Tasks solved | 21 of 24 | 21 of 24 |
+| Time | 33 min | 27.5 min |
+| Average prompt | 22.9K tokens | 16.6K tokens |
+
 ## The tasks
 
 Every task runs in a new temporary folder, with every permission granted (`yolo` mode) and web

docs/configuration.md

@@ -37,6 +37,7 @@ lcode config path                      # print the file location
 | `repo_map` | `true` | Give the model a ranked [map of the repository](search.md) |
 | `embed_model` | `auto` | Embedding model for [semantic code search](search.md#semantic-code-search): `auto`, `off` or a model name |
 | `lsp` | `auto` | Use installed [language servers](lsp.md) for code navigation and errors after edits: `auto` or `off` |
+| `compact_at` | `65536` | Make room (prune, then summarize) once the conversation reaches this many tokens, even in a larger window (at least 16K); `0` leaves only the 85% rule ([why](how-it-works.md#context-management)) |
 | `prune` | `true` | When the context is 85% full, first remove old tool output, and summarize the conversation only if that's not enough ([how](how-it-works.md#context-management)) |
 | `subagents` | `true` | Let the model hand tasks to [subagents](agents.md) with their own context |
 | `max_parallel_agents` | `1` | Subagents that run at the same time; more needs `OLLAMA_NUM_PARALLEL` on the Ollama server ([details](agents.md#several-at-once)) |

docs/how-it-works.md

@@ -73,21 +73,32 @@ in a temporary git worktree, and their changes come back as a diff.
 ## Context management
 
 Ollama keeps the processed conversation in its KV cache, so each turn only processes the new tokens.
-lcode tracks how full the window is from Ollama's token counts. Most of a long session's context is
-old tool output, so when the window is 85% full:
+lcode tracks how full the window is from Ollama's token counts. It makes room when the window is
+85% full, or when the conversation reaches 64K tokens (`compact_at`), whichever comes first.
+Small models lose track in long contexts well before a large window is full: in one long session,
+Qwen3.6 35B started repeating itself and overlooking what it had read once the conversation passed
+about 75K tokens. Most of a long session's
+context is old tool output, so:
 
 1. **lcode removes old tool output first.** File contents, command output and search results from
    before your last two requests become one-line notes that say what was there, such as
-   `[lcode: read src/app.py earlier; removed to save context. Read it again if you need it.]`.
+   `[lcode: read src/app.py earlier; its content was removed …. Read it again before answering anything about it.]`.
    An earlier copy of a file that was read again later goes too, even in recent requests. Subagent
-   reports, skill instructions and the approved plan stay.
-2. **If that brings the conversation under 70%,** the work simply continues: nothing is summarized,
+   reports, skill instructions and the approved plan stay. The files whose only copy went are also
+   named next to the request the model is working on, since models tend to overlook a note deep in
+   the conversation and answer from memory instead of reading the file again.
+2. **If that brings the conversation well under the limit** (70% of the window, or about 53K
+   tokens with `compact_at`), the work simply continues: nothing is summarized,
    and the conversation itself stays word for word.
    If it's not, lcode also removes the tool output of your previous request, keeping only the
    latest one's. (`lcode config set prune false` skips this step.)
 3. **Otherwise lcode summarizes.** It asks whether anything is worth [remembering](memory.md), since
    the summary leaves details out, then has the model write a summary of the conversation (goals,
-   findings, files changed, next steps) and continues from it.
+   findings, files changed, next steps) and continues from it. Your latest requests (up to eight)
+   are kept word for word under the summary, so the goal isn't lost in the retelling.
+
+`lcode config set compact_at 128k` moves the ceiling, and `0` turns it off, leaving only the 85%
+rule. `/context` shows where the current session makes room.
 
 Long tool output (over 30,000 characters) keeps its beginning, its end and the lines in between
 that look like errors. The full output is saved under `~/.local/state/lcode/outputs/` (for a week)

src/lcode/agent.py

@@ -234,6 +234,7 @@ class Settings:
     notify_after: int = 30  # seconds: a request this long notifies when it's done or waits for an answer
     max_parallel_agents: int = 1
     secret_check: bool = True  # scan what git commit and git push would add (lcode.secretscan)
+    compact_at: int = 0  # tokens: make room at this size even when the window is larger (0: only at 85%)
     repeat_limit: int = 3  # identical calls with identical results before the model is told (lcode.repeats)
 
 
@@ -270,6 +271,7 @@ def __init__(self, ollama: Ollama, settings: Settings, cwd: Path, console: Conso
         self.agent_runs: list[subagents.Record] = []  # subagents run in this session, for /agents
         self.plan = ""  # the plan the user approved; kept through compaction
         self.pruned = self.compacted = 0  # how often the context was pruned or summarized automatically
+        self.summarized_size = 0  # tokens right after the last summary (0: none in this conversation)
         self._extensions: tuple[Path, extensions.Extensions] | None = None
         self.skills_loaded: set[str] = set()  # skills whose instructions are in the conversation
         # Set on subagents (see lcode.subagents):
@@ -514,6 +516,7 @@ def web_prompt(self) -> str:
         return WEB_PROMPT.format(tools=tools, date=dt.date.today().isoformat())
 
     def reset(self) -> None:
+        self.summarized_size = 0
         self.messages = [{"role": "system", "content": self.system_prompt()}]
         self._mcp_prompt = ""
         self.tools.read_mtimes.clear()
@@ -1046,23 +1049,45 @@ def missing_image(shown: str) -> None:
         return text + ("\n\n" + "\n\n".join(attached) if attached else "")
 
     # -- context management
+    def compact_limit(self) -> int:
+        """The tokens in use at which lcode makes room: 85% of the window, or `compact_at` when that's lower.
+        Small models lose track in long contexts well before a large window is full."""
+        limit = int(AUTO_COMPACT_RATIO * self.settings.context)
+        if self.settings.compact_at > 0:
+            limit = min(limit, self.settings.compact_at)
+        return limit
+
     def maybe_compact(self) -> None:
-        if self.ctx_used <= AUTO_COMPACT_RATIO * self.settings.context:
+        limit = self.compact_limit()
+        if self.ctx_used <= limit:
             return
+        enough = int(limit * PRUNED_ENOUGH / AUTO_COMPACT_RATIO)  # after pruning, a summary isn't needed below this
         if self.settings.prune:
-            saved = context_tools.prune(self.messages)  # tool output from before the last two requests
-            if self.ctx_used - saved // 3 > PRUNED_ENOUGH * self.settings.context:
-                saved += context_tools.prune(self.messages, keep=1)  # not enough: before the last request
+            removed: list[str] = []  # files and pages whose only copy goes
+            saved = context_tools.prune(self.messages, removed=removed)  # tool output from before the last two requests
+            if self.ctx_used - saved // 3 > enough:
+                saved += context_tools.prune(
+                    self.messages, keep=1, removed=removed
+                )  # not enough: before the last request
+            context_tools.note_removed(self.messages, removed)
             if saved:
                 self.pruned += 1
                 self.ctx_used = max(len(self.messages[0]["content"]) // 3, self.ctx_used - saved // 3)
                 self.console.print(
                     f"[dim]Removed old tool output from the conversation (~{format_tokens(saved // 3)} tokens) to "
                     "make room.[/]"
                 )
-                if self.ctx_used <= PRUNED_ENOUGH * self.settings.context:
+                if self.ctx_used <= enough:
                     return
-        self.console.print("[yellow]Context is nearly full — compacting the conversation…[/]")
+        window_full = self.ctx_used > AUTO_COMPACT_RATIO * self.settings.context
+        if not window_full and self.summarized_size and self.ctx_used - self.summarized_size < limit // 4:
+            return  # summarized not long ago: summarizing again so soon would only lose more
+        if limit < AUTO_COMPACT_RATIO * self.settings.context:
+            self.console.print(
+                f"[yellow]The conversation reached {format_tokens(limit)} tokens (compact_at) — summarizing it…[/]"
+            )
+        else:
+            self.console.print("[yellow]Context is nearly full — compacting the conversation…[/]")
         self.compacted += 1
         self.compact()
 
@@ -1083,9 +1108,13 @@ def compact(self, focus: str = "") -> None:
         with self.console.status("[cyan]Compacting…[/]"):
             for chunk in self.chat([*self.messages, {"role": "user", "content": instructions}], None, False):
                 summary += chunk.get("message", {}).get("content", "")
+        requests = context_tools.user_requests(self.messages)
         self.messages = [
             {"role": "system", "content": self.system_prompt()},
-            {"role": "user", "content": f"[Summary of our conversation so far]\n\n{summary}"},
+            {
+                "role": "user",
+                "content": f"{sessions.SUMMARY_PREFIX}\n\n{summary}" + context_tools.requests_block(requests),
+            },
             {"role": "assistant", "content": "Got it — I have the context from the summary and will continue."},
         ]
         if self.plan:  # the approved plan stays word for word
@@ -1097,6 +1126,7 @@ def compact(self, focus: str = "") -> None:
             )
             self.skills_loaded.clear()
         self.ctx_used = sum(len(m["content"]) for m in self.messages) // 3
+        self.summarized_size = self.ctx_used
         self.reflected = len(self.messages)
         self.console.print(Panel(Markdown(summary), title="Compacted summary", border_style="blue"))
 

src/lcode/api.py

@@ -196,6 +196,7 @@ def open_agent(cwd: Path, options: Options, console: Console, hw: Hardware | Non
         notify_after=cfg["notify_after"],
         secret_check=cfg["secret_check"],
         repeat_limit=cfg["repeat_limit"],
+        compact_at=cfg["compact_at"],
     )
     agent = Agent(ollama, settings, cwd, console=console)
     agent.interactive = options.interactive

src/lcode/cli.py

@@ -579,6 +579,7 @@ def cmd_bench(args) -> None:
             web="off",
             checkpoints=False,
             prune=not args.no_prune,
+            compact_at=cfg["compact_at"] if args.compact_at is None else args.compact_at,
             repo_map=args.repo_map,
             embed_model=cfg["embed_model"] if args.code_search else "off",
         )
@@ -678,6 +679,14 @@ def cmd_chat(args) -> None:
         sys.exit(exit_code)
 
 
+def compact_tokens(value: str) -> int:
+    """`--compact-at`: a token count such as 64k, or 0."""
+    try:
+        return config.parse_compact_at(value)
+    except ConfigError as e:
+        raise argparse.ArgumentTypeError(str(e)) from e
+
+
 def build_parser() -> argparse.ArgumentParser:
     parser = argparse.ArgumentParser(
         prog="lcode",
@@ -765,6 +774,12 @@ def build_subparsers() -> dict[str, argparse.ArgumentParser]:
     p.add_argument("--repo-map", action="store_true", help="give the model the repository map (off by default)")
     p.add_argument("--code-search", action="store_true", help="index each task's folder and offer semantic code search")
     p.add_argument("--no-prune", action="store_true", help="don't remove old tool output (to compare, with --session)")
+    p.add_argument(
+        "--compact-at",
+        type=compact_tokens,
+        metavar="TOKENS",
+        help="summarize at this many tokens, e.g. 64k (default: the compact_at setting; 0: only at 85%%)",
+    )
     subs["bench"] = p
     p = argparse.ArgumentParser(
         prog="lcode index",

src/lcode/config.py

@@ -61,6 +61,7 @@
     "notify_after": (30, int, "seconds a request runs before it notifies (with notify on)"),
     "repeat_limit": (3, int, "identical tool calls with identical results before lcode steps in (0: never)"),
     "secret_check": (True, bool, "before git commit and git push, look for passwords, keys and tokens they'd add"),
+    "compact_at": (65536, int, "summarize at this many tokens even when the window is larger (0: only at 85%)"),
     "max_parallel_agents": (1, int, "subagents that may run at the same time (more needs OLLAMA_NUM_PARALLEL)"),
 }
 ENV_OVERRIDES = {
@@ -81,6 +82,19 @@ class ConfigError(ValueError):
     pass
 
 
+MIN_COMPACT_AT = 16384  # below this, the system prompt and a summary alone come close to the ceiling
+
+
+def parse_compact_at(value: str | int) -> int:
+    """compact_at: a token count like parse_context's, at least 16K, or 0 (off)."""
+    if str(value).strip().lower() in ("0", "off", "false", "none"):
+        return 0
+    tokens = parse_context(value)
+    if tokens < MIN_COMPACT_AT:
+        raise ConfigError(f"compact_at {tokens} is too small: use at least 16k, or 0 to turn it off")
+    return tokens
+
+
 def parse_context(value: str | int) -> int:
     """Parse '131072', '128k', '128K' or '1m' into a token count (k = 1024)."""
     if isinstance(value, int):
@@ -118,6 +132,8 @@ def coerce(key: str, value: object) -> object:
     _, typ, _ = SETTINGS[key]
     if key == "context":
         return parse_context(value)  # type: ignore[arg-type]
+    if key == "compact_at":
+        return parse_compact_at(value)  # type: ignore[arg-type]
     if key in ("ollama_host", "searxng_url"):
         return normalize_host(str(value))
     if key == "base_url":