Skip to content

fix(ai): distinguish automatic prompt-cache lifetimes - #835

Draft
Altairpaca wants to merge 3 commits into
code-yeongyu:mainfrom
Altairpaca:fix/831-deepseek-cache-lifetime
Draft

fix(ai): distinguish automatic prompt-cache lifetimes#835
Altairpaca wants to merge 3 commits into
code-yeongyu:mainfrom
Altairpaca:fix/831-deepseek-cache-lifetime

Conversation

@Altairpaca

@Altairpaca Altairpaca commented Aug 12, 2026

Copy link
Copy Markdown

Summary

Direct DeepSeek API sessions are currently treated as if they had a deterministic 5-minute prompt-cache TTL. That propagates into Goal monitor scheduling (a continuation every 270s — 5m minus the default 30s safety buffer) and into TUI copy claiming the wake stays inside a "5m prompt-cache TTL" and keeps tokens warm with estimated savings.

DeepSeek's official context cache is automatic and best-effort: enabled by default for all users, no client-visible TTL contract, and unused entries are cleared only after hours to days (api-docs.deepseek.com/guides/kv_cache). This PR fixes the cache-lifetime classification rather than tuning DeepSeek-specific numbers.

Root cause

resolvePromptCacheTtlSeconds() (packages/ai/src/utils/prompt-cache-ttl.ts) returned 300s for every openai-completions model without Anthropic-style cache control — including direct DeepSeek. PR #767 then consumed that budget: resolvePromptCacheSafeWaitSeconds() derived 270s, and the Goal monitor scheduled its continuation there. Returning undefined alone would not fix it: Goal maps unknown budgets to the legacy 240s fallback, which is still a cache-preservation framing.

Behavior before/after

Before (direct DeepSeek, Goal with a live wake source):

  • TTL estimate: 300s (fabricated 5m TTL)
  • Goal monitor wake: every 4m 30s
  • TUI: "Continuation deferred 4m 30s - the timed wake stays inside the 5m prompt-cache TTL." / "~9.5M tokens kept warm · est. $1.31 saved vs a cold re-read"

After:

  • New browser-safe PromptCacheLifetime semantic: fixed(ttlSeconds) | automatic | disabled | unknown, resolved by resolvePromptCacheLifetime(); resolvePromptCacheTtlSeconds() becomes a backwards-compatible wrapper (public signature unchanged, every non-DeepSeek result identical).
  • Direct built-in DeepSeek classifies as automatic — no fixed TTL (300 or 3600) is ever reported, including under PI_CACHE_RETENTION=long.
  • Goal schedules automatic-cache lanes at the configured liveness backstop (promptCache.goalBackstopMaxSeconds, default 3570s → 59m30s wake) instead of the 270s/240s cache-preservation wakes.
  • Cache-warm entries for automatic lanes carry cacheLifetime: "automatic", omit ttlSeconds/estimatedSavedUsd, and render neutrally: "Continuation deferred 59m 30s - provider caching is automatic; the timed wake only keeps the goal alive." / "~9.5M tokens cached after the prior turn".
  • Fixed-TTL lanes (Anthropic 300/3600, Bedrock, OpenRouter cache-control, other openai-completions) keep their exact existing scheduling, metrics, and rendering.

RED -> GREEN

All captures under local-ignore/qa-evidence/20260812-issue-831-deepseek-cache-lifetime/:

  • RED red-01-ai-prompt-cache-ttl.txt: 6 failing new cases — resolvePromptCacheLifetime is not a function (API contract absent)
  • RED red-02-prompt-cache-budget.txt: expected 270 to be undefined — DeepSeek had a 270s budget
  • RED red-03-issue-831-regression.txt: 4 failing — 240s fallback delay; "~120K tokens stayed warm in the prompt cache"
  • GREEN green-01..06: ai TTL 38/38, ai adjacent 35 (+4 pre-existing key-gated skips), prompt-cache budget 11/11, issue-831 regression 4/4, Goal cache-warm/metrics/renderer/monitor suites 51/51, cache-keepalive 7/7 (untouched)

Verification

  • npm run check — Biome + tsc + browser-smoke (covers the new pi-ai root export)
  • npm test — full workspace suite
  • senpi-qa real-CLI/local-mock channels: mock-loop self-test, cli-smoke self-test, rpc-drive — evidence in the same directory
  • git diff --check clean; package CHANGELOG.md untouched (maintainer-owned per CONTRIBUTING.md); required changes.md entries added (packages/ai/src, coding-agent core, goal)

Scope/non-goals

  • Classification change is limited to direct built-in DeepSeek; every other lane keeps its previous conservative classification (remaining openai-completions lanes stay 300s).
  • The cache-keepalive extension is untouched (Anthropic-first-party-only, out of scope).
  • Separate finding, deliberately NOT fixed here: with cacheRetention: "long", openai-completions buildParams may send prompt_cache_key and prompt_cache_retention: "24h" to DeepSeek (its compat allows long retention). Default short retention sends neither field. This deserves its own focused follow-up.

Evidence

local-ignore/qa-evidence/20260812-issue-831-deepseek-cache-lifetime/ — RED and GREEN vitest captures, npm check/test receipts, senpi-qa channel outputs. No credentials or environment dumps are included.

Fixes #831


Summary by cubic

Classifies provider prompt-cache lifetimes and treats direct DeepSeek as automatic (no fixed TTL) to stop 4m30s “cache-preservation” wakes. Goal monitor now uses the liveness backstop for DeepSeek, and UI copy no longer claims TTL warmth or savings. Fixes #831.

  • Bug Fixes

    • Added resolvePromptCacheLifetime and PromptCacheLifetime (fixed|automatic|disabled|unknown) in @earendil-works/pi-ai; resolvePromptCacheTtlSeconds stays as a wrapper for fixed TTLs.
    • Direct DeepSeek (openai-completions with provider: "deepseek" or a deepseek.com base URL) is classified as automatic; no 300s/3600s TTL is reported, even with PI_CACHE_RETENTION=long.
    • Goal monitor schedules automatic-cache lanes at promptCache.goalBackstopMaxSeconds (default 3570s → 59m30s) instead of 270s/240s; fixed-TTL lanes keep existing timing.
    • Cache-warm metrics add cacheLifetime: "automatic" and omit ttlSeconds and estimatedSavedUsd; renderer explains the wake as a liveness backstop and reports cached tokens neutrally.
    • Scope limited to direct DeepSeek; all other providers keep prior behavior.
  • Migration

    • No changes required.
    • To use the new semantics, import resolvePromptCacheLifetime and PromptCacheLifetime from @earendil-works/pi-ai.
    • If shorter DeepSeek wakes are desired, tune promptCache.goalBackstopMaxSeconds; cache-aware budgets are skipped for automatic-cache lanes.

Written for commit 6a7bb25. Summary will update on new commits.

Review in cubic

Direct DeepSeek (openai-completions) now classifies as automatic caching
with no client-visible TTL; resolvePromptCacheTtlSeconds() becomes a
backwards-compatible wrapper over the new resolvePromptCacheLifetime().

Fixes code-yeongyu#831
Automatic-cache providers (direct DeepSeek) schedule Goal monitor
continuations at the configured promptCache.goalBackstopMaxSeconds
liveness backstop instead of 270s/240s cache-preservation wakes, and
cache-warm entries no longer claim TTL-warmth or savings for them.
settings.md gains the promptCache.goalBackstopMaxSeconds row and names
automatic-cache providers as budget-free; providers.md's goal-monitor
timing note describes the liveness-backstop scheduling for lanes like
direct DeepSeek.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(prompt-cache): direct DeepSeek is treated as a fixed 5m TTL, forcing unnecessary 4m30 Goal cache-warm wakes

1 participant