fix(ai): distinguish automatic prompt-cache lifetimes - #835
Draft
Altairpaca wants to merge 3 commits into
Draft
Conversation
Direct DeepSeek (openai-completions) now classifies as automatic caching with no client-visible TTL; resolvePromptCacheTtlSeconds() becomes a backwards-compatible wrapper over the new resolvePromptCacheLifetime(). Fixes code-yeongyu#831
Automatic-cache providers (direct DeepSeek) schedule Goal monitor continuations at the configured promptCache.goalBackstopMaxSeconds liveness backstop instead of 270s/240s cache-preservation wakes, and cache-warm entries no longer claim TTL-warmth or savings for them.
settings.md gains the promptCache.goalBackstopMaxSeconds row and names automatic-cache providers as budget-free; providers.md's goal-monitor timing note describes the liveness-backstop scheduling for lanes like direct DeepSeek.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Direct DeepSeek API sessions are currently treated as if they had a deterministic 5-minute prompt-cache TTL. That propagates into Goal monitor scheduling (a continuation every 270s — 5m minus the default 30s safety buffer) and into TUI copy claiming the wake stays inside a "5m prompt-cache TTL" and keeps tokens warm with estimated savings.
DeepSeek's official context cache is automatic and best-effort: enabled by default for all users, no client-visible TTL contract, and unused entries are cleared only after hours to days (api-docs.deepseek.com/guides/kv_cache). This PR fixes the cache-lifetime classification rather than tuning DeepSeek-specific numbers.
Root cause
resolvePromptCacheTtlSeconds()(packages/ai/src/utils/prompt-cache-ttl.ts) returned 300s for everyopenai-completionsmodel without Anthropic-style cache control — including direct DeepSeek. PR #767 then consumed that budget:resolvePromptCacheSafeWaitSeconds()derived 270s, and the Goal monitor scheduled its continuation there. Returningundefinedalone would not fix it: Goal maps unknown budgets to the legacy 240s fallback, which is still a cache-preservation framing.Behavior before/after
Before (direct DeepSeek, Goal with a live wake source):
300s(fabricated 5m TTL)4m 30sAfter:
PromptCacheLifetimesemantic:fixed(ttlSeconds) | automatic | disabled | unknown, resolved byresolvePromptCacheLifetime();resolvePromptCacheTtlSeconds()becomes a backwards-compatible wrapper (public signature unchanged, every non-DeepSeek result identical).automatic— no fixed TTL (300 or 3600) is ever reported, including underPI_CACHE_RETENTION=long.promptCache.goalBackstopMaxSeconds, default 3570s → 59m30s wake) instead of the 270s/240s cache-preservation wakes.cacheLifetime: "automatic", omitttlSeconds/estimatedSavedUsd, and render neutrally: "Continuation deferred 59m 30s - provider caching is automatic; the timed wake only keeps the goal alive." / "~9.5M tokens cached after the prior turn".RED -> GREEN
All captures under
local-ignore/qa-evidence/20260812-issue-831-deepseek-cache-lifetime/:red-01-ai-prompt-cache-ttl.txt: 6 failing new cases —resolvePromptCacheLifetime is not a function(API contract absent)red-02-prompt-cache-budget.txt:expected 270 to be undefined— DeepSeek had a 270s budgetred-03-issue-831-regression.txt: 4 failing — 240s fallback delay; "~120K tokens stayed warm in the prompt cache"green-01..06: ai TTL 38/38, ai adjacent 35 (+4 pre-existing key-gated skips), prompt-cache budget 11/11, issue-831 regression 4/4, Goal cache-warm/metrics/renderer/monitor suites 51/51, cache-keepalive 7/7 (untouched)Verification
npm run check— Biome + tsc + browser-smoke (covers the new pi-ai root export)npm test— full workspace suitegit diff --checkclean; packageCHANGELOG.mduntouched (maintainer-owned per CONTRIBUTING.md); requiredchanges.mdentries added (packages/ai/src, coding-agent core, goal)Scope/non-goals
openai-completionslanes stay 300s).cacheRetention: "long",openai-completionsbuildParamsmay sendprompt_cache_keyandprompt_cache_retention: "24h"to DeepSeek (its compat allows long retention). Default short retention sends neither field. This deserves its own focused follow-up.Evidence
local-ignore/qa-evidence/20260812-issue-831-deepseek-cache-lifetime/— RED and GREEN vitest captures, npm check/test receipts, senpi-qa channel outputs. No credentials or environment dumps are included.Fixes #831
Summary by cubic
Classifies provider prompt-cache lifetimes and treats direct DeepSeek as automatic (no fixed TTL) to stop 4m30s “cache-preservation” wakes. Goal monitor now uses the liveness backstop for DeepSeek, and UI copy no longer claims TTL warmth or savings. Fixes #831.
Bug Fixes
resolvePromptCacheLifetimeandPromptCacheLifetime(fixed|automatic|disabled|unknown) in@earendil-works/pi-ai;resolvePromptCacheTtlSecondsstays as a wrapper for fixed TTLs.openai-completionswithprovider: "deepseek"or adeepseek.combase URL) is classified asautomatic; no 300s/3600s TTL is reported, even withPI_CACHE_RETENTION=long.promptCache.goalBackstopMaxSeconds(default 3570s → 59m30s) instead of 270s/240s; fixed-TTL lanes keep existing timing.cacheLifetime: "automatic"and omitttlSecondsandestimatedSavedUsd; renderer explains the wake as a liveness backstop and reports cached tokens neutrally.Migration
resolvePromptCacheLifetimeandPromptCacheLifetimefrom@earendil-works/pi-ai.promptCache.goalBackstopMaxSeconds; cache-aware budgets are skipped for automatic-cache lanes.Written for commit 6a7bb25. Summary will update on new commits.