diff --git a/PROMPTS.md b/PROMPTS.md index 7b751da..30d9275 100644 --- a/PROMPTS.md +++ b/PROMPTS.md @@ -118,7 +118,7 @@ name, the conversation id, the user, and every gen_ai and tool span with its token counts. ``` -### Step 3. Ship the regression +### Step 4. Ship the regression ```text Compare cached input tokens per model call between the last two releases of @@ -155,7 +155,7 @@ Find the latest trace in the pr-reviewer project. Show the lead span, the two subagent spans, and the total input tokens for the run. ``` -### Step 3. Ship the regression +### Step 4. Ship the regression ```text Compare the last two releases of the pr-reviewer project: total input diff --git a/apps/slack-agent/regressions/README.md b/apps/slack-agent/regressions/README.md index 0e3b2ff..4e9a600 100644 --- a/apps/slack-agent/regressions/README.md +++ b/apps/slack-agent/regressions/README.md @@ -7,3 +7,7 @@ Patches that reintroduce a bug for a lab exercise. every turn. Apply from the repository root with `git apply apps/slack-agent/regressions/drop-prompt-cache.patch`, so `git apply` finds the `apps/slack-agent/...` paths. + +In the presenter flow, the same change ships as a pull request from the +branch `lab4-drop-prompt-cache` into `main`, and the Cursor automation +"Sentry issue to fix PR" restores it with a pull request of its own. diff --git a/labs/04-slack-agent.md b/labs/04-slack-agent.md index c0003d3..5706e46 100644 --- a/labs/04-slack-agent.md +++ b/labs/04-slack-agent.md @@ -102,21 +102,46 @@ Ask your agent: ## Step 4. Ship the regression -The presenter tags a release and applies -`apps/slack-agent/regressions/drop-prompt-cache.patch`, then restarts the bot -and sends three more messages in the same thread. +The regression ships the way regressions ship: as a merged pull request. The +presenter merges the branch `lab4-drop-prompt-cache` into `main`, tags a +release from the new `main` commit, restarts the bot on it, and sends three +more messages in the same thread. Working alone, apply +`apps/slack-agent/regressions/drop-prompt-cache.patch` instead and tag a +release from your own commit. Ask: > Compare cached input tokens per model call between the last two releases of > the slack-agent project. -Before the patch, the second and later turns in a thread show cached input -tokens. After it, they show zero, because the patch removes the cache +Before the regression, the second and later turns in a thread show cached +input tokens. After it, they show zero, because the change removes the cache breakpoint on the system prompt: Anthropic no longer has anything to match against. The cost per conversation rises several times. This is the regression from the talk, now in your own project. +## Step 5. Let an agent fix it + +The monitor "Cache misses on model calls" counts model calls that read zero +cached tokens and opens an issue when it sees more than two in ten minutes. +That issue is the trigger for a Cursor automation named "Sentry issue to fix +PR". The automation is generic on purpose, so you can copy it: + +- Trigger: a Sentry internal integration posts every issue event in the org + to the automation's webhook. If your Cursor account is connected to your + Sentry org, use Cursor's own Sentry trigger instead and pick the project. +- Repository and branch: this repo, `main`. +- Tools: Open Pull Request and the Sentry MCP, so the agent reads the issue + and the trace the way you did in Lab 3. +- Prompt: act only on created issues. Read the issue and the release it first + appeared in. Map the Sentry project to the app directory. Diff that release + against the previous one for the same app. Make the smallest fix. Open one + pull request against `main` that links the issue. Never merge. Stop if a + pull request for the issue already exists or the cause is unclear. + +The pull request it opens restores the one line. A human reviews and merges +it, and `main` is healthy again. + ## What you learned - The prompt names the unit of work. In Slack it is one message, so the