From 7d77fb8f407a0c4df809dcb12ba1e9b9740ea358 Mon Sep 17 00:00:00 2001 From: Sergiy Dybskiy Date: Fri, 2 Oct 2026 16:52:34 -0400 Subject: [PATCH] Ship the Lab 4 regression as a pull request and let a generic automation fix it Lab 4 Step 4 now merges the regression branch into main and tags the release from main. Step 5 describes the Cursor automation "Sentry issue to fix PR": a webhook fed by a Sentry internal integration, the Sentry MCP as a tool, and a prompt that maps the project to the app, diffs the last two releases, and opens one pull request against main. The prompt sheet step number now matches the lab. Co-authored-by: Claude --- PROMPTS.md | 4 +-- apps/slack-agent/regressions/README.md | 4 +++ labs/04-slack-agent.md | 35 ++++++++++++++++++++++---- 3 files changed, 36 insertions(+), 7 deletions(-) diff --git a/PROMPTS.md b/PROMPTS.md index 7b751da..30d9275 100644 --- a/PROMPTS.md +++ b/PROMPTS.md @@ -118,7 +118,7 @@ name, the conversation id, the user, and every gen_ai and tool span with its token counts. ``` -### Step 3. Ship the regression +### Step 4. Ship the regression ```text Compare cached input tokens per model call between the last two releases of @@ -155,7 +155,7 @@ Find the latest trace in the pr-reviewer project. Show the lead span, the two subagent spans, and the total input tokens for the run. ``` -### Step 3. Ship the regression +### Step 4. Ship the regression ```text Compare the last two releases of the pr-reviewer project: total input diff --git a/apps/slack-agent/regressions/README.md b/apps/slack-agent/regressions/README.md index 0e3b2ff..4e9a600 100644 --- a/apps/slack-agent/regressions/README.md +++ b/apps/slack-agent/regressions/README.md @@ -7,3 +7,7 @@ Patches that reintroduce a bug for a lab exercise. every turn. Apply from the repository root with `git apply apps/slack-agent/regressions/drop-prompt-cache.patch`, so `git apply` finds the `apps/slack-agent/...` paths. + +In the presenter flow, the same change ships as a pull request from the +branch `lab4-drop-prompt-cache` into `main`, and the Cursor automation +"Sentry issue to fix PR" restores it with a pull request of its own. diff --git a/labs/04-slack-agent.md b/labs/04-slack-agent.md index c0003d3..5706e46 100644 --- a/labs/04-slack-agent.md +++ b/labs/04-slack-agent.md @@ -102,21 +102,46 @@ Ask your agent: ## Step 4. Ship the regression -The presenter tags a release and applies -`apps/slack-agent/regressions/drop-prompt-cache.patch`, then restarts the bot -and sends three more messages in the same thread. +The regression ships the way regressions ship: as a merged pull request. The +presenter merges the branch `lab4-drop-prompt-cache` into `main`, tags a +release from the new `main` commit, restarts the bot on it, and sends three +more messages in the same thread. Working alone, apply +`apps/slack-agent/regressions/drop-prompt-cache.patch` instead and tag a +release from your own commit. Ask: > Compare cached input tokens per model call between the last two releases of > the slack-agent project. -Before the patch, the second and later turns in a thread show cached input -tokens. After it, they show zero, because the patch removes the cache +Before the regression, the second and later turns in a thread show cached +input tokens. After it, they show zero, because the change removes the cache breakpoint on the system prompt: Anthropic no longer has anything to match against. The cost per conversation rises several times. This is the regression from the talk, now in your own project. +## Step 5. Let an agent fix it + +The monitor "Cache misses on model calls" counts model calls that read zero +cached tokens and opens an issue when it sees more than two in ten minutes. +That issue is the trigger for a Cursor automation named "Sentry issue to fix +PR". The automation is generic on purpose, so you can copy it: + +- Trigger: a Sentry internal integration posts every issue event in the org + to the automation's webhook. If your Cursor account is connected to your + Sentry org, use Cursor's own Sentry trigger instead and pick the project. +- Repository and branch: this repo, `main`. +- Tools: Open Pull Request and the Sentry MCP, so the agent reads the issue + and the trace the way you did in Lab 3. +- Prompt: act only on created issues. Read the issue and the release it first + appeared in. Map the Sentry project to the app directory. Diff that release + against the previous one for the same app. Make the smallest fix. Open one + pull request against `main` that links the issue. Never merge. Stop if a + pull request for the issue already exists or the cause is unclear. + +The pull request it opens restores the one line. A human reviews and merges +it, and `main` is healthy again. + ## What you learned - The prompt names the unit of work. In Slack it is one message, so the