Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions PROMPTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,14 @@ as a handled tool error with the order id as an attribute. Then rerun the
refund in the chat and show me the new trace.
```

### Step 4. Triage it

```text
Triage the refund issue: assign it to me, run Seer on it and compare its
root cause with yours, then resolve it and reference it in the fix commit
so Sentry links the two.
```

## Lab 4. Slack agent

Directory: `apps/slack-agent`. Full lab: [labs/04-slack-agent.md](labs/04-slack-agent.md).
Expand Down Expand Up @@ -173,3 +181,12 @@ org with two widgets: total input and output tokens per day grouped by
tokens with their user. Then open it.
```

### Step 5. The alert fired

```text
The refund alert fired. Open the issue it points to, tell me how many
users hit it and in which release, get Seer's root cause, and if it is
the Lab 3 bug again, assign it to me and link the fix. Otherwise tell me
what is new.
```

11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,17 @@ that matter for each critical path.
Nobody instruments by hand. Every step is a prompt. The workshop teaches what
to ask for and how to check the result.

## Three ways in

Your coding agent reaches Sentry in three ways. The labs use all three, and
each lab names the one that did the work.

| Way | What it is | Where the labs use it |
|---|---|---|
| MCP server | `mcp.sentry.dev`. Tools such as `search_issues`, `search_events`, `get_trace_details`, `analyze_issue_with_seer`, and `update_issue`. The agent calls them while it works. | Lab 3 finds the issue, pulls the trace and the replay, and triages it. Lab 6 triages the issue an alert opens. |
| Plugin skills | `sentry-instrument`, `sentry-debug-issue`, `sentry-create-alert`, and others from <https://github.com/getsentry/plugin-claude>. Step-by-step instructions the agent follows on top of the MCP tools. Claude Code gets them from the `sentry` plugin. Other agents can copy them. | Labs 2 and 4 instrument. Lab 3 debugs. Lab 6 creates alerts. |
| `sentry` CLI | `sentry issue list`, `view`, `explain`, `plan`, `resolve`; `sentry agent-conversation list`, `view`; `sentry alert`; `sentry dashboard create`. Runs in any terminal and in CI, and the agent can call it too. | Lab 3 shows the same issue from the terminal. Lab 6 creates the dashboard and lists the alerts. |

| App | Environment | Runtime | You do |
| --- | --- | --- | --- |
| `apps/storefront` | Browser chat on a Next.js app | Next.js, Vercel AI SDK, Postgres on Neon | Hands-on |
Expand Down
31 changes: 26 additions & 5 deletions RUNSHEET.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,16 @@ applied with `scripts/solution.sh <app>`. Time on the day: 55 minutes.

- **Outcome.** The agent finds the new issue, names the failing span and the
cause from the trace, and applies a fix that keeps the error visible as a
handled tool error with the order id. The rerun trace is green.
handled tool error with the order id. The rerun trace is green. Then it
triages the issue: assigned, Seer analysis compared, resolved with the fix
commit.
- **Mechanism.** Say the names out loud once. MCP: `search_issues`,
`get_trace_details`, `get_sentry_resource` for the replay,
`analyze_issue_with_seer`, `update_issue`; the `sentry-debug-issue` skill
is the script the agent follows. CLI: `sentry issue list`, `view --spans
all`, `explain`, `resolve`. Seer refuses a project with no connected
repository, so link the fork first or skip the Seer comparison. Debugging
is tool calls, and tool calls run from any agent and from CI.
- **Prompt.** In the chat: "Show me my recent orders", then "Refund order
1029" (1029 has no payment row; 1036 and 1042 refund fine). Then the two
lab 3 prompts: diagnose without changing code, then apply the fix.
Expand Down Expand Up @@ -142,7 +151,15 @@ applied with `scripts/solution.sh <app>`. Time on the day: 55 minutes.
10374603 (refund), 10374604 (failure rate), 10374605 (slow turns, p95 >
15 s), 10374606 (input tokens per hour, anomaly), dashboard 10117079 ("Storefront agent: critical paths", from `scripts/dashboards/agent-critical-paths.mjs`). Every detector was
accepted on the first POST once the payload copied an existing one.
- **Prompt.** The two lab 6 prompts, alert first, dashboard second.
- **Prompt.** The two lab 6 prompts, alert first, dashboard second. While
the room waits for the alert email, the Step 5 triage prompt on the issue
the alert opened.
- **Mechanism.** Alerts: the `sentry-create-alert` skill over the workflow
engine API; `sentry alert metrics list` shows the result. Dashboard:
`sentry dashboard create`. Triage: the lab 3 MCP tools again, started by
an alert. Name the automated version: a webhook action on the workflow, or
a scheduled agent over `sentry issue list --query "is:unresolved
firstSeen:-1h"`.
- **Code.** The detector and workflow payloads the agent sends (Sentry's
workflow engine, not the legacy alert-rules API) and the dashboard JSON
the `sentry` CLI posts. Reference dashboards live in
Expand All @@ -151,9 +168,13 @@ applied with `scripts/solution.sh <app>`. Time on the day: 55 minutes.
the new dashboard. Say why there is no per-conversation alert: the
detector API refuses `groupBy` ("Group by Metric Alerts feature must be
enabled"), so the top-ten table is the answer.
- **Pitfall.** Dashboard creation needs the `org:write` CLI scope;
`sentry auth login` alone gives 403 on `POST dashboards/`. Setup says
`sentry auth refresh --scope org:write`.
- **Pitfall.** A 403 from the CLI is almost always the account, not the
scopes. The `sentry auth login` default scope set covers every lab
(`event:write` for Seer, `alerts:write` for the detectors, `org:read` for
dashboards). Check `sentry auth status`. A 403 on every command means the
CLI is logged in as an account that is not a member of the org. Do not
pass `--scope` to `sentry auth refresh` to fix it. That flag replaces the
whole scope set with what you pass.
- **Takeaway.** Start from the path that costs money, not the metric the
tool offers. Alerts and dashboards are prompts too. The attribute names
are in the trace.
2 changes: 1 addition & 1 deletion labs/00-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ your organization name.

- Node 22 or newer: `node --version`
- git
- The `sentry` CLI: `curl https://cli.sentry.dev/install -fsS | bash`, then `sentry auth login` and `sentry auth refresh --scope org:write` (lab 6 creates dashboards; the default token cannot)
- The `sentry` CLI: `curl https://cli.sentry.dev/install -fsS | bash`, then `sentry auth login`. Pick the account that is a member of your Sentry org; `sentry auth status` shows which one is logged in.

## 4. Clone and run the setup script

Expand Down
42 changes: 40 additions & 2 deletions labs/03-debug-refund.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,19 @@ The cause: the payments table only has rows for orders placed after the June
2026 payments launch. Older orders have nothing to refund against, and the
tool throws instead of telling the model so.

### What did the work

Your agent used Sentry's MCP server. With the plugin installed, the
`sentry-debug-issue` skill was its script. `search_issues` found the issue,
`get_trace_details` pulled the span tree, and `get_sentry_resource` fetched
the replay. Ask the agent to list the Sentry tools it called and these
names come back.

The `sentry` CLI shows the same issue from a terminal, and works in CI:

sentry issue list --query "is:unresolved" <org>/<project>
sentry issue view --spans all <ISSUE-ID>

## Step 3. Decide the fix, then apply it

There are two fixes, and the trace tells you which one is right:
Expand All @@ -50,11 +63,34 @@ For a shop, the first is correct. Ask your agent:
> as a handled tool error with the order id as an attribute. Then rerun the
> refund in the chat and show me the new trace.

## Step 4. Triage it

The fix in the repo is half of the job. The issue in Sentry needs an owner,
a root cause on record, and a resolution tied to the fix. Ask your agent:

> Triage the refund issue: assign it to me, run Seer on it and compare its
> root cause with yours, then resolve it and reference it in the fix commit
> so Sentry links the two.

Watch for two MCP calls: `analyze_issue_with_seer` for the root cause and
`update_issue` for the assignee and the status. From a terminal, the same
two are:

sentry issue explain <ISSUE-ID>
sentry issue resolve <ISSUE-ID>

A commit message with `Fixes <ISSUE-ID>` in it does the last step for you.
Sentry resolves the issue when the release with that commit goes out.

Seer only runs on a project with a connected repository. If it answers
"requires repositories to be connected", link your fork under Project
Settings > Source Code, or skip the Seer comparison and keep the rest.

## Checkpoint

The new trace has the same tool span, now without an error, and the model's
reply names the reason. The issue from Step 1 can be resolved. Ask the agent
to resolve it and link the fix commit.
reply names the reason. The issue from Step 1 is resolved, assigned to you,
and shows the Seer analysis and the fix commit.

## What you learned

Expand All @@ -63,3 +99,5 @@ to resolve it and link the fix commit.
reproducing.
- Tool errors that the model recovers from still deserve to be recorded.
Silent recovery is how agents hide failures.
- Debugging is tool calls. The MCP tools and CLI commands that found and
triaged this issue run from any coding agent, and from CI.
27 changes: 25 additions & 2 deletions labs/06-critical-paths.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ Ask your coding agent:
> 5 minutes. Look at an existing detector in the org for the payload shape
> first. Show me both alerts and the query each one uses.

The Sentry skills use the workflow engine API for this. The legacy
The `sentry-create-alert` skill does this through Sentry's workflow engine
API: one detector per condition and one workflow for the email. The legacy
alert-rules endpoint fails on this org, and the detector payload needs
values no error message tells you (`events_analytics_platform`,
`trace_item_span`, the 75/50/0 condition codes), which is why the prompt
Expand All @@ -48,6 +49,10 @@ hour". The agent has to know `detectionType: dynamic` and the
`anomaly_detection` condition shape; the presenter's copy is in
`scripts/alerts/storefront-critical-paths.json`.

List what the agent created from a terminal:

sentry alert metrics list <org>

## Step 3. One dashboard by CLI

The Sentry plugin has no dashboard skill, so this goes through the `sentry`
Expand Down Expand Up @@ -76,14 +81,32 @@ failures, and model latency. Add `--push` to create it through the CLI.
- PR reviewer: the cache hit rate per release widget, with the Lab 5
regression visible as a drop between two releases.

## Step 5. The alert fired. Now what

An alert starts a triage loop. The refund alert points at an issue. Hand it
to your agent while you wait for the email:

> The refund alert fired. Open the issue it points to, tell me how many
> users hit it and in which release, get Seer's root cause, and if it is
> the Lab 3 bug again, assign it to me and link the fix. Otherwise tell me
> what is new.

This is Lab 3 again, started by an alert instead of a person:
`search_issues`, `get_trace_details`, `analyze_issue_with_seer`,
`update_issue`. Take the person out and the same loop runs on its own. The
alert's workflow can call a webhook, or a scheduled coding agent can run
this prompt over `sentry issue list --query "is:unresolved firstSeen:-1h"`.

## Checkpoint

Trigger the refund failure four times in ten minutes. The alert fires on
more than 3 failures. The alert email arrives a few minutes later. Paste
the alert link in the workshop channel.
the alert link in the workshop channel, then run Step 5 on it.

## What you learned

- Start from the path that costs money, not from the metric the tool offers.
- Alerts and dashboards are prompts too. The agent needs the attribute names,
and the trace has them.
- An alert opens a triage loop. The agent that debugged Lab 3 runs that loop
from the alert, and can run it from a schedule.
4 changes: 2 additions & 2 deletions solutions/02-storefront/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Solution: instrument the storefront

`instrumentation.patch` adds Sentry (`@sentry/nextjs` 11.0.0-rc.0) to
`instrumentation.patch` adds Sentry (`@sentry/nextjs` 11.0.0) to
`apps/storefront`, wired for agent tracing on the Vercel AI SDK route.

## Files it adds or changes
Expand All @@ -25,7 +25,7 @@
Sentry's Node SDK auto-instruments into `db.query` spans once the server
config is loaded.

SDK: @sentry/nextjs 11.0.0-rc.0
SDK: @sentry/nextjs 11.0.0

## Env vars it needs

Expand Down
Loading
Loading