Give the Slack agent a Postgres analytics warehouse and finish its Sentry lab

PROMPTS.md

@@ -92,16 +92,19 @@ Directory: `apps/slack-agent`. Full lab: [labs/04-slack-agent.md](labs/04-slack-
 ```text
 Instrument this Node.js Slack bot with Sentry. Create a new Sentry project
 for it in my organization. I need:
-- Agent tracing for the Vercel AI SDK call in `src/agent.ts`, with inputs
-and outputs recorded.
+- Agent tracing for the Vercel AI SDK call in `src/agent.ts`.
 - One transaction per Slack message handled in `src/app.ts`, so each
-message shows as one trace with the model calls and tool calls inside it.
+message shows as one trace with the model call and its tool calls inside it.
 - The Slack thread timestamp set as the conversation id, and the Slack
 user id set as the Sentry user.
-- Database spans for the Postgres queries under `../storefront/lib/db`.
-- Full trace sampling for the workshop.
-Use the 11.0 release candidate of the Sentry SDK, npm tag `next`. Add only
-what these points need.
+- Prompts and replies recorded only when the message is in a public or
+private channel, never in a direct message.
+- One span per warehouse query in `src/analytics/tools.ts`, tagged with the
+metric name, the date range length, and the row count, so a direct message
+trace still shows something useful without the prompt and the reply.
+- The release read from `SENTRY_RELEASE`, and full trace sampling for the
+workshop.
+Use the 11.0 release of the Sentry SDK. Add only what these points need.
 The process starts with `npm run dev`. When done, start it, wait for the
 "connected over Socket Mode" line, and tell me how to confirm the first
 trace arrived.

README.md

@@ -23,7 +23,7 @@ each lab names the one that did the work.
 | App | Environment | Runtime | You do |
 | --- | --- | --- | --- |
 | `apps/storefront` | Browser chat on a Next.js app | Next.js, Vercel AI SDK, Postgres on Neon | Hands-on |
-| `apps/slack-agent` | Long-lived Slack bot | Node, Bolt (Socket Mode), Vercel AI SDK, same Postgres | Follow along |
+| `apps/slack-agent` | Long-lived Slack bot: product analytics assistant over a Postgres warehouse (Neon) | Node, Bolt (Socket Mode), Vercel AI SDK | Follow along |
 | `apps/pr-reviewer` | GitHub Actions job | Node, Flue (OpenTelemetry), OpenRouter | Follow along |
 
 All three call Claude Sonnet 5 through OpenRouter: the storefront and the
@@ -42,7 +42,7 @@ Do these once, before the day. Each takes a few minutes. See
    workshop, so no account is needed.
 3. A coding agent with Sentry's MCP server attached. Claude Code users install
    the `sentry` plugin. Other agents add <https://mcp.sentry.dev/mcp>.
-4. Node 22 and git.
+4. Node 22.18 or newer, and git.
 
 Then:
 

RUNSHEET.md

@@ -78,26 +78,39 @@ applied with `scripts/solution.sh <app>`. Time on the day: 55 minutes.
   three levels up. Tool errors the model recovers from still deserve a
   record, because silent recovery is how agents hide failures.
 
-## Lab 4. The same agent in Slack (10 minutes, follow-along)
+## Lab 4. A different agent in Slack (10 minutes, follow-along)
 
 - **Outcome.** One trace per Slack message: a root span with the thread
   timestamp as conversation id and the Slack user as Sentry user, the model
-  call and tool spans inside. After the regression patch, cached input
-  tokens on later turns drop from thousands to zero.
-- **Prompt.** The lab 4 prompt (no session replay, no logs; the trace is all
-  you get), then "Find the latest trace in the slack-agent project ...",
-  then "Compare cached input tokens per model call between the last two
-  releases".
-- **Code.** `src/instrument.ts`: six lines, imported first. `src/app.ts`: the
+  call and its `analytics.scan` tool spans inside, each with a `db` child
+  span for the Postgres query. A channel message records the prompt and the
+  reply; a direct message never does, but its scan spans still carry the
+  metric, the range length, and the row count. After the regression patch,
+  cached input tokens on later turns drop from thousands to zero.
+- **Prompt.** The lab 4 instrument prompt (one transaction per message,
+  content recorded only outside direct messages, one span per warehouse
+  scan), then "Find the latest trace in the slack-agent project ...", then
+  the same question asked in a channel and in a DM, then "Compare cached
+  input tokens per model call between the last two releases".
+- **Code.** `src/instrument.ts`: Sentry init plus the `dataCollection` block
+  that turns content recording off by default. `src/app.ts`: the
   `Sentry.startSpan` wrapper around one message with
-  `gen_ai.conversation.id` and `setUser`. Then the regression:
+  `gen_ai.conversation.id`, `setUser`, and the `slack.record_content`
+  attribute. `src/agent.ts`: the per-call `telemetry` override that follows
+  the same channel-vs-DM rule. `src/analytics/tools.ts`: one
+  `Sentry.startSpan` per warehouse scan. Then the regression:
   `regressions/drop-prompt-cache.patch` removes one line, the
   `providerOptions` cache breakpoint on the system prompt.
-- **Sentry UI.** The trace. Explore > Spans, query
+- **Sentry UI.** The channel trace next to the DM trace for the same
+  question: one shows the prompt, the reply, and the tool arguments; the
+  other shows only the scan spans' attributes and their `db` child spans.
+  Then Explore > Spans, query
   `gen_ai.usage.cache_read.input_tokens` grouped by release: one release
   shows 3114, the next shows 0.
 - **Takeaway.** The prompt names the unit of work. Conversation id and user
-  come from the platform. A cache regression is one attribute per release.
+  come from the platform. Content recording can follow a rule the platform
+  already gives you, such as channel type. A cache regression is one
+  attribute per release.
 
 ## Lab 5. The same agent in GitHub Actions (10 minutes, follow-along)
 

apps/slack-agent/.env.example

@@ -8,9 +8,17 @@ SLACK_APP_TOKEN=
 # OpenRouter API key — shared key from the presenter
 OPENROUTER_API_KEY=
 
-# Copy from apps/storefront/.env.local - the bot reads the same database.
+# Same Neon connection string as apps/storefront/.env.local
 DATABASE_URL=
 
-# Optional. One of the SUPPORTED_MODELS in ../storefront/lib/ai/models.ts.
-# Defaults to anthropic/claude-sonnet-5.
+# Optional. One of anthropic/claude-sonnet-5, anthropic/claude-haiku-4.5,
+# openai/gpt-5-mini, google/gemini-2.5-flash. Defaults to
+# anthropic/claude-sonnet-5.
 # OPENROUTER_MODEL=anthropic/claude-sonnet-5
+
+# Sentry DSN for this project (Project Settings > Client Keys)
+SENTRY_DSN=
+# Optional. Defaults to "development".
+SENTRY_ENVIRONMENT=
+# Optional. Tags every span and error with the release that produced it.
+SENTRY_RELEASE=

apps/slack-agent/README.md

@@ -1,46 +1,96 @@
 # slack-agent
 
-A Slack bot that answers shopping questions in a DM or when mentioned. It
-reuses the storefront's tools (`searchProducts`, `getProduct`,
-`getAccountInfo`, `refundOrder`) and the same Postgres database, so it can
-search the catalog and manage the same demo customer's orders. It reads
-the whole thread on every message, so follow-ups like "refund that order"
-work.
+A Slack bot that answers product analytics questions in a DM or when
+mentioned. It reads from `analytics_daily`, a Postgres table on Neon with
+ninety days of history, seeded from a deterministic generator in
+`src/analytics/generate.ts`. It reads the whole thread on every message, so
+follow-ups like "and by device?" work.
+
+## What it can answer
+
+Four tools back every answer:
+
+- `list_metrics` - the metric catalog: `signups`, `active_users`,
+  `page_views`, `checkout_conversion`, `error_rate`, each with its unit and
+  the dimensions it can be grouped by.
+- `query_metric` - one metric over a date range, with an optional
+  `groupBy` for a per-dimension breakdown.
+- `compare_periods` - one metric's total in a current range against a
+  previous range, with the absolute and percent change.
+- `top_movers` - which values of a dimension moved the most between the
+  first and second half of a date range.
+
+The system prompt tells the assistant to call a tool before stating any
+number, name the metric and the exact date range it used, and compare
+against the previous period of the same length when asked how something
+"did".
+
+## Data collection
+
+The assistant records its prompts and replies only when the message came
+from a public or private channel - never from a direct message. A DM still
+produces a trace: the root `slack.message` span and, for every warehouse
+query, an `analytics.scan` span with the metric name, the range length, and
+the row count, with a `db` span inside it for the Postgres query, none of
+which need the prompt or the reply to be useful.
 
 ## Setup
 
-1. From the repo root, run `scripts/setup.sh` first. It installs the
-   storefront, provisions a Postgres database, and seeds it.
-2. `npm run manifest:link` and open the printed URL in a Slack Developer
+1. `npm run manifest:link` and open the printed URL in a Slack Developer
    Program sandbox to create the app from this app's manifest.
-3. Install the app to your workspace.
-4. On the app's Basic Information page, create an app-level token with the
+2. Install the app to your workspace.
+3. On the app's Basic Information page, create an app-level token with the
    `connections:write` scope.
-5. Copy `.env.example` to `.env.local`. Fill in `SLACK_BOT_TOKEN` (from OAuth
-   & Permissions) and `SLACK_APP_TOKEN` (the token from step 4), plus
-   `OPENROUTER_API_KEY` and the `DATABASE_URL` from
-   `apps/storefront/.env.local`.
+4. Copy `.env.example` to `.env.local`. Fill in `SLACK_BOT_TOKEN` (from OAuth
+   & Permissions), `SLACK_APP_TOKEN` (the token from step 3), and
+   `OPENROUTER_API_KEY`.
+5. Paste the storefront's `DATABASE_URL` (from `apps/storefront/.env.local`)
+   into `.env.local`.
 6. `npm install`
-7. `npm run dev`
-8. DM the bot: "show me the apparel collection".
+7. `npm run db:seed`. The warehouse ends on the day you seed, so run this
+   again on the workshop day.
+8. `npm run dev`
+9. DM the bot: "How did signups do last week compared with the week before?"
 
 ## What the agent does in Slack
 
 - It shows a status message while it works.
 - It streams the reply as it writes it.
 - It shows each tool call as a task in a timeline.
-- It adds a card for each product that the reply names. The button on a card
-  asks the agent about that product in the same thread.
-- It adds a table row for each order that the reply names.
-- It adds a warning line when a tool call fails, for example a refund.
+- It adds a warning line when a tool call fails, for example a query outside
+  the 90-day retention window.
 - It names each thread after the first message.
 - It stops the model call when the user presses Slack's stop button.
 - It adds feedback buttons under each reply.
 - In a DM, the Messages tab shows four suggested prompts from the manifest.
 
+Every reply ends with a chart card and a table built from that turn's tool
+results. QuickChart renders the chart from a URL. Slack fetches the image.
+The bot makes no request. A URL over 3000 characters drops the chart, so
+long ranges are bucketed by week.
+
 Note: the manifest uses Slack's agent messaging experience (`agent_view`). An
 app created from an older manifest must have its manifest replaced on the App
 Manifest page at api.slack.com, then be reinstalled. Slack does not let an
 app go back to the older assistant experience.
 
-Sentry is added to this app in `labs/04-slack-agent.md`.
+## Sentry
+
+Set `SENTRY_DSN` in `.env.local` (`SENTRY_ENVIRONMENT` and `SENTRY_RELEASE`
+are optional). Each handled Slack message becomes one trace: a root
+`slack.message` span tagged with the Slack channel, the thread timestamp as
+the conversation id, and whether content recording is on for that channel,
+with the model call and its tool calls nested inside as `gen_ai` spans and
+one `analytics.scan` span per warehouse query, each with a `db` child span
+from the `pg` integration. The dev script starts Node with
+`--import ./src/instrument.ts`, so Sentry loads before `pg`. A runner that
+compiles TypeScript itself, such as tsx, skips that hook and shows no db
+spans. The Sentry user is the Slack user who sent the message. Errors go to
+Sentry as issues.
+
+## Regression
+
+`regressions/drop-prompt-cache.patch` removes the cache breakpoint from the
+system prompt, so Anthropic caches nothing and cached input tokens fall to
+zero on every turn. Apply from the repository root with
+`git apply apps/slack-agent/regressions/drop-prompt-cache.patch`.