Auto-apply high-confidence frontend-triage results and report them to Slack

[msujaws]

frontend-triage produces a root-cause analysis and fix plan and records it as Bugzilla actions — but nothing applied them, so every triage needed a human to click Apply in the hackbot UI, and nothing told the team a triage had happened at all.

This closes that loop for the results the agent is confident about. Three commits, each standing alone.

1. Auto-apply, gated on confidence

AgentSpec gains auto_apply_confidence: the set of findings.confidence levels whose actions may be applied unattended, layered on top of the existing auto_apply_actions switch. frontend-triage opts in at {"high"} only, so widening the policy later — to also accept medium — is an edit to that set rather than to the applier.

The decision is a pure predicate over (spec, run) so the policy is testable on its own, and it fails closed: a run whose findings carry no usable confidence never qualifies. Because confidence is parsed out of the agent's free-form JSON block, it's compared case- and whitespace-insensitively, so "High" doesn't silently mean "never apply".

Medium and low results are unchanged from today: still recorded, still visible in the UI, still appliable by hand. So are all the other agents — there's a test asserting that.

2. Tell the agent what its confidence rating now causes

confidence used to be advisory metadata a human read off the run. Now it's a control input that decides whether a comment reaches a real bug unreviewed — and the agent was being asked to self-report it without being told what it does, which invites grading on a curve.

3. Report each outcome to its team's Slack channel

Routed per component, because a channel belongs to the team that owns the component: notification_slack_emails maps "<Product> :: <Component>" to an address, with an optional default for components that have no channel of their own. A component matching neither sends nothing — posting one team's triage into another team's channel is worse than silence.

The component comes from Bugzilla rather than from the run: the agent doesn't report it, Bugzilla is authoritative (a bug moved mid-triage reaches whoever owns it now), and it keeps this self-contained — no new agent input, and it works for runs from the already-deployed agent image.

Delivered as email to a Slack channel address ("Email to channel"), which is why this is an email module rather than a Slack client: Slack renders an inbound message's subject as the title, so the one-line summary goes there and the links go in the body. Via SendGrid because Cloud Run has no local MTA and blocks outbound port 25 — and because the project already holds a sendgrid-api-key secret and a verified sender for the pulse listener, so this adds no new credentials.

It reports what actually happened rather than what was decided: deciding to apply and Bugzilla accepting it are different, so a rejected PUT reads as failed, not posted.

[frontend-triage] Bug 2037489 — Lists widget focus jumps outside the widget when an item is checked
Posted to Bugzilla.
Confidence: high
Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=2037489
Run: https://hackbot-ui-xxxx.run.app/runs/…

It sits at the end of on_run_completed rather than on its own run.completed subscription, because it has to know whether Bugzilla was written and a second consumer would race the applier. Run.notified_at makes it at-most-once, since Pub/Sub is at-least-once and an email — unlike an action row — is not idempotent. Both blocking calls (the Bugzilla read and the SendGrid send) go through asyncio.to_thread, as pubsub.py does.

Deploying

Safe to land before any config: with SENDGRID_API_KEY, NOTIFICATION_SENDER or NOTIFICATION_SLACK_EMAILS unset, the notification is a logged no-op. To turn it on:

• SENDGRID_API_KEY and NOTIFICATION_SENDER — mount the existing sendgrid-api-key secret and reuse the pulse listener's verified sender
• NOTIFICATION_SLACK_EMAILS — JSON, e.g. {"Firefox :: New Tab Page": "…@mozilla.org.slack.com"}
• alembic upgrade head for runs.notified_at

BUGZILLA_API_KEY is already set on the service and working ([email protected] has been applying comments), so nothing new is needed for the apply path.

Testing

124 passed in services/hackbot-api. The 8 errors in test_list_runs_api.py / test_create_run_api.py (fixture 'client' not found) are pre-existing on master — verified by running master's suite unmodified in a clean worktree (80 passed, 8 errors).

requests and sendgrid are declared explicitly rather than relied on transitively, since the workspace venv otherwise hides what uv sync --package hackbot-api actually installs in the image.

Follow-up

mozilla/bugbot will start triggering these runs automatically for newly filed Firefox :: New Tab Page bugs from staff — land this first, so a triggered run auto-applies and reports.