Skip to content

feat(observability): emit a CloudWatch metric when webhook guardrail screening fails (fail-open silently degrades enrichment) #794

Description

@ClintEastman02

Non-blocking follow-up from the PR #788 review (silent-failure-hunter finding).

Problem

The three webhook screening paths — jira-webhook-processor.ts screenCommentsOrDrop, and linear-webhook-processor.ts screenProjectDocsOrDrop / screenCommentsOrDrop — fail open on a Bedrock Guardrail error: they logger.warn(...) and drop the third-party content so the task proceeds. That posture is correct (see the ADR proposed in #793), but the only signal is a log line. A sustained Bedrock Guardrail outage would silently degrade comment/doc enrichment fleet-wide, discoverable only via log search — there is no metric and nothing to alarm on.

Proposed fix

Emit a CloudWatch metric (e.g. WebhookScreeningFailed with a source dimension — jira-comments / linear-docs / linear-comments) on each fail-open path, so the degradation is graphable and alarmable. Wire it into the shared alarm plane rather than a bespoke alarm.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2lowest priorityinfra-cdkCDK stacks/constructs, bootstrap, deploy topology, tags, IAM wiring, teardownsecurityCedar/HITL, IAM least-privilege, secrets, PII/DLP, guardrails, supply-chain/CVE

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions