feat: dead letter queue for failed message processing (Closes #124)

[aaronmanuel309-bot]

Summary

Implements Issue #124 — Dead Letter Queue for Failed Message Processing for the webhook delivery service.

Until now, a webhook that exhausted its retry budget (or was rejected by the SSRF shield) was silently dropped. That made a downstream receiver outage or a mis-configured endpoint look like silent data loss. This change routes permanently failed deliveries into a bounded dead letter queue so every message is preserved for inspection and operator-driven redelivery.

Changes

New: webhook-delivery-service/src/deadLetterQueue.ts

A bounded, in-memory DLQ keeping the failed payload, target URL, signing material (secret/private key), attempt history, failure reason, and last error for each dead letter.

• reportDeadLetter(...): moves a permanently failed job into the DLQ.
• getDeadLetters() / getDeadLetter(id) / getDeadLetterCount(): inspection.
• popDeadLetter(id): atomic removal for redelivery.
• removeDeadLetter(id) / purgeDeadLetters(): discard operations.
• Bounded retention: defaults to 1,000 entries; when full, the oldest entry is evicted (FIFO) and tracked as discarded for alerting.

Delivery integration (delivery.ts)

• Max retries exhausted → dead-lettered with reason: MAX_ATTEMPTS_EXHAUSTED.
• SSRF-blocked → dead-lettered with reason: SSRF_BLOCKED (no HTTP attempt is made).
• requeueDeadLetter(id): reconstructs a fresh delivery job from the stored entry (fresh retry budget) and re-enqueues it through the full signing + security + retry pipeline.

HTTP API (index.ts)

• GET /deadletter — list all dead letters (newest first) with count.
• GET /deadletter/:id — fetch a single dead letter.
• POST /deadletter/:id/requeue — push a dead letter back onto the active queue.
• DELETE /deadletter/:id — remove a single dead letter.
• DELETE /deadletter?confirm=true — purge the entire queue (explicit confirmation required).
• DLQ size also surfaced via /health.

Metrics (metrics.ts)

• webhook_dead_letter_queue_size_current
• webhook_dead_letter_enqueued_total{reason}
• webhook_dead_letter_requeued_total
• webhook_dead_letter_discarded_total

Tests (deadLetterQueue.test.ts)

13 new tests covering reporting, FIFO eviction, popping/redelivery, removal/purge, and full delivery integration (permanent failure → DLQ, SSRF → DLQ, requeue → successful redelivery, plus HTTP endpoint coverage). The new module is at 100% statement/branch coverage.

Docs

Updated WEBHOOK_ARCHITECTURE.md (DLQ design section + new metrics) and WEBHOOK_RUNBOOK.md (DLQ inspection / redelivery / purge runbook).

Verification

• tsc --noEmit passes.
• Full Jest suite: 32 tests passing (19 pre-existing + 13 new). Pre-existing tests unchanged and green.

Closes #124