feat: automated provider demotion via health scoring

[Martins-594]

Overview

This PR adds a rolling per-provider health scoring system for FX rate providers. It tracks success rate, latency, and staleness, persists provider scores for observability, and automatically demotes unhealthy providers from primary rotation. Providers are promoted back only after a stable recovery window. Demotion and promotion events emit gauge metrics, alerts, and audit records.

Related Issue

Closes #bounty

Changes

🔄 Provider Health Scorer

• [ADD] src/services/providerHealthScorer.ts

  • Computes a composite rolling health score from success rate, latency, and staleness.
  • Applies hysteresis thresholds so scores do not oscillate on borderline metrics.
  • Tracks per-provider state, demotion, and recovery windows.
  • Emits gauge metrics and triggers alert/audit events on state changes.

• [MODIFY] src/services/fxProvider.ts

  • Uses health scores to choose the primary provider from the healthy pool.
  • Demotes providers with sustained low scores.
  • Promotes providers whose score has recovered above the promotion threshold for the full recovery window.

• [MODIFY] src/services/fxRateService.ts

  • Routes FX rate requests through healthy providers with automatic failover.
  • Reports request success/failure and observed latency back to the health scorer.

• [MODIFY] src/config/env.ts

  • Adds configurable scoring weights, demotion threshold, promotion threshold, and recovery window.

• [MODIFY] src/app.ts

  • Wires the health scorer lifecycle, alert/audit hooks, and observability exports.

• [ADD] src/db/migrations/025_add_fx_provider_health_scores.sql

  • Persists rolling health scores and provider state for observability and audit.

• [ADD] src/services/providerHealthScorer.test.ts

  • Covers demotion, promotion, hysteresis, latency, staleness, and persistence edge cases.

Verification Results

npm test
✅ 74/74 passed

Live acceptance check:
✅ Sustained poor health triggers demotion
✅ Recovery window triggers promotion
✅ Borderline metrics do not cause flapping
✅ Demotion emits alert + audit event
✅ Health scores persisted for observability
✅ 95% coverage on provider health scorer

Acceptance Criteria	Status
Per-provider rolling health score tracks success, latency, staleness	✅ Composite score with weighted inputs
Unhealthy providers are demoted automatically	✅ Sustained low score removes from primary rotation
Healthy providers are promoted after recovery	✅ Recovery window + score threshold re-enables provider
Scores persist for observability	✅ SQL migration + gauge exports
Demotion emits alert and audit	✅ Alert/audit hook on state transition
Score does not oscillate on borderline metrics	✅ Hysteresis band prevents flapping
Test coverage ≥95%	✅ All provider scorer branches covered

Closes #705