feat: structured logging, rate limiting, RPC resilience & mutation testing

[Lekan101]

Summary

Closes #914
Closes #912
Closes #913
Closes #910

This PR resolves four high-priority Stellar Wave issues covering backend infrastructure quality and observability.

---

Issue #910 — Structured logging with correlation IDs

Files changed: backend/services/shared/logging.ts, backend/services/logging.ts, ml-service/main.py

• backend/services/shared/logging.ts — full rewrite:
  • AsyncLocalStorage-based correlation ID propagation that survives async/await boundaries
  • Module-scoped child loggers via logger.child('payments')
  • Per-module log-level overrides via BACKEND_LOG_LEVELS=payments:debug,auth:warn env var
  • PII field redaction through piiClassifier
  • In-memory ring buffer with queryLogs() / clearLogBuffer() for tests and dashboard
  • Elasticsearch remote sink + optional HTTP endpoint (LOG_REMOTE_ENDPOINT)
  • runWithLogContext() helper for request-scoped correlation ID injection in middleware
• backend/services/logging.ts — simplified to re-export from shared logger (removes duplication)
• ml-service/main.py — fixed all missing imports (Field, Any, UserChurnData, RevenueObservation, registry, os), added StructuredLogger with JSON output, correlation ID middleware using ContextVar, typed all endpoints

---

Issue #912 — Timeout and circuit breaker for external blockchain RPC calls

Files changed: backend/services/shared/rpcTimeout.ts, backend/services/shared/rpcResilienceMiddleware.ts

• Updated issue references to #912 in both files
• rpcTimeout.ts: withRpcTimeout() (signal-aware factory) + wrapWithTimeout() (wraps existing promises), RpcCallTimeoutError / RpcCallCancelledError, defaultTimeoutForChain() per EVM chain, jitter support
• rpcResilienceMiddleware.ts: ResilientEthersProvider wiring together circuit breaker + per-call timeout + URL fallback; createResilientProvider(), getOrCreateResilientProvider(), clearProviderRegistry()

---

Issue #913 — Rate limiting per user and per API key

Files changed: backend/services/shared/rateLimitMiddleware.ts

• Updated issue reference to #913
• Added RATE_LIMIT_HEADERS named constants (no more magic strings)
• Added RateLimitRequest / RateLimitResponse type aliases for cleaner consumer imports
• Added createRateLimitStatusMiddleware() — attaches X-RateLimit-* headers read-only without enforcing limits (useful on public endpoints)
• Existing createRateLimitMiddleware() already enforces both per-key and per-user limits; createIpRateLimitMiddleware() covers unauthenticated paths

---

Issue #914 — Mutation testing with Stryker for frontend and backend

Files changed: stryker.conf.json, stryker.backend.conf.json, jest.config.js, jest.backend.config.js, package.json, reports/mutation/README.md

• stryker.conf.json: added json reporter, structured HTML output path (reports/mutation/frontend/), timeoutFactor: 2, maxTestRunnerReuse: 50, ignorePatterns, exclude index.ts / animations / i18n from mutation targets
• stryker.backend.conf.json: matching improvements, excludes migrations/ and server.ts from mutation
• jest.config.js: added coverageThreshold (60 % branches, 70 % lines/functions/statements), coverageReporters: ['text','lcov','json-summary'], refined collectCoverageFrom
• jest.backend.config.js: added collectCoverageFrom, coverageThreshold (50 % break, aligned with Stryker), coverageReporters
• package.json: added mutation:test:backend, mutation:test:ci, mutation:test:backend:ci scripts
• reports/mutation/README.md: usage docs, threshold table, output path reference

---

Testing

All changes are covered by existing test suites:

• backend/services/shared/__tests__/logging.test.ts — validates structured log entries, correlation ID propagation, sensitive field redaction, child logger filtering
• backend/services/shared/__tests__/rateLimiting.test.ts — validates per-key and per-user rate limiting, bypass, custom limits, analytics
• backend/services/shared/__tests__/rpcTimeout.test.ts — validates timeout, cancellation, jitter, error types
• backend/services/shared/__tests__/rpcResilienceMiddleware.test.ts — validates provider creation, send delegation, health snapshot
• ml-service/tests/test_structured_logging.py — validates JSON log format, correlation ID injection, middleware echo behaviour

[drips-wave]

@Lekan101 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits