Skip to content

Add Scheduled Background Job Runner - #155

Merged
Kaylahray merged 3 commits into
learnault:mainfrom
Ezeh20:background-job
Aug 30, 2026
Merged

Add Scheduled Background Job Runner#155
Kaylahray merged 3 commits into
learnault:mainfrom
Ezeh20:background-job

Conversation

@Ezeh20

@Ezeh20 Ezeh20 commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Closes #152

Summary

Queue drains were fired from the request that enqueued them, so retries only happened if unrelated traffic came in. This adds one scheduler process that owns every recurring drain on a timer.

  • src/workers/scheduler.worker.ts. Entrypoint, configurable interval
  • Registers the email, notification, webhook, stellar-funding, data-export and account-lifecycle queues
  • Each tick takes a lease via job-lease.service.ts, so replicas can't double-process
  • Removed the fire-and-forget processQueue() calls; enqueue is unchanged
  • Replaced the setInterval in server.ts; SCHEDULER_IN_PROCESS=true runs it in-process instead
  • Per-queue depth/due/lag/attempt/failure metrics, and leases are released on SIGTERM
  • Added the scheduler service to compose and the production image

Evidence

pnpm scheduler:verify

SCENARIO 1 — idle instance
API on :5000   : not running — no HTTP traffic is possible
before         : status=pending  attemptCount=1  error=previous attempt failed
after          : status=sent  attemptCount=2
after SIGTERM  : 6/6 leases released
RESULT         : PASS

SCENARIO 2 — two replicas, 40 due rows
attemptCounts  : attemptCount=1 -> 40 rows
lease races    : replica-A skipped 4, replica-B skipped 2
RESULT         : PASS — 40/40 processed exactly once, 0 duplicates
image

daxvinci
daxvinci previously approved these changes Aug 30, 2026
@Kaylahray
Kaylahray requested a review from 3m1n3nc3 August 30, 2026 09:50
@Kaylahray
Kaylahray merged commit 8dd4e81 into learnault:main Aug 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Add Scheduled Background Job Runner

4 participants