Skip to content

scheduler.md

Latest commit

 

History

History
132 lines (98 loc) · 4.34 KB

scheduler.md

File metadata and controls

132 lines (98 loc) · 4.34 KB
Raw

ProofScout Local Scheduling and Automation

ProofScout operates strictly local-first with zero hosted cloud dependencies. Scheduled scans, change detection, watchlists, and alerts can run in-process or via native OS scheduling facilities.

1. Operating Modes

Social discovery rate policy (ticket 09)

When the scoped approval names the optional social-lane providers, scheduled runs must respect their polite rates:

  • reddit-pullpush: at most one mirror request per subreddit per 20 seconds with exponential backoff on 429/5xx (shared RateLimiter). Do not schedule social-lane scans more often than the subreddit count allows — e.g. 8 subreddits need at least ~3 minutes of fetch time per run. Prefer daily or twice-daily cadence.
  • x-ingest: file ingest only — no requests are issued, so no rate constraint. Drop a fresh last30days skill output file before the run and point the provider at it.

A. On-Demand / Ad-Hoc Execution

Execute a one-off scan with change detection and alert generation:

proofscout schedule-run --days 30 --approval path/to/approval.json --format markdown

B. In-Process Interval Execution

For local development or long-lived local worker processes:

import { Scheduler } from "./scheduler.js"

const scheduler = new Scheduler(dependencies, {
  days: 30,
  alertWebhookUrl: "http://localhost:8080/alerts",
  alertFilePath: "./alerts.md",
})

// Run every 60 minutes
const handle = scheduler.start(60 * 60 * 1000)

// Stop on process shutdown
process.on("SIGINT", () => {
  handle.stop()
  process.exit(0)
})

2. OS Cron Setup (Linux / macOS)

Using native crontab guarantees reliable execution without background Node.js process drift.

Crontab Configuration

Open crontab:

crontab -e

Run every morning at 8:00 AM:

0 8 * * * /usr/bin/node /path/to/proofscout/dist/cli.js schedule-run --days 30 --approval /path/to/approval.json --alert-file /path/to/alerts.md >> /var/log/proofscout.log 2>&1

Run every 4 hours during business days:

0 */4 * * 1-5 /usr/bin/node /path/to/proofscout/dist/cli.js schedule-run --days 14 --approval /path/to/approval.json >> /var/log/proofscout.log 2>&1

3. Systemd Timer Setup (Linux)

For production Linux environments, systemd timers provide robust logging and retry semantics.

/etc/systemd/system/proofscout.service

[Unit]
Description=ProofScout Market Intelligence Scan
After=network.target

[Service]
Type=oneshot
User=operator
WorkingDirectory=/home/operator/proofscout
ExecStart=/usr/bin/node dist/cli.js schedule-run --days 30 --approval /home/operator/proofscout/approval.json --alert-file /home/operator/proofscout/alerts.md
StandardOutput=journal
StandardError=journal

/etc/systemd/system/proofscout.timer

[Unit]
Description=Run ProofScout scan periodically

[Timer]
OnCalendar=*-*-* 08:00:00
Persistent=true

[Install]
WantedBy=timers.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now proofscout.timer

4. Windows Task Scheduler Setup

On Windows, use schtasks or PowerShell to configure periodic execution:

schtasks /create /tn "ProofScoutDailyScan" /tr "node C:\proofscout\dist\cli.js schedule-run --days 30 --approval C:\proofscout\approval.json" /sc daily /st 08:00

5. Change Detection and Alert Filtering

  1. Deterministic Windowing: Each run scans from now - days to now.
  2. Audit Diffs: Every change is stored in the SQLite opportunity_diffs table:
    • new-opportunity: newly discovered opportunity.
    • route-changed: canonical URL or source URL changed.
    • compensation-changed: salary/compensation representation updated.
    • deadline-passed: expiration date reached.
    • risk-flag-added: new contradiction or risk signals detected.
  3. Shortlist-Ready Alerts Only:
    • Alerts page only for new or materially changed shortlist-ready opportunities (status === "verified", actionable next step, no risk flags, confidence >= 0.7).
    • Unverified needs_review items and minor metadata noise do not trigger alerts.
  4. Watchlists:
    • Named companies, authors, and keywords can be registered via watchlist-add.
    • Watchlist matches without an actionable next step enter the review queue with evidence as hiring-signal, never as qualified opportunities.