Skip to content

CHECKPOINT.md

Latest commit

 

History

History
261 lines (192 loc) · 14.5 KB

CHECKPOINT.md

File metadata and controls

261 lines (192 loc) · 14.5 KB
Raw

ProofScout — Harness Handoff Checkpoint

Created: 2026-09-11 Purpose: Resume this work in a different agent harness without losing context. Repo: C:/Users/oneda/Projects/Research - General/proofscout Git author: itw-code <[email protected]> (correct — NOT ihsanwanda-pro) Branch: master · Remote: none configured yet Target GitHub account when pushed: itw-code

1. Read these first (in order)

  1. CONTEXT.md — canonical domain glossary. Use these exact terms in specs/code/tests.
  2. docs/agents/domain.md — single-context project rules.
  3. docs/agents/issue-tracker.md — issues live as markdown in .scratch/.
  4. docs/agents/triage-labels.md — canonical labels.
  5. README.md — product overview + full CLI reference.
  6. .sureforge/task-ledger.md — SureForge gate record from the previous phase.
  7. AGENTS.md — agent instructions.

2. Why this repo exists (the actual point)

From PLAN.md:

Find the opportunities where a strong shipped project can beat a conventional credential, then help me prove I am the person who can ship it.

ProofScout is a personal opportunity radar + proof-of-work assistant. Not a job board, not an auto-apply bot.

It must surface opportunities where:

  • the work is genuinely remote / location-flexible,
  • compensation is high or verifiable,
  • the employer evaluates shipped work, take-home, paid trial, or challenge rather than credentials,
  • the opportunity is current, authentic, and has a reachable next step (application URL or sponsor contact),
  • it matches the owner's proof portfolio.

Owner's proof portfolio (from src/domain/profile.ts):

  • ColdStart — visual-first computer-use evaluation — proof: https://github.com/itw-code/solari-cookbook
  • Nakama — multi-tenant agent platform QA
  • Analytics engineering — data work
  • Automation work — AI automation, workflow design

THE CORE GATE = CRAWL BREADTH. The owner explicitly said: "add more boards, add more crawl machine, that's the main point of this repo first gate — if it's not much what's the point of creating this repo."

3. Current state — WHAT IS ALREADY WORKING

3.1 Committed

Commit Message
369dffa feat(sources): enable approved live boardUrls for real-time ATS job scanning
0f71677 feat(proofscout): initialize standalone repository with clean audit trail and resolved slices

3.2 Uncommitted work (this is the valuable part)

 M live-approval.json                          # expanded to 34 live boards
?? reports/generate-live-explorer.mjs          # live-scan HTML dashboard generator
?? reports/live-scan-34boards.json             # ACTUAL 34-board crawl output (6009 records)

3.3 Live crawl — VERIFIED WORKING (fresh run this session)

node dist/src/cli.js scan --approval live-approval.json --days 30 --format json

providers total: 34
available:       34
unavailable:      0
total records: 6009
shortlist:       10
authorization:   currently_authorized

Provider breakdown (34 boards, 3 ATS API families):

Greenhouse (20 boards): affirm(203), airtable(16), anthropic(597), asana(103), cloudflare(353), coinbase(221), datadog(451), dropbox(43), elastic(352), figma(158), gitlab(228), mongodb(404), reddit(145), robinhood(129), stripe(624), twilio(151), vercel(84), + 3 more

Ashby (13 boards): character(13), confluent(21), cursor(128), linear(30), midjourney(16), notion(127), openai(785), perplexity(114), plaid(102), quora(5), ramp(144), replit(78), runway(4), sentry(41), supabase(59), zapier(6)

Lever (1 board): spotify(74)

Top 10 shortlist from this crawl (real application URLs):

# Score Role Location Route
1 50 Manager, GTM Analytics (Zapier) NAMER jobs.ashbyhq.com/zapier/...
2 46 Workplace Technology Operation (Notion) Dublin jobs.ashbyhq.com/notion/...
3 45 Senior ML Engineer, Ads (Quora) Remote - Multiple jobs.ashbyhq.com/quora/...
4 45 Field Engineer, SLED (Cursor) Remote jobs.ashbyhq.com/cursor/...
5 45 Customer Success Manager (Linear) North America jobs.ashbyhq.com/linear/...
6 45 Enablement Specialist (Perplexity) San Francisco jobs.ashbyhq.com/perplexity/...
7 45 Director, India (Zapier) India jobs.ashbyhq.com/zapier/...
8 45 Staff Data Scientist (Quora) Remote - Multiple jobs.ashbyhq.com/quora/...
9 45 Senior SWE, Backend — Core AI Automation (Coinbase) Remote - USA coinbase.com/careers/...
10 45 Senior, Business Risk & AI Automation (Reddit) Remote - Ontario job-boards.greenhouse.io/reddit/...

3.4 Test suite baseline

npm test        → 34 files / 302 tests PASSED
npm run typecheck → exit 0
npm run build     → exit 0

Re-verify after any code change. Do not regress.

4. Architecture map (where things live)

src/
  cli.ts                    # CLI entry + all subcommands
  scan.ts                   # scan orchestration, approval gating, dedupe
  ranking.ts                # scoring: compensation, remote, projectFirst, portfolioFit, freshness, authenticity, risk
  evaluator.ts              # precision/recall on fixtures
  validation-gate.ts        # fixture gate (85% precision target)
  holdout-gate.ts           # real-source holdout gate
  review-report.ts          # review model (shortlist / needs_review / rejected)
  review-explorer.ts        # HTML renderer for review model
  decision-views.ts         # shortlist policy decisions
  route-evidence.ts         # application URL / sponsor contact extraction
  provider-approval.ts      # ProviderApprovalLifecycle: scoped, expiring, revocable
  hiring-region.ts          # region buckets: global/americas/emea/apac/multi/onsite
  domain/profile.ts         # TargetProfile + portfolio evidence cards
  sources/
    structured.ts           # ★ ATS adapters: greenhouse|ashby|lever|workable|smartrecruiters|careers
    rate-limiter.ts         # polite polling, backoff on 429/5xx
    reddit-pullpush.ts      # Reddit via PullPush mirror (social lane)
    x-ingest.ts             # X via operator last30days file (social lane)
    social-discovery.ts     # wires reddit-pullpush + x-ingest behind approval
    solari-visual.ts        # visual capture lane
    visual.ts               # screenshot -> OCR -> route
  social/index.ts           # social normalization, hiring-signal vs opportunity
  storage/sqlite-evidence-store.ts  # immutable evidence store (node:sqlite)

test/                       # 34 spec files, all offline, 302 tests
reports/                    # HTML dashboards + generators
.scratch/                   # issue tracker (all 41 issues resolved)
docs/adr/                   # 16 ADRs — read before changing behavior
.sureforge/                 # SureForge ledgers (gitignored)
live-approval.json          # ★ scoped provider approval for live scans

Key seam: src/sources/structured.ts

createApprovedStructuredAdapters(approval, options) reads:

  • approval.allowedProviderIds — which provider keys are authorized
  • approval.boardUrls — map of providerKey -> board URL

It supports two modes:

  1. Known provider names (greenhouse, ashby, lever, workable, smartrecruiters, careers) — uses boardUrls[name] or the default placeholder URL.
  2. Custom keys (e.g. openai, cursor, figma-gh) — infers the base ATS from the URL substring, then overrides the adapter id to the custom key so approval checks match.

live-approval.json uses mode 2 for every board (each company gets its own provider key).

5. How to run it

npm install
npm run build

# Live crawl (34 boards)
node dist/src/cli.js scan --approval live-approval.json --days 30 --format markdown

# Fixture quality gates (offline, deterministic)
node dist/src/cli.js validate      --fixture test/fixtures/validated-local-scan-v1.json
node dist/src/cli.js evaluate      --fixture test/fixtures/validated-local-scan-v1.json
node dist/src/cli.js review-report --fixture test/fixtures/validated-local-scan-v1.json --policy shortlist-ready
node dist/src/cli.js holdout-gate  --manifest test/fixtures/holdout/sanitized-holdout-v1.json \
                                   --policy   test/fixtures/holdout/holdout-policy-v1.json

# HTML dashboards
node reports/generate-records-explorer.mjs     # fixture-based review explorer
node reports/generate-live-explorer.mjs        # live crawl dashboard
start reports/live-scan.html
start reports/records-explorer.html

6. What is NOT done yet (the actual remaining work)

P0 — the core gate the owner asked for: MORE CRAWL BREADTH

Current: 3 ATS families (Greenhouse, Ashby, Lever) across 34 company boards.

Remaining work:

  1. Add the other 3 structured providers live. workable, smartrecruiters, and careers adapters exist and are unit-tested but have zero live boards wired. Every slug probed so far returned 404 or 0. Need to find real company slugs for these ATS platforms.

    • Workable: https://apply.workable.com/api/v1/widget/accounts/<slug>
    • SmartRecruiters: https://api.smartrecruiters.com/v1/companies/<slug>/postings
    • Careers: Schema.org JobPosting pages (per-company)

  2. Wire the social lane live. Code exists but is never activated:

    • reddit-pullpush — PullPush mirror. Note: returned HTTP 429 during probing; needs polite 20s+ interval and retry.
    • x-ingest — needs an operator-supplied /last30days raw markdown file.
    • These need "social" added to allowedLanes in live-approval.json, plus wiring createApprovedSocialDiscoveryProviders into main() in src/cli.ts (it is currently only used in tests).

  3. Wire the visual lane. solari-visual.ts + visual.ts exist; never activated live. → CLOSED BY DESIGN 2026-09-11 (see ledger F4): the lane is operator-driven by spec, not autonomously activatable. ADR 0003 reserves Solari/CUA for consented capture and explicitly rejects a Solari-first crawler; every capture fails closed unless it has (a) an operator-supplied ColdStart SolariDriver (external SDK, not in this repo), (b) a per-capture consent record with userAction + approvalId, and (c) an OCR/vision extraction client. Visual capture produces evidence merged into existing opportunities (mergeVisualEvidence), not discovered opportunities — so it has no autonomous scan-lane role. The seam already exists: executeCli accepts dependency adapters with lane: "visual" and the approval gate checks the visual lane. To use it: supply a driver + consent records via dependencies in a hosted/operator session.

  4. Expand company coverage. Many more Greenhouse/Ashby boards exist. Probe candidate slugs, add the ones that return data.

  5. Improve crawl throughput. Currently the scan fetches boards sequentially with concurrencyCap: 1. Consider raising concurrency per provider while respecting rate limits. → DONE 2026-09-11 (44987b9): order-preserving bounded concurrency (default 4) in scan.ts; per-provider RateLimiters and approval budget untouched; 89-provider crawl verified ~95s.

  6. Persist crawl results. The scan uses an in-memory SQLite store (new SqliteEvidenceStore(":memory:") in main()), so every run starts empty. Use a file-backed store so outcomes/watchlists/diffs/alerts survive between runs. → DONE 2026-09-11 (984b569): data/evidence.db (env PROOFSCOUT_STORE_PATH), mkdir-on-first-use, gitignored; watchlist persistence verified across processes.

  7. A real README tutorial — the README documents commands but there is no end-to-end "here is how you go from fresh clone to a shortlist" walkthrough. → DONE 2026-09-11 (956553f): six-step tutorial + offline gates, all commands verified against current behavior; stale social-lane claim corrected.

  8. Regenerate reports/live-scan.html from the saved scan output so the dashboard reflects the current crawl (the old HTML was from the 7-board run). → DONE 2026-09-11 (3cd5f91, f190490): regenerated from live-scan-90boards.json — 85 named boards, 10,516 records.

  9. Fix reports/generate-live-explorer.mjs — it re-ran the live scan inline; it should be able to consume the saved JSON instead. → DONE 2026-09-11 (3cd5f91): saved-JSON is the default; --fresh still available.

P2 — known non-blocking residuals (already disclosed)

  1. Fixture precision gate is 57.14% vs 85% target on the raw fixture — this is the documented baseline, reproduced exactly by tests. The production shortlist-ready policy does surface 100% precision (0 false positives on shortlist). See .scratch/precision-remediation/validation-report.md.
  2. 5 qualified fixture records lack route evidence by design (spec forbids synthesizing routes).
  3. Outer repo is still the home directory — ProofScout now has its own .git, and /Projects/Research - General/proofscout/ is in C:/Users/oneda/.gitignore.

7. Rules that must be honored

  • Approval-gated crawling (ADR 0006, 0013, 0014, 0015). No provider is fetched unless named in a scoped approval. Keep this invariant.
  • No credentials, no CAPTCHA bypass, no unattended credential automation. CREDENTIAL_PATTERNS in src/holdout-manifest.ts scans CLI args and evidence.
  • Never synthesize a route. Prose like "apply here" is not a route. See CONTEXT.md → Route evidence.
  • Immutable evidence. Raw payloads are never overwritten.
  • Human review gate. Nothing is auto-sent or auto-applied.
  • Use canonical glossary terms from CONTEXT.md in all new code/specs/tests.
  • Git identity must stay itw-code. Never commit as ihsanwanda-pro.
  • Scope git adds narrowly — the parent is the home directory.

8. Suggested first actions for the new harness

  1. Read the files in §1.
  2. Run the verification baseline in §3.4 and confirm 302/302.
  3. Run the live crawl in §5 and confirm 34 providers / ~6000 records.
  4. Commit the pending work (§3.2) with author itw-code.
  5. Pick up P0 items in §6 — starting with more live boards, since that is the owner's stated core gate.

9. File inventory for this checkpoint

Path Status Note
CHECKPOINT.md new this file
live-approval.json modified 34 live boards wired
reports/generate-live-explorer.mjs new dashboard generator
reports/live-scan-34boards.json new verified 34-board crawl output (6009 records)
reports/live-scan.html stale reflects old 7-board run; regenerate
reports/records-explorer.html ok fixture-based review explorer