Skip to content

PLAN.md

Latest commit

 

History

History
247 lines (178 loc) · 15.1 KB

PLAN.md

File metadata and controls

247 lines (178 loc) · 15.1 KB
Raw

ProofScout - job-search intelligence plan

Status: planning
Date: 2026-09-06
Working title: ProofScout (a name for the product, not a commitment)

1. The outcome we are optimizing for

ProofScout should not be a generic job board or an auto-apply bot. Its job is to find a small number of unusually good opportunities where:

  • the work is remote or explicitly location-flexible;
  • compensation is unusually high or there is enough evidence to verify it;
  • the employer evaluates a project, trial, challenge, or shipped work rather than only a certificate;
  • the opportunity is current, authentic, and reachable through a real application or founder contact;
  • the opportunity matches the proof we can show from projects such as ColdStart, Nakama, analytics engineering, and automation work.

The first success metric is qualified opportunities with an evidence bundle, not the number of posts collected. A useful first release should produce 3-5 opportunities per week that a human would genuinely consider applying to, with the reason each one is worth attention visible in one screen.

2. Product shape

The user loop

  1. Define a target profile: remote preference, time-zone overlap, minimum compensation, contract/intern/full-time tolerance, skills, and willingness to build a take-home project.
  2. Run a scan on a schedule or on demand.
  3. Collect candidate postings and hiring signals from structured job boards, company career pages, X, LinkedIn, and the wider web.
  4. Extract a normalized opportunity record, including salary, remote scope, project-assessment language, credential requirements, deadline, and application route.
  5. Attach evidence: canonical URL, capture time, quoted text, screenshot/crop for visual posts, and confidence per field.
  6. Rank and deduplicate candidates, then show a review queue rather than applying automatically.
  7. For a selected opportunity, generate a tailored project pitch, a 24-48 hour build plan, a short outreach draft, and links to the strongest portfolio evidence.
  8. Record the result: ignored, researching, contacted, applied, interview, rejected, or hired. This feedback improves ranking.

The product promise

“Find the opportunities where a strong shipped project can beat a conventional credential, then help me prove I am the person who can ship it.”

3. What to search, and how

Lane Sources Why it matters Initial policy
Structured jobs Greenhouse, Ashby, Lever, Workable, SmartRecruiters, direct careers pages Highest precision, stable fields, easy dedupe Use public GET/posting feeds and first-party pages first
Social hiring signals X posts, founder/operator posts, public LinkedIn posts, Reddit, HN, Bluesky Finds “we are hiring”, project challenges, and roles never syndicated to boards Use official/authorized APIs where possible; browser access is opt-in and rate-limited
Discovery web Search provider results, company sites, incubator/accelerator pages, challenge pages Finds new companies and canonical links Treat snippets as leads, never as proof of salary or legitimacy
Visual posts LinkedIn/X images, collages, screenshots, flyers Captures posts whose important text is not in the HTML body Screenshot -> OCR -> VLM extraction -> human confirmation
Company signals Careers pages, hiring clusters, founder announcements, funding context Helps detect a real hiring push and strategic fit Keep as supporting evidence; do not infer a job from hiring signals alone

The existing /last30days implementation is a valuable starting adapter rather than something to rewrite immediately. It already has jobs retrieval with ATS detection and tiers, a LinkedIn text adapter, normalization, dedupe, and hiring-signal analysis. Its LinkedIn lane is text-oriented; image/collage extraction is a deliberate ProofScout extension.

4. Architecture

source adapters
  -> raw evidence store (immutable payload + provenance)
  -> extraction (rules, OCR, VLM)
  -> entity resolution + dedupe
  -> opportunity scoring
  -> review queue / alerts
  -> project brief + outreach draft
  -> outcome feedback

Core records

Every surfaced claim should be traceable to evidence. The minimum schema is:

Opportunity
  id, canonical_url, source, captured_at, published_at
  company, author, role_title, opportunity_type
  compensation: min, max, currency, period, confidence
  remote_scope, timezone_overlap, location_restrictions
  project_assessment, credential_stance, required_skills
  application_url, deadline, authenticity_signals, risk_flags
  fit_score, evidence_score, overall_score

Evidence
  opportunity_id, source_url, source_kind, quoted_text
  screenshot_path, crop_path, ocr_text, visual_regions
  extraction_method, captured_at, field_confidence

Keep raw source payloads immutable and store derived fields separately. That makes corrections auditable and lets us re-run extraction when the parser or model improves.

Suggested implementation stack

  • TypeScript/Node for the product and orchestration, matching the Solari/ColdStart codebase.
  • SQLite for the first local release; DuckDB or Postgres only when alerting/history volume justifies it.
  • JSONL or compressed raw artifacts for source evidence; never store credentials in the repository.
  • A small adapter interface: discover(), fetch(), normalize(), provenance().
  • A model interface with separate cheap extraction and expensive reasoning/VLM passes.
  • Markdown and JSON reports first; a web UI can follow once the ranking is trustworthy.

5. Visual-first lane

This is where Solari can make the project distinctive, but it should be invoked selectively.

Two-pass extraction

  1. Candidate pass: capture the visible post/card, run inexpensive OCR and simple hiring-language detection (hiring, intern, project, currency, remote, DM, deadline).
  2. Evidence pass: only for likely candidates, send the screenshot and cropped regions to a vision model with a strict JSON schema. Ask it to return field values, bounding regions, uncertainty, and whether the text is an offer, a repost, or an advertisement.
  3. Human confirmation: require a click-through and confirmation for ambiguous salary, identity, deadline, or application instructions.

Store a screenshot hash and crop coordinates so a reviewer can see exactly what the extractor saw. OCR is a helper, not the authority; the image and canonical URL remain the evidence.

Solari's role

Reuse the proven ColdStart primitives:

  • screenshot -> bounded mouse/keyboard action -> screenshot loop;
  • no DOM/accessibility text exposed to the model;
  • harness-side grounding only for click precision;
  • action traces, screenshots, and fail-closed verification.

Use Solari for:

  • a logged-in, user-consented LinkedIn/X visual capture session;
  • verifying that a post really opens, contains the claimed visual text, and links to the expected company/application;
  • replayable extraction tests against synthetic layout variations;
  • optional isolated browsing when a source needs a disposable browser.

Do not use Solari as the default high-volume crawler, as a way around CAPTCHAs, or as an unattended application submitter. Its value here is trustworthy visual evidence and a measurable browser task, not raw page throughput.

6. Ranking model

Start with transparent scoring so every recommendation can be explained:

overall =
  0.25 compensation quality
  + 0.20 remote/time-zone fit
  + 0.20 project-assessment signal
  + 0.15 skill/portfolio fit
  + 0.10 freshness/deadline
  + 0.10 source authenticity
  - risk penalties

Hard penalties: pay-to-apply, unverifiable recruiter identity, missing canonical link, contradictory compensation, obvious repost spam, or a request for sensitive credentials. Missing salary should reduce confidence, not automatically discard an otherwise strong founder post.

The result must explain itself in plain language: “high because it is remote, names a paid project trial, and matches ColdStart; medium confidence because compensation appears only in an image.”

7. Source and compliance decisions

  1. Prefer public, first-party, and officially documented interfaces.
  2. Do not depend on LinkedIn's undocumented endpoints. LinkedIn's official Job Posting API is restricted to approved developers and currently says it is not accepting new partnerships, so it is not a safe foundation for a personal crawler.
  3. Treat third-party LinkedIn/X search providers as replaceable adapters. Keep their raw output, terms, rate limits, and attribution visible.
  4. Browser sessions require explicit user consent, a visible profile, low request rates, and no credential persistence in project artifacts.
  5. Never auto-send a DM, submit an application, or publish a project without a final human approval step.
  6. Add a source-level kill switch and deletion command before adding more providers.

8. Phased build plan

Phase 0 - evidence and thesis (half day)

  • Define the target profile and minimum acceptable compensation.
  • Assemble a manually verified seed set of 30-50 opportunities/posts across ATS, X, LinkedIn, and the web.
  • Label each item: remote, compensation, project-assessable, certificate requirement, authenticity, and “would I apply?”.
  • Decide whether the $300K example is a real annualized figure, total contract value, stipend, or a viral misreading. Never optimize around the number until the unit and currency are verified.

Gate: at least 20 labeled items and a clear definition of “qualified”.

Phase 1 - structured radar (2-3 days)

  • Build ATS/careers adapters and reuse /last30days jobs normalization where practical.
  • Add X/text and web discovery as optional adapters.
  • Implement the schema, provenance, dedupe, transparent score, and Markdown/JSON report.
  • Add tests using recorded fixtures, including duplicate cross-posts and missing salary.

Gate: >=85% precision on “is this actually a job/opportunity?” and every field in the report has evidence or an explicit “unknown”.

Phase 2 - visual capture (2-3 days)

  • Add a browser capture adapter with a user-controlled session.
  • Implement screenshot hashing, OCR, structured VLM extraction, and crop/evidence storage.
  • Add Solari replay fixtures and fail-closed checks for role, compensation, remote scope, and application URL.

Gate: >=90% accuracy for explicit salary/remote fields on a 30-item visual test set; no silent extraction when confidence is low.

Phase 3 - personalized proof-of-work kit (2-4 days)

  • Create a portfolio evidence map from existing projects.
  • Match job requirements to evidence, identifying gaps rather than inventing experience.
  • Generate a project proposal, acceptance criteria, build sequence, demo script, and concise outreach draft.
  • Keep output review-only; no automatic send/apply.

Gate: each generated brief contains source citations, a realistic scope, and a falsifiable demo outcome.

Phase 4 - operations (later)

  • Scheduled scans, change detection, and alerts.
  • Founder/company watchlists and hiring-signal history.
  • Feedback learning from ignored/applied/interviewed outcomes.
  • Multi-user or hosted deployment only after local precision and compliance are proven.

9. First experiment to run

Build a local CLI with one command:

proofscout scan --profile remote-project-first --days 30

It should return:

  • the top 10 ranked opportunities;
  • one evidence sentence per score component;
  • canonical links and capture dates;
  • a “why this is worth your time” paragraph;
  • an explicit uncertainty/risk line;
  • a suggested proof project using an existing portfolio asset.

The first scan should deliberately mix:

  • direct ATS boards from AI/automation/data companies;
  • founder/operator “we are hiring” posts on X;
  • public LinkedIn text results;
  • a small, consented visual sample rather than pretending image extraction is solved at scale.

Confirmed MVP contract

  • Audience: personal use first, focused on remote AI/automation/data work and adjacent founder challenges.
  • Window/output: a 30-day scan producing a top-10 shortlist in JSON and Markdown.
  • Source order: ATS/careers pages, then X/web text, then public LinkedIn text, then visual browser capture for shortlisted candidates.
  • Compensation: no hard retrieval floor; rank explicit high compensation highest and retain unknowns with field-level uncertainty.
  • Compensation bands: Top $200K+, Strong $100K-$199K, Good $50K-$100K, Fine $30K-$50K, plus Unknown/other when normalization is not justified.
  • Matching: begin with manually curated portfolio evidence cards for ColdStart, Nakama, analytics engineering, and automation projects.
  • Side effects: recommendation, dossier, and draft only; the review gate remains mandatory.

10. Decisions we should make next

  1. Target profile: default to remote AI/automation/data roles with project-based evaluation, while allowing any title if compensation and fit are strong.
  2. Compensation normalization: represent annual, monthly, hourly, stipend, equity, and one-off project payments separately. Do not compare them as one number until normalized.
  3. Authentication boundary: start with public ATS + web + X text. Add LinkedIn visual capture only after the review/consent flow exists.
  4. Product boundary: keep ProofScout as one product with an optional solari-vision package. Split repositories only if visual capture becomes independently useful.
  5. Application boundary: recommendation and evidence-pack generation first; submission automation is out of scope for the MVP.

11. Current evidence and references

  • Existing repo anchor for the original hiring signal: Harry Chow's X post, recorded in SWE-Intern-Pinetree-Proposal.md:277 and duplicated in coldstart/solari-cookbook/docs/proposals/v1-witness.md:277. The local notes interpret it as the "$300K annualized, no resume, no cover letter, no grades" challenge to fork Solari and build a real use case. The post itself should still be re-opened before treating compensation or wording as independently verified.
  • Existing ColdStart action loop and hybrid grounding: coldstart/solari-cookbook/src/agent/action.ts, src/agent/loop.ts.
  • Existing /last30days jobs source: C:\Users\oneda\.agents\skills\last30days\scripts\lib\jobs.py.
  • Existing /last30days LinkedIn text source: C:\Users\oneda\.agents\skills\last30days\scripts\lib\linkedin.py.
  • LinkedIn Job Posting API overview - approved-partner restriction and current partnership notice.
  • Greenhouse Job Board API - public GET job-board data without authentication.
  • Ashby Job Postings API - published postings API with optional compensation data.
  • OpenAI computer use documentation - screenshot-driven decisions and structured mouse/keyboard actions.

Claims about a specific $300K offer, a particular person, or current platform pricing must be verified from the original post before being used as product requirements.