diff --git a/CLAUDE.md b/CLAUDE.md index f722057..34a060c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -203,7 +203,8 @@ GUI (`nuauto gui`; details: docs/GUI.md) `python -m` or file paths. COMPANY-SITE AGENT (`nuauto assist`; every rule: docs/ASSIST.md, read it before changing the agent) -- Only Needs Human rows stopped at an external application, Approved Other jobs rows, and (2026-10-16, my call) Needs +- Only Needs Human rows stopped at an external application, Approved Other jobs rows, Applied rows still owing their + company site (SITE_MARK, the posting's own link: assist.site_target; 2026-11-02), and (2026-10-16, my call) Needs Human rows stopped on NUworks itself (anything but an external application, e.g. a cover letter, transcript, an answer in my words; a NUworks job link, deadline not passed, no unresolved Submit click: assist.by_hand_target): on those it works NUworks itself (their own job page only: a prompt rule; code checks only the starting link), one at a time on NUworks (not beside an Apply run). diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 11a1421..294a743 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -191,7 +191,7 @@ SHEET (columns in sheet.HEADERS, made by setup_sheet.py) Needs Human sheet.set_status (apply.py stops, deadline passed); mark_submit_started (the row reads Needs Human between the Submit click and the result, so it can't be sent twice). set_status only changes rows still Approved. Notes starting sheet.SITE_MARK are written by apply.submit_flow and cleared - by sheet.mark_site_done. The weekly cap counts sent rows (sheet.SENT) dated in the current week (sheet.week_window); the + by sheet.mark_site_done (Mark the company site done, or a y after the assistant's session on that row). The weekly cap counts sent rows (sheet.SENT) dated in the current week (sheet.week_window); the total counts those NUworks has (sheet.total_count). The Other jobs tab has neither. REPO LAYOUT (what is not a module) diff --git a/docs/ASSIST.md b/docs/ASSIST.md index 9094ccd..afe42dc 100644 --- a/docs/ASSIST.md +++ b/docs/ASSIST.md @@ -12,6 +12,12 @@ WHICH ROWS (assist.py + prompts/ASSIST_PROMPT.md, `nuauto assist `; added 2 exact resume label (assist.hand_note; the rules' NUWORKS ROWS): it clicks the job's Apply, fills the popup, uploads the transcript or the cover letter, and its Submit asks me like any other. After y: Applied ("on NUworks"), no NUworks side after. The weekly cap is checked at its start and the Submit question says it counts. +- (2026-11-02, my request) Still owed rows: Applied NUworks-tab rows whose Notes start sheet.SITE_MARK (the posting + wanted the company site too; NUworks gave no external link): assist.site_target. The link is the posting's own + (jobs.site_link: the link inside the sentence that asks, else the posting's only link; several and none there: no + start, `nuauto assist --url `); never NUworks. No weekly / total check (it went out on NUworks already); + the session is told so (assist.site_note), and the Submit question says it counts toward no limit. After y: + sheet.mark_site_done (stays Applied, date kept), nothing sent on NUworks. The homelab's queue never starts it by itself. - Other jobs (added 2026-10-07): `nuauto assist other `: an Approved row of the Other jobs tab (https, never NUworks: assist.other_target), same agent, rules and hooks, with the layered answer bank (CLAUDE.md, ANSWER BANK). After y: Applied (dated today, no cap), nothing sent to NUworks; n = stays Approved. `nuauto assist other add diff --git a/docs/GUI.md b/docs/GUI.md index cff839b..c37d776 100644 --- a/docs/GUI.md +++ b/docs/GUI.md @@ -118,7 +118,8 @@ A question from a run (an unknown form field, an option to pick, "did it submit? needed (Needs Human rows stopped at an external application: the assistant), By hand on NUworks (Needs Human rows only you can finish there: a cover letter, a transcript; the assistant can work them on NUworks with you, or a chip says why not), Still owed (submitted on NUworks, the company site still wants you: - Mark the company site done; the NUworks side to send again: Submit on NUworks too), Applied ("not on + the assistant's Apply when the posting's link to that site is known (assist.site_target; else a "no + assistant" chip says why), and Mark the company site done; the NUworks side to send again: Submit on NUworks too), Applied ("not on NUworks" chip when NUworks has no application for it: it doesn't count toward the total; the job's own hiring stage at the last check (jobs.application_answer): "job interviewing", then amber "job stopped interviewing" or "job filled" (sheet.maybe_filled: it may be filled; withdrawing on NUworks frees its diff --git a/docs/PIPELINE.md b/docs/PIPELINE.md index 99608bf..165171e 100644 --- a/docs/PIPELINE.md +++ b/docs/PIPELINE.md @@ -15,7 +15,10 @@ Data comes from NUworks' own search/job data, not page text. search's postdate) and "featured". 2. Claude triage (Sonnet, prompts/TRIAGE_PROMPT.md, 250 jobs a batch): title, company, place and a snippet; drops only clearly unrelated roles. -3. details + hard rules (script; one NUworks job page each). +3. details + hard rules (script; one NUworks job page each). Each also keeps the posting's https links (jobs.page_links: + address, what it shows, the text before it; not NUworks / Northeastern ones), so a posting that asks for the company + site too has its link there (jobs.site_link, the assistant's Still owed rows). Details read before 2026-11-02 have + none: every update's check of the sent rows adds them (jobs.keep_links). 4. Claude match % vs your resume (Sonnet, prompts/SCORE_PROMPT.md, 20 jobs a batch). The formula is fixed in the prompt: 4-8 key requirements, each required or preferred, each met / partial / missing in the resume; match = 100 * (2 * required met + required partial + preferred met + 0.5 * preferred partial) / diff --git a/docs/REMOTE.md b/docs/REMOTE.md index 8cc78fa..202d6fb 100644 --- a/docs/REMOTE.md +++ b/docs/REMOTE.md @@ -94,6 +94,8 @@ THE QUEUE (remote.py; nuauto-assist.timer every 10 minutes runs `nuauto assist a next run. When it starts sessions, Discord says which. - By-hand NUworks rows (stopped on NUworks for a cover letter, a transcript...: docs/ASSIST.md) are listed too but never start by themselves (they need you there: NUworks' sign-in in its browser view, your letter): Start now. +- Still owed rows (Applied, the company site still wanted: docs/ASSIST.md, WHICH ROWS; added 2026-11-02) are listed too, + never start by themselves (remote.SITE_WHY): Start now. They count toward no limit (their NUworks side went out). - What it starts: Approved rows of the Other jobs tab and Needs Human rows stopped at a company site (not past their deadline), best first: closing within 7 days (jobs.URGENT_DAYS, soonest first), then match % (the "match NN%" in Notes), then NUworks rows before Other jobs, then sheet order. Into free slots (assist_slots, 1-3, the same setting diff --git a/docs/SAFETY.md b/docs/SAFETY.md index 4bb22d1..5b88d5a 100644 --- a/docs/SAFETY.md +++ b/docs/SAFETY.md @@ -158,7 +158,7 @@ COMPANY-SITE AGENT (assist.py, prompts/ASSIST_PROMPT.md) mail.pick, find_code, find_link, about_site, lookup, check_account; assist.email_cmd | test_mail - Guesses (2026-10-11): recorded with a reason, one line, an option when there are options; never over a saved answer, one given this run, or a leave-blank one; settled only by the user's answer at the end (save / once / confirm / blank / alias); an essay is confirmed, never saved | assist.record_guess, settle, guess_list | test_assist - Every Submit-type question (click, Enter, type submit) lists the open guesses, the answers the user changed, how many were confirmed, and values typed with no saved answer or guess behind them (the guard's own record of type / fill_form / select_option) | assist.submit_ask, review_lines, unexplained, record_typed | test_assist; that the agent records every guess is prompt-only: the typed-values list is the check -- Rows the agent may take: Needs Human + "External application" (never NUworks, even with --url: check_link), Approved Other jobs rows, and the by-hand rows above | assist.assist_target, other_target, by_hand_target, check_link | test_assist +- Rows the agent may take: Needs Human + "External application" (never NUworks, even with --url: check_link), Approved Other jobs rows, the by-hand rows above, and Applied rows still owing their company site (the posting's own link, never a guess; y only clears the mark, nothing sent on NUworks again; never started by the queue) | assist.assist_target, other_target, by_hand_target, site_target, check_link; jobs.site_link; remote.plan (SITE_WHY) | test_assist, test_jobs, test_remote - Sheet touched only by the launcher, after you answer y; agent has no sheet access | assist.run (mark_applied_by_hand), tools limited to the list above | test_assist (target rules only; the y/n flow is not tested) - NUworks side after a company-site submit reuses apply code, outcome appended to Notes | apply.submit_nuworks_side, sheet.append_note | no test (the NUworks browser busy with another run: "NOT submitted" in Notes, nothing opened, retry allowed: test_apply) diff --git a/prompts/ASSIST_PROMPT.md b/prompts/ASSIST_PROMPT.md index cd273da..b2a5d14 100644 --- a/prompts/ASSIST_PROMPT.md +++ b/prompts/ASSIST_PROMPT.md @@ -145,7 +145,8 @@ THE USER'S (ask right away: these can't wait for the end list) When the application is submitted (the user approved Submit and the site confirms), tell the user to type /exit; the terminal then asks whether it was submitted and updates the sheet (for a company-site job of the NUworks -tab it also submits the job on NUworks; an Other jobs row is not on NUworks; a NUworks row was NUworks itself). +tab it also submits the job on NUworks, unless THIS RUN says that went out already; an Other jobs row is not on +NUworks; a NUworks row was NUworks itself). Enforced by code (a blocked action comes back as an error; don't look for a way around it, tell the user): asking before Submit-type clicks and Enter (with every unanswered guess shown to the user); no touching captchas; password boxes take diff --git a/src/nuauto/assist.py b/src/nuauto/assist.py index 7d6acaa..4999e6c 100644 --- a/src/nuauto/assist.py +++ b/src/nuauto/assist.py @@ -181,6 +181,34 @@ def by_hand_target(row): return row.url, None +def site_target(row, url_override=None): + """(url, None) if this Applied NUworks-tab row still owes its company-site application (Notes start + sheet.SITE_MARK: the posting asked for both) and the posting's own link to that site is known (jobs.site_link; + --url: yours instead), else (None, reason). The NUworks side has gone out already: the agent works only that + site, and a y at the end clears the mark (sheet.mark_site_done); nothing is sent on NUworks again.""" + from nuauto import jobs + from nuauto import sheet + if row.status != "Applied" or not (row.notes or "").startswith(sheet.SITE_MARK): + return None, "not an Applied row whose company site is still owed" + if url_override: + return check_link(url_override) + try: + with open(jobs.path("details", f"{jobs.job_id(row.url)}.json")) as f: + d = json.load(f) + except (OSError, ValueError): + return None, "NUauto has no saved copy of this posting: open it on NUworks and follow its link" + url, why = jobs.site_link(d) + return check_link(url) if url else (None, why) + + +def site_note(row): + """THIS RUN's lines for a Still owed row (site_target): the NUworks side went out; what the posting said.""" + from nuauto import sheet + said = (row.notes or "")[len(sheet.SITE_MARK):].split("). ", 1)[0].strip(" (") + return ("This job was already submitted on NUworks. Its posting also asks for an application on the company's own " + f"site, which is this run: {said[:200]}\nApply there only; never open NUworks.\n") + + NUWORKS_BUSY = "Not now: an assistant is applying on NUworks (a by-hand row): one application at a time there. Finish it first." @@ -442,6 +470,8 @@ def review_lines(state): f"{cut(letter_['paragraphs'][0])!r}), uploaded as {os.path.basename(letter_['pdf'])}.") if state.get("hand"): lines.append("This is a NUworks application: it counts toward your weekly limit.") + if state.get("site"): + lines.append("This job is already Applied on NUworks: this is its company-site application (it counts toward no limit).") return lines @@ -1574,14 +1604,20 @@ def attention_main(kind): # ---------- launcher (nuauto assist) ---------- def find_row(number, url_override): - """The NUworks tab's row: a company-site application (assist_target), or one that stopped on NUworks for something - only you can give (by_hand_target: the agent works it on NUworks itself). (rows, row, url, by_hand).""" + """The NUworks tab's row: a company-site application (assist_target), one that stopped on NUworks for something + only you can give (by_hand_target: the agent works it on NUworks itself), or an Applied one whose company site is + still owed (site_target). (rows, row, url, kind): kind "site", "hand" or "" (a company-site application).""" from nuauto import sheet ws = sheet.open_worksheet() rows = sheet.read_rows(ws) row = next((r for r in rows if r.number == number), None) + if row is not None and row.status == "Applied" and (row.notes or "").startswith(sheet.SITE_MARK): + url, why = site_target(row, url_override) + if why: + sys.exit(f"Row {number} ({row.company}): {why}.") + return rows, row, url, "site" if row is None or row.status != "Needs Human": - sys.exit(f"Row {number} is not a Needs Human row.") + sys.exit(f"Row {number} is not a Needs Human row (or an Applied one whose company site is still owed).") if (row.notes or "").startswith("External application"): url, why = assist_target(row.notes, url_override) hand = False @@ -1592,7 +1628,7 @@ def find_row(number, url_override): hand = True if why: sys.exit(f"Row {number} ({row.company}): {why}.") - return rows, row, url, hand + return rows, row, url, "hand" if hand else "" def find_other_row(number, url_override): @@ -1773,12 +1809,13 @@ def rules_text(answer_cmd): def context_text(where, row, url, other, resume, read_roots, answer_cmd, writeups=(), view_url=None, transcript=None, - hand=False): + hand=False, site=False): email = accounts.account_email() own = accounts.own_login(url) # one of your own logins (imported from Bitwarden) for this site return (f"\n\nTHIS RUN\n{where.capitalize()}: {row.company} | {row.title}\nPosting: {url}\n" + (OTHER_NOTE if other else "") + (hand_note(row) if hand else "") + + (site_note(row) if site else "") + ("This browser runs on the user's homelab, not in front of them. They read your messages and answer on " "their phone (a review page), and they see and use this browser there too (for a captcha: say there is " "one and wait for them). A Submit-type action waits for their approval on that page: if it comes back " @@ -1853,12 +1890,15 @@ def run(number, url_override, other=False): if other: rows, row, url = find_other_row(number, url_override) where = f"{sheet.OTHER_TAB} row {number}" + kind = "" else: - rows, row, url, hand = find_row(number, url_override) - if not sheet.reapplying(row): # (one being re-applied is the same application: it adds nothing) + rows, row, url, kind = find_row(number, url_override) + # (one being re-applied is the same application: it adds nothing; a Still owed row's went out already) + if not sheet.reapplying(row) and kind != "site": sheet.check_limits(rows) where = f"row {number}" - hand = not other and hand + hand = not other and kind == "hand" + site = not other and kind == "site" if hand: # one application at a time on NUworks: not while an Apply run submits there from nuauto import browser holder = browser.profile_holder() or {} @@ -1902,7 +1942,7 @@ def run(number, url_override, other=False): print(f"(Note: {note}.)") try: if not hand: - return run_locked(number, other, row, url, where, tab, slot, slot_lock) + return run_locked(number, other, row, url, where, tab, slot, slot_lock, site=site) from nuauto import remote try: # one machine submits on NUworks at a time (with NUauto on the homelab, both can apply there) with remote.nuworks_turn(): @@ -1917,9 +1957,10 @@ def run(number, url_override, other=False): lock.close() -def run_locked(number, other, row, url, where, tab, slot, slot_lock, hand=False): +def run_locked(number, other, row, url, where, tab, slot, slot_lock, hand=False, site=False): """The session itself, once run() holds this row, a slot and the job site (and, from the laptop, the homelab's - claim): the agent in this terminal, then "Did you press Submit?". hand: a by-hand row, worked on NUworks itself.""" + claim): the agent in this terminal, then "Did you press Submit?". hand: a by-hand row, worked on NUworks itself; + site: an Applied row whose company site is still owed (site_target).""" from nuauto import browser from nuauto import sheet log_ = browser.RunLog(f"assist_{'other_' if other else ''}row{number}") @@ -1949,7 +1990,7 @@ def run_locked(number, other, row, url, where, tab, slot, slot_lock, hand=False) "transcript_src": os.path.realpath(transcript_src) if transcript else None, "transcript": transcript, "company": row.company, "job_title": row.title, "job_url": url, "row_url": row.url, "password_sites": [], "guesses": {}, "typed": {}, "account_email": accounts.account_email() or "", "where": where, - "slot": slot, "view_url": view.url if view else None, "tmux": tmux_session() if view else None, "hand": hand, + "slot": slot, "view_url": view.url if view else None, "tmux": tmux_session() if view else None, "hand": hand, "site": site, # the pane this runs in: the review page types only there (never another window of that tmux session) "tmux_pane": os.environ.get("TMUX_PANE") if view and re.fullmatch(r"%\d+", os.environ.get("TMUX_PANE", "")) else None} with open(os.path.join(d, "state.json"), "w") as f: @@ -1959,7 +2000,8 @@ def run_locked(number, other, row, url, where, tab, slot, slot_lock, hand=False) if hand and not config.DEMO: seed_nuworks(profile, d, log_) cmd = session(d, answer_cmd, read_roots, profile, rules_text(answer_cmd) + context_text( - where, row, url, other, resume, read_roots, answer_cmd, writeups, view.url if view else None, transcript, hand), where, + where, row, url, other, resume, read_roots, answer_cmd, writeups, view.url if view else None, transcript, hand, + site), where, view.display if view else None) log_.write(f"{where.capitalize()}: {row.company} | {row.title} -> {host(url)} (assistant slot {slot} of {slots()})") if view: # the password only here, on your screen: never in a log or a Discord message @@ -2007,6 +2049,10 @@ def run_locked(number, other, row, url, where, tab, slot, slot_lock, hand=False) log_.write(f"Not submitted; {where} stays {row.status}.") return ws = sheet.open_other() if other else sheet.open_worksheet() + if site: # the NUworks side went out before: the company site was the part still owed + sheet.mark_site_done(ws, row.number, row.url) + log_.write(f"{where.capitalize()}: company site marked done (it stays Applied; nothing sent on NUworks again).") + return sheet.mark_applied_by_hand(ws, row.number, row.url, how="on NUworks (nuauto assist; you pressed Submit)" if hand else "on the company site (nuauto assist; you pressed Submit)") log_.write(f"{where.capitalize()} marked Applied (dated today).") diff --git a/src/nuauto/demo.py b/src/nuauto/demo.py index 5d8bfe3..b7ad160 100644 --- a/src/nuauto/demo.py +++ b/src/nuauto/demo.py @@ -506,6 +506,12 @@ def setup(state_names=()): json.dump(details(j, today), f, indent=1) listed[j["id"]] = listed_entry(j, today) scores[j["id"]], cats[j["id"]] = score(j, today), j["category"] + # the Still owed row's posting (Willow Devices, row 900130): its link to the company site (jobs.site_link) + jobs.save("details/900130.json", {**details(JOBS[0], today), "id": "900130", "title": "Firmware Co-op", + "company": "Willow Devices", "applied": True, + "description": "Firmware work. You must also apply on our careers site HERE.", + "links": [{"url": "https://careers.willow.example/jobs/130", "text": "HERE", + "near": "You must also apply on our careers site HERE"}]}) jobs.save("list.json", listed) jobs.save("triage.json", {i: {"keep": True, "why": "demo"} for i in listed}) jobs.save("scores.json", scores) diff --git a/src/nuauto/gui.py b/src/nuauto/gui.py index f8313e2..be1c1f1 100644 --- a/src/nuauto/gui.py +++ b/src/nuauto/gui.py @@ -612,7 +612,8 @@ def view(r, **extra): hand, hand_why = assist.by_hand_target(r) # the assistant may work it on NUworks itself other.append(view(r, why=why, hand=bool(hand), hand_why=hand_why)) elif r.status == "Applied" and r.notes.startswith(sheet.SITE_MARK): - site.append(view(r)) + url, why = assist.site_target(r) # the posting's link to the company site: the assistant can apply there + site.append(view(r, target=url, host=urlparse(url).hostname if url else None, why=why)) elif r.status == "Applied" and any(m in r.notes for m in NUWORKS_SIDE_NOT_SENT) \ and assist.nuworks_side_blocked(r.notes) is None: retry.append(view(r)) diff --git a/src/nuauto/gui_static/app.js b/src/nuauto/gui_static/app.js index 8f64336..b5b0104 100644 --- a/src/nuauto/gui_static/app.js +++ b/src/nuauto/gui_static/app.js @@ -1348,10 +1348,13 @@ screens.apply = async (view) => { about = r.notes || r.why; next = r.hand ? assistantStart(d, r, "jobs", `assist-${r.row}`, { body: { row: r.row } }) : el("span", { class: "chip", testid: `assist-${r.row}-why`, text: `no assistant: ${r.hand_why}` }); - } else if (key === "owed" && r.owed === "site") { + } else if (key === "owed" && r.owed === "site") { // the assistant applies on the company site (its link from the posting) chips = [dueChip(r)]; about = (r.notes || "").slice(0, 160); - next = el("button", { class: "btn small primary", testid: `site-btn-${r.row}`, text: "Mark the company site done", onclick: () => markRow("site", r) }); + const done = el("button", { class: "btn small" + (r.target ? "" : " primary"), testid: `site-btn-${r.row}`, text: "Mark the company site done", onclick: () => markRow("site", r) }); + next = el("span", { class: "row" }, r.target ? assistantStart(d, r, "jobs", `assist-${r.row}`, { body: { row: r.row } }) : + el("span", { class: "chip", testid: `assist-${r.row}-why`, text: "no assistant", title: r.why || "" }), done); + if (r.target) open = linkOut(r.target, "Open"); } else if (key === "owed") { about = (r.notes || "").slice(-160); next = el("button", { class: "btn small primary", text: "Submit on NUworks too", testid: `retry-btn-${r.row}`, diff --git a/src/nuauto/jobs.py b/src/nuauto/jobs.py index ea403e8..5ffbdcc 100644 --- a/src/nuauto/jobs.py +++ b/src/nuauto/jobs.py @@ -320,6 +320,61 @@ def external_hint(d): return " ".join(text[(starts[-1] if starts else 0):end].split())[:220] +NUWORKS_LINK = re.compile(r"(^|\.)symplicity\.com$|(^|\.)northeastern\.edu$|(^|\.)neu\.edu$") +PLAIN_URL = re.compile(r"https://[^\s<>\"')\]]+") + + +def page_links(*htmls): + """The https links a posting's HTML (description, qualifications) holds, in order, once each: [{"url", "text", + "near"}]. text: what the link shows ("HERE"; a plain address in the text: itself); near: the posting's text up to + and including it (the last 120 characters, spaces squeezed): site_link reads which sentence it is in. Links to + NUworks and Northeastern are left out (Outlook safelinks are unwrapped first).""" + from nuauto import apply # unwrap + out, seen = [], set() + + def add(url, text, before): + url = apply.unwrap(html.unescape(url).strip().rstrip(".,;:")) + h = (urlparse(url).hostname or "").lower() + if urlparse(url).scheme != "https" or not h or NUWORKS_LINK.search(h) or url in seen: + return + seen.add(url) + near = " ".join((strip_html(before) + " " + text).split())[-120:] + out.append({"url": url, "text": " ".join(text.split())[:80], "near": near}) + + for h_ in htmls: + h_ = h_ or "" + for m in re.finditer(r"(?is)]*\bhref\s*=\s*[\"']([^\"']+)[\"'][^>]*>(.*?)", h_): + add(m.group(1), strip_html(m.group(2)), h_[:m.start()]) + bare = re.sub(r"(?is)", " ", h_) # addresses written out in the text (not inside a link) + for m in PLAIN_URL.finditer(html.unescape(re.sub(r"<[^>]+>", " ", bare))): + text = m.group(0).rstrip(".,;:") + add(text, text, html.unescape(re.sub(r"<[^>]+>", " ", bare))[:m.start()]) + return out + + +def site_link(d): + """(url, None): the company-site link of a posting that asks for an application there too (external_hint), or + (None, why). Only a link the posting itself gives, never a guess: the one inside that sentence ("... on the company + website HERE."), else the posting's only link; several and none in that sentence: None (pass --url).""" + if "links" not in (d or {}): + return None, "NUauto read this posting before it kept its links: they are read at the next update" + links = d["links"] + if not links: + return None, "the posting gives no link to the company site: open it on NUworks and follow what it says" + hint = " ".join((external_hint(d) or "").split()).lower() + inside = [] + for x in links: + tail = x["near"][-40:].rsplit(". ", 1)[-1] # the words right before it, in its own sentence + if hint and tail.lower() in hint: + inside.append(x["url"]) + if len(inside) == 1: + return inside[0], None + if len(links) == 1 and not inside: + return links[0]["url"], None + return None, (f"the posting has {len(links)} links and NUauto can't tell which is the application: " + "nuauto assist --url ") + + URGENT_DAYS = 7 # approve shows jobs closing within this many days first NEW_DAYS = 7 # a posting this new gets PREFS["new_bonus"] (the Apply rows' "posted" chip: green, then yellow to 14, grey to 21) @@ -601,6 +656,7 @@ def slim(raw): "applied": bool(raw.get("applied")), "description": strip_html(raw.get("job_desc")), "qualifications": strip_html(raw.get("qualifications")), + "links": page_links(raw.get("job_desc"), raw.get("qualifications")), # (site_link: the company site's) "fetched": date.today().isoformat(), } @@ -750,6 +806,22 @@ def application_answer(raw, prev, now): "filled_since": (prev.get("filled_since") or now) if hiring == "Filled" else None, "checked": now} +def keep_links(i, raw): + """A sent job's saved details get its posting's links (page_links), if they have none yet: details read before + links were kept (2026-11-02) get them from the next check, so a Still owed row's company site is known.""" + p = path("details", f"{i}.json") + try: + with open(p) as f: + d = json.load(f) + except (OSError, ValueError): + return + if "links" in d: + return + d["links"] = page_links(raw.get("job_desc"), raw.get("qualifications")) + with open(p, "w") as f: + json.dump(d, f, indent=1) + + def check_applications(rows): """Ask NUworks about every sent row of the NUworks tab (Applied, Interview, Accepted, Rejected; its job's own data, see application_answer) and keep its answers in data/applications.json, each with the row's @@ -778,6 +850,7 @@ def check_applications(rows): if raw and raw.get("job_id") == i: seen[i] = {**application_answer(raw, seen.get(i), now), "row": r.number, "fingerprint": sheet.fingerprint(r)} + keep_links(i, raw) browser.pause(page, 1.5, 3) log.write(f"asked NUworks about {len(sent)} sent rows: " f"{sum(1 for r in sent if (seen.get(job_id(r.url)) or {}).get('applied'))} it has") diff --git a/src/nuauto/remote.py b/src/nuauto/remote.py index 6042959..bcbbc0a 100644 --- a/src/nuauto/remote.py +++ b/src/nuauto/remote.py @@ -96,8 +96,9 @@ def match_of(notes): def candidates(main_rows, other_rows, closes=None, today=None, other_closes=None): """The rows the queue may start, best first: [{"tab", "row", "url", "row_url", "host", "company", "title", "closes", - "match", "laptop", "hand"}] (host: its job site, assist.site_key; row_url: the sheet row's own link; hand: a row - worked on NUworks itself (assist.by_hand_target), which only you start: it needs you there). + "match", "laptop", "hand", "site"}] (host: its job site, assist.site_key; row_url: the sheet row's own link; hand: a + row worked on NUworks itself (assist.by_hand_target), which only you start: it needs you there; site: an Applied + row whose company site is still owed (assist.site_target): only you start it, and it counts toward no limit). closes(row) -> its closing date or None (NUworks rows; apply.row_closes); other_closes: the same for the Other jobs tab's rows (intern.closes_func). A row past its deadline is never a candidate. Pure apart from those.""" from nuauto import assist @@ -105,20 +106,25 @@ def candidates(main_rows, other_rows, closes=None, today=None, other_closes=None today = today or date.today() out = [] for r in main_rows or []: - if r.status != "Needs Human": + site = False + if r.status == "Applied": # its company site still owed (assist.site_target): yours to start + url, _ = assist.site_target(r) + site, hand = bool(url), False + elif r.status != "Needs Human": continue - url, _ = assist.assist_target(r.notes) - hand = False - if not url: - url, _ = assist.by_hand_target(r) - hand = bool(url) + else: + url, _ = assist.assist_target(r.notes) + hand = False + if not url: + url, _ = assist.by_hand_target(r) + hand = bool(url) if url: day = closes(r) if closes else None if day and day < today: continue # past its deadline out.append({"tab": "jobs", "row": r.number, "url": url, "row_url": r.url, "host": assist.site_key(url), "company": r.company, "title": r.title, "closes": day, "match": match_of(r.notes), "laptop": assist.LAPTOP_MARK in r.notes, - "hand": hand}) + "hand": hand, "site": site}) for r in other_rows or []: url, _ = assist.other_target(r) if url: @@ -137,6 +143,7 @@ def candidates(main_rows, other_rows, closes=None, today=None, other_closes=None LAPTOP_WHY = "marked: do it on the laptop (the homelab's browser was refused there)" +SITE_WHY = "Applied on NUworks, its company site still owed: yours to start" HAND_WHY = "by hand on NUworks: yours to start (it needs you there: the NUworks login, your cover letter)" AUTO_OFF = {"jobs": "NUworks rows don't start by itself (off in Settings): yours to start", "other": "Other jobs rows don't start by itself (off in Settings): yours to start"} @@ -163,11 +170,13 @@ def plan(cands, running, claims, records, free, jobs_left=None, manual=False, ta why = OUTCOME_UNKNOWN elif c.get("hand") and not manual: # never by itself: it needs you there why = HAND_WHY + elif c.get("site") and not manual: # (Still owed) never by itself either: you start it + why = SITE_WHY elif same and not manual: # any earlier session (yours or the queue's): never again by itself why = "had a session already: yours to start now" elif not manual and tabs is not None and c["tab"] not in tabs: why = AUTO_OFF[c["tab"]] - elif c["tab"] == "jobs" and jobs_left is not None and jobs_left <= 0: + elif c["tab"] == "jobs" and not c.get("site") and jobs_left is not None and jobs_left <= 0: why = "this week's (or the total) NUworks applications are used up" elif c["host"] in busy_hosts: why = f"another session is on {c['host']}" @@ -176,7 +185,7 @@ def plan(cands, running, claims, records, free, jobs_left=None, manual=False, ta else: go.append((c, None)) busy_hosts.add(c["host"]) - if c["tab"] == "jobs" and jobs_left is not None: + if c["tab"] == "jobs" and not c.get("site") and jobs_left is not None: jobs_left -= 1 continue rest.append((c, why)) diff --git a/src/nuauto/review.py b/src/nuauto/review.py index 45d8360..68e6315 100644 --- a/src/nuauto/review.py +++ b/src/nuauto/review.py @@ -515,7 +515,8 @@ def queue_state(c, inputs, by_itself, here, tabs): if c["tab"] in tabs and first is None: return "next", "starts next by itself", True return "ready", ("had a session already: start it yourself" if first and first.startswith("had a session") - else "by hand on NUworks: start it yourself" if first == remote.HAND_WHY else "ready to start"), True + else "by hand on NUworks: start it yourself" if first == remote.HAND_WHY + else "company site still owed: start it yourself" if first == remote.SITE_WHY else "ready to start"), True return ("laptop" if why == remote.LAPTOP_WHY else "check" if why == remote.OUTCOME_UNKNOWN else "wait"), why, False diff --git a/tests/test_assist.py b/tests/test_assist.py index 2143b19..f940e42 100644 --- a/tests/test_assist.py +++ b/tests/test_assist.py @@ -338,6 +338,25 @@ def ok(state, tool, inp): assert "not a NUworks job page" in assist.by_hand_target(hand_row("Popup requires Cover Letter", url=f"https://{H}/jobs/detail/1"))[1] assert "not a NUworks job page" in assist.by_hand_target(hand_row("Popup requires Cover Letter", url="https://northeastern-csm.symplicity.com/students/app/home"))[1] assert "not Needs Human" in assist.by_hand_target(hand_row("x", status="Approved"))[1] +# an Applied row whose company site is still owed (sheet.SITE_MARK): the posting's own link (jobs.site_link), or --url; +# never NUworks, never a row without the mark +from nuauto import jobs as jobs_ # noqa: E402 +_data = config.DATA_DIR +config.DATA_DIR = os.path.join(tmp, "data") +os.makedirs(jobs_.path("details"), exist_ok=True) +with open(jobs_.path("details", "abc123.json"), "w") as f: + json.dump({"id": "abc123", "description": "Please apply on both NUworks and on the company website HERE.", + "qualifications": "", "links": [{"url": f"https://{H}/careers/job/7", "text": "HERE", + "near": "on both NUworks and on the company website HERE"}]}, f) +owed = sheet_.SITE_MARK + ' (posting: "Please apply on both NUworks and on the company website HERE."). Submitted via apply.py' +assert assist.site_target(hand_row(owed, status="Applied")) == (f"https://{H}/careers/job/7", None) +assert assist.site_target(hand_row(owed, status="Applied"), f"https://{H}/other") == (f"https://{H}/other", None) +assert "NUworks itself" in assist.site_target(hand_row(owed, status="Applied"), NUJOB)[1] +assert "still owed" in assist.site_target(hand_row("Submitted via apply.py", status="Applied"))[1] +assert "still owed" in assist.site_target(hand_row(owed, status="Needs Human"))[1] +assert "no saved copy" in assist.site_target(hand_row(owed, url=NUJOB + "x", status="Applied"))[1] +assert "company website HERE" in assist.site_note(hand_row(owed, status="Applied")) +config.DATA_DIR = _data # links in Notes assert assist.link_from_notes(f"External application: {H} -> https://{H}/careers/job/X_1") == f"https://{H}/careers/job/X_1" diff --git a/tests/test_jobs.py b/tests/test_jobs.py index f84b0c9..8d54198 100644 --- a/tests/test_jobs.py +++ b/tests/test_jobs.py @@ -189,6 +189,26 @@ def update(self, range_name, values, value_input_option=None): assert jobs.external_hint(d(description="Please apply on our careers page.")) assert jobs.external_hint(d(description="Apply machine learning to telematics.")) is None +# the posting's links (page_links) and which one is the company site's (site_link: the one in that sentence, else the +# only one; never a guess) +html_ = ('

To be considered for this position, please apply on both NUworks and on the company website ' + 'HERE. 

About Us. See ' + 'example.com, NUworks and https://example.org/privacy.

') +links = jobs.page_links(html_, None) +assert [x["url"] for x in links] == ["https://careers.example.com/job/123", "https://www.example.com", "https://example.org/privacy"], links +assert links[0]["text"] == "HERE" and links[0]["near"].endswith("on the company website HERE") +assert jobs.page_links(None, "") == [] +safe_ = "https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjobs.example.net%2F9&data=x" +assert [x["url"] for x in jobs.page_links(f'apply')] == ["https://jobs.example.net/9"] +desc = jobs.strip_html(html_) +assert jobs.site_link(d(description=desc, links=links)) == ("https://careers.example.com/job/123", None) +one = [{"url": "https://jobs.example.net/9", "text": "apply", "near": "Read our values apply"}] +assert jobs.site_link(d(description="You must also apply on our careers site.", links=one)) == ("https://jobs.example.net/9", None) +two = [{**one[0], "url": "https://a.example/1"}, {**one[0], "url": "https://b.example/2"}] +assert jobs.site_link(d(description="You must also apply on our careers site.", links=two))[0] is None +assert jobs.site_link(d(description="x", links=[]))[0] is None +assert "next update" in jobs.site_link(d(description="x"))[1] # read before links were kept + ws = FakeWS([HEADERS, ["https://a", "A", "T", "Applied", "", "2026-10-01"]]) n = sheet.add_proposed(ws, [{"url": "https://a", "company": "A", "title": "T", "notes": ""}, {"url": "https://b", "company": "B", "title": "U", "notes": "suggested"}]) diff --git a/tests/test_remote.py b/tests/test_remote.py index c609486..8c5dd44 100644 --- a/tests/test_remote.py +++ b/tests/test_remote.py @@ -99,7 +99,8 @@ def ext(host, path="/job/1"): c3 = dict(cands[0]) assert c3.pop("row_url") == next(r.url for r in main_rows if r.number == 3) # the sheet row's own link (Drop checks it) assert c3 == {"tab": "jobs", "row": 3, "url": "https://beta.icims.com/job/1", "host": "beta.icims.com", "company": "Co3", - "title": "Intern", "closes": TODAY + timedelta(days=2), "match": 50, "laptop": False, "hand": False} + "title": "Intern", "closes": TODAY + timedelta(days=2), "match": 50, "laptop": False, "hand": False, + "site": False} # the laptop mark in a row's Notes (assist.LAPTOP_MARK) shows as "laptop" on that candidate, on either tab laptop_notes = ext("acme.wd5.myworkdayjobs.com") + " | match 80% | " + assist.LAPTOP_MARK + " (why)" lc = remote.candidates([row(2, "Needs Human", notes=laptop_notes), row(3, "Needs Human", notes=ext("b.example") + " | match 70%")], @@ -123,6 +124,21 @@ def ext(host, path="/job/1"): assert go == [(hc[0], None)] and rest == [] go, rest = remote.plan(hc, [], [], {}, 3, jobs_left=0, manual=True) # it counts toward the limit assert go == [] and "used up" in rest[0][1] +# an Applied row whose company site is still owed (sheet.SITE_MARK, the posting's link known: assist.site_target): +# a candidate marked site, which never starts by itself, and counts toward no limit (its NUworks side went out already) +os.makedirs(jobs.path("details"), exist_ok=True) +with open(jobs.path("details", "owed1.json"), "w") as f: + json.dump({"id": "owed1", "description": "Please apply on both NUworks and on the company website HERE.", "qualifications": "", + "links": [{"url": "https://careers.example.com/j/1", "text": "HERE", + "near": "Please apply on both NUworks and on the company website HERE"}]}, f) +owed = sheet.SITE_MARK + ' (posting: "apply on the company website HERE."). Submitted via apply.py' +sc = remote.candidates([row(8, "Applied", nu[:-3] + "owed1", owed), row(9, "Applied", nu[:-3] + "nodetails", owed), + row(10, "Applied", nu[:-3] + "owed1", "Submitted via apply.py")], [], None, TODAY) +assert [(c["row"], c["site"], c["hand"], c["url"]) for c in sc] == [(8, True, False, "https://careers.example.com/j/1")], sc +go, rest = remote.plan(sc, [], [], {}, 3) +assert go == [] and rest == [(sc[0], remote.SITE_WHY)] +go, rest = remote.plan(sc, [], [], {}, 3, jobs_left=0, manual=True) +assert go == [(sc[0], None)] and rest == [] assert remote.tab_word("other") == "other" and remote.tab_word("jobs") == "nuworks" assert jobs.URGENT_DAYS == 7 # the order test above assumes it @@ -868,7 +884,7 @@ def isatty(self): remote.unclaim_on_homelab = lambda tab, rown: unclaims.append((tab, rown)) -def locked(*a): +def locked(*a, **k): ran.append(a) if len(ran) == 99: raise AssertionError @@ -880,7 +896,7 @@ def claim_fn(tab, rown, host): claims.append((tab, rown, host)) return claim remote.claim_on_homelab = claim_fn - assist.run_locked = (lambda *a: (_ for _ in ()).throw(RuntimeError("session crashed"))) if fail else locked + assist.run_locked = (lambda *a, **k: (_ for _ in ()).throw(RuntimeError("session crashed"))) if fail else locked config.IS_SERVER, config.REMOTE_ASSIST = server, remote_on real_stdin, sys.stdin = sys.stdin, TTY() out, sys.stdout = sys.stdout, io.StringIO() @@ -1272,6 +1288,34 @@ def drive(session_code, server=True, answer="n", approved=False): sheet.open_other, sheet.open_worksheet = patched["open_other"], patched["open_worksheet"] config.IS_SERVER = REAL_FLAGS[0] +# a Still owed row (site: its NUworks side went out already): y clears the mark (sheet.mark_site_done) and nothing more: +# not marked Applied again, no NUworks side; the session is told so (THIS RUN, assist.site_note) +calls = [] +real_site_done, real_by_hand, real_nuworks_side = sheet.mark_site_done, sheet.mark_applied_by_hand, assist.nuworks_side +try: + screen.Screen = FakeScreen + prompts = [] + assist.session = lambda d, a, r, p, prompt, *x, **k: prompts.append(prompt) or [sys.executable, "-c", "pass"] + assist.ask_line = lambda prompt, timeout=None: "y" + sheet.open_worksheet = lambda *a, **k: "WS" + sheet.mark_site_done = lambda ws, n, url: calls.append(("site_done", ws, n, url)) + sheet.mark_applied_by_hand = lambda *a, **k: calls.append(("applied",)) + assist.nuworks_side = lambda *a, **k: calls.append(("nuworks_side",)) + owed_row = sheet.Row(8, nu[:-3] + "owed1", "OwedCo", "Intern", "Applied", sheet.SITE_MARK + ' (posting: "apply HERE."). Submitted', "") + slot, lock = assist.take_slot({"row": 8, "tab": "jobs"}) + out, sys.stdout = sys.stdout, io.StringIO() + try: + assist.run_locked(8, False, owed_row, "https://careers.example.com/j/1", "row 8", "jobs", slot, lock, site=True) + finally: + sys.stdout = out + lock.close() + assert calls == [("site_done", "WS", 8, owed_row.url)], calls + assert "already submitted on NUworks" in prompts[0] and "apply HERE." in prompts[0], prompts[0][-600:] +finally: + sheet.mark_site_done, sheet.mark_applied_by_hand, assist.nuworks_side = real_site_done, real_by_hand, real_nuworks_side + screen.Screen, assist.session, assist.ask_line = patched["screen.Screen"], patched["session"], patched["ask_line"] + sheet.open_worksheet = patched["open_worksheet"] + # its hours are New York time whatever clock the process runs on (a shell over ssh is often UTC) from datetime import timezone # noqa: E402 from zoneinfo import ZoneInfo # noqa: E402