Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# AGENTS.md

Instructions for AI agents working on the arJev codebase. If you are here to
**install arJev for a person**, stop and follow
[docs/agent-setup.md](docs/agent-setup.md) instead.

arJev is a Python 3.11+ CLI: a daily arXiv digest ranked against an Obsidian
vault, with an optional Slack surface. Runtime dependencies are PyYAML, httpx,
and pypdf; nothing else.

## Commands

```bash
uv sync --extra dev # set up the environment
uv run --extra dev pytest # the full suite (~10 s, no network)
uv run --extra dev pytest tests/test_slack.py -k who_line # one area
uv run --extra dev ruff check . # lint (CI runs exactly this)
uv run arjev --help # the CLI from the working tree
```

CI (`.github/workflows/ci.yml`) runs `ruff check .` and `pytest` on Python 3.11.
Both must pass before a merge.

## Layout

| Path | What lives there |
|---|---|
| `src/arjev/cli.py` | every verb: argparse tree + dispatch; nothing else |
| `src/arjev/config.py` · `init.py` | the TOML config and the commented starter config `arjev init` writes |
| `src/arjev/feed.py` | arXiv RSS fetch and parse (author names are split here) |
| `src/arjev/fold.py` · `profile.py` · `directives.py` | the vault scan, the taste profile, and taste directives |
| `src/arjev/score.py` · `rank.py` · `rerank.py` · `jev.py` | lexical scoring, ranking, the Jev layer, and the digest journal |
| `src/arjev/digest.py` · `render.py` | one digest run and its sinks (stdout, vault note, Slack) |
| `src/arjev/affil.py` | the who-line: affiliations parsed from a paper's own PDF |
| `src/arjev/slack.py` · `ingest.py` | the Slack client, pick rendering, reaction harvest; seeding from BibTeX/PDFs/Slack |
| `src/arjev/keep.py` · `labels.py` · `ledger.py` · `state.py` | keeps, the label ledger, posted ids |
| `src/arjev/paper_content.py` | PDF text extraction and the library |
| `src/arjev/calibrate.py` · `review.py` · `rate.py` | calibration, the week-in-review, rating proposals |
| `src/arjev/daily.py` | `arjev daily`: the scheduled run and its retry/weekend rules |
| `src/arjev/inspect.py` · `sanitize.py` · `latex.py` | config verification, fixture sanitizing, LaTeX → Unicode |
| `tests/` · `tests/fixtures/` | the suite; recorded, sanitized fixtures (never live data) |
| `contrib/` | systemd and launchd units that run `arjev daily` |
| `scripts/` | `brand.py` regenerates `docs/brand/`; `capture_fixtures.py` records fixtures (networked, maintainer-only) |
| `docs/` | user docs, [ADRs](docs/adr/), [history](docs/history/) |

## Invariants — do not break these

1. **The tool never writes prose.** Note bodies and why-lines come from an
agent or a person; arJev renders deterministically and Jev only makes typed
decisions ([ADR 0001](docs/adr/0001-vault-generation-is-the-agents-contract.md)).
2. **Never edit an existing note.** arJev writes new files only. The single
exception is `arjev rate accept`, which a human invokes.
3. **Never default a category.** Empty `feeds` blocks the digest with an error
naming the slot.
4. **Attribution is printed or curated, never inferred.** Affiliations come
from the paper's own PDF and are never bound to an author; groups come only
from the roster ([ADR 0002](docs/adr/0002-at-digest-time-affiliations-ride-from-the-papers-own-pdf.md)).
5. **External seams fail open.** A Jev, PDF, or enrichment failure degrades
that feature and the run continues; the `mode:` line says what ran.
6. **State lives under XDG state** (`~/.local/state/arjev/`), never in the
vault or a repo. Tests isolate it automatically (`tests/conftest.py`).
7. **Secrets never touch config or the repo.** Keys and tokens come from env
vars or mode-600 files (`ARJEV_JEV_KEY_FILE`, `ARJEV_SLACK_TOKEN_FILE`).
8. **Persisted schemas are frozen.** Journal rows, the label ledger, and posted
state keep bare arXiv ids; extend with new fields, never change existing ones.
9. **The Slack content boundary.** Posts carry feed-derived text, printed
affiliations, curated roster labels, and matched terms only — never vault
paths, note filenames, or a note's `why` text.

## How to change things

- **Test first.** Write the failing test, then the code. Tests exercise public
behavior; network calls go through injectable seams (`SlackClient(transport=…)`,
`http_transport=`, `pdf_fetcher=`, `JevClient` stubs), never real services.
- **The docs lint is real.** `tests/test_docs_commands.py` checks that every
arjev command shown in `README.md`, `CONTEXT.md`, `AGENTS.md`, and `docs/`
exists in the CLI, verb and flags. Add the flag before you document it, and
never put a placeholder verb inside a code span.
- **Docs have one job each.** `README.md` is for a human deciding whether to use
arJev; setup steps go in `docs/setup.md` (human) and `docs/agent-setup.md`
(agent); how it works goes in `docs/architecture.md`. New domain terms go in
`CONTEXT.md`; hard-to-reverse decisions get an ADR in `docs/adr/`.
- **Record user-visible changes** in `CHANGELOG.md` with the release that ships them.

## Releasing

1. Bump `version` in `pyproject.toml` — the only place it lives (`arjev.__version__`
reads the installed metadata; `tests/test_packaging.py` enforces this).
2. Add the `CHANGELOG.md` entry.
3. Open the PR, wait for CI to pass, merge.
4. Tag the **merge commit on `main`** and push the tag:
`git tag vX.Y.Z -m vX.Y.Z && git push origin vX.Y.Z`. Never tag before the
merge lands — a tag on an unmerged branch ships an install without the fix.

## External behavior worth knowing

Lessons from live runs that the tests now pin:

- arXiv RSS puts **all authors in one comma-joined field**; `feed.py` splits it.
- arXiv publishes **no new listings on Saturday or Sunday**; empty weekend
feeds are expected (`daily.py`).
- arXiv RSS is **occasionally empty mid-day on weekdays**; retry, don't alarm.
- Slack's read methods (`reactions.get`, `conversations.replies`) **ignore JSON
bodies**; every call is form-encoded (`slack.py`).
- Slack needs a channel **ID**, not a name, for history and ingest.
- No external index (arXiv API, Semantic Scholar, INSPIRE, OpenAlex) carries
affiliations on publication day; the paper's PDF is the only source.
101 changes: 101 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Changelog

All notable changes, newest first. Dates are release tags (UTC). Install a
specific release with `uv tool install git+https://github.com/harmoniqs/[email protected]`.

## v0.6.0 — 2026-10-26

**One command for the whole day, and a setup kit in the repo.**

- **`arjev daily`** runs the full day for a timer: the digest (retried with
backoff when arXiv's RSS is flaky), Slack ingest and reaction sync, and on
Sundays the calibration report and week-in-review. On Saturdays and Sundays
arXiv publishes no new listings, so an empty feed is logged as a quiet day
instead of an outage. One failing step never stops the others.
- **`ARJEV_SLACK_TOKEN_FILE`** reads the Slack token from a mode-600 file, the
same way `ARJEV_JEV_KEY_FILE` already worked for the Jev key.
- **`contrib/`** ships a systemd service + timer and a macOS launchd plist that
call `arjev daily`.
- The version is single-sourced from `pyproject.toml`.
- Digest headers say "ranked against your vault" instead of "the lab corpus",
since arJev serves individual researchers as well as labs.
- Docs reorganized: a human-first README, a step-by-step `docs/setup.md`,
`AGENTS.md` for agents working on the repo, and this changelog in place of
per-release notes files.

## v0.5.6 — 2026-10-26

- `arjev digest --post slack,vault` ranks once and writes to every listed sink:
the dated vault note first, then Slack. If one sink fails the others still
run, the failed sink is named on stderr, and the exit code is 1. Config
errors are caught before ranking, so a misconfigured run spends no Jev calls.

## v0.5.5 — 2026-10-25

- **Fixed:** Slack Web API calls are form-encoded. `reactions.get` and
`conversations.replies` ignore JSON bodies, so `arjev slack sync` never
worked against real Slack before this release.
- The Slack app manifest adds `channels:join`, so the bot can join its channel.

## v0.5.4 — 2026-10-25

- `arjev review --post slack` posts a Slack-native week-in-review: bold section
titles, titles linked to arXiv, readable dates, plain labels. The full report
(stdout, `review-YYYYMMDD.md`) is unchanged.
- `\( … \)` inline math in titles is cleaned like `$ … $`.

## v0.5.3 — 2026-10-25

- **Fixed:** the week-in-review ranked by Jev's confidence in *any* label, so
papers Jev was surest were *irrelevant* led "The Buried". It now orders by
label, then confidence, and only relevant papers are listed. The digest was
never affected.

## v0.5.2 — 2026-10-21

- Affiliation parsing strips email parentheticals and handles an abstract
heading run into its first word (`ABSTRACTCloud…`).

## v0.5.1 — 2026-10-21

- **Fixed:** arXiv RSS puts every author in one comma-joined field, and names
were never split. Splitting them fixes the who-line's author list and lets
the tracked-author lane match anyone on the paper, not just the last name.

## v0.5.0 — 2026-10-21

- **The who-line:** every Slack pick shows its authors (first three + `+N`),
tracked people as `⭐ Name (Group)`, and the institutions printed on the
paper's own PDF. Affiliations are never inferred or bound to individual
authors; an ambiguous parse shows authors only (ADR 0002).
- Roster entries may carry a display-only group: `"M. Lukin — Harvard-Lukin"`.
- New config slot `[slack] who_style = "full" | "authors" | "none"`
(default `authors`; `full` fetches each pick's PDF).

## v0.4.1 — 2026-10-12

- `arjev review --days 7`: the week-in-review, a read-only re-read of the
journal (papers that missed the daily cap, tracked-author matches,
Jev-vs-keyword divergences, the week's labels).

## v0.4.0 — 2026-10-08

- The context engine: an opt-in `budget-greedy` state policy that fills a byte
budget with directives and per-paper taste cards (your why, the abstract, the
conclusions), weighted by recency. `arjev calibrate --replay-arms` measures
each arm on your labels; defaults change only on a measured win.
- `arjev fetch` and `arjev backfill` extract PDF text into the library.

## v0.3.0 — 2026-10-02

- Field-neutral: `feeds` defaults to empty and the digest refuses to run until
you choose categories. `arjev init` writes a self-explaining config and never
overwrites an existing one.
- New verbs: `arjev inspect` (verify the config) and `arjev staged` (the notes
awaiting prose).

## Earlier

v0.1–v0.2 (2026-10-01): the first digest, Slack picks and reaction labels,
taste directives, multi-feed digests, `arjev ingest`, and Jev-first ranking.
The original spec and plan are in [`docs/history/`](docs/history/).
Loading
Loading