Repository navigation
Utopia: mirror every issue, pull request, tag and release (ontology v5) - #2802
Conversation
|
Lore is reviewing this PR — 17a1ff15-b020-42ce-8248-f2b9b35a8e9a. Submit a review that requests changes to have Lore push a fix for it. Comment |
Lore Spec ImpactLore API unreachable (HTTP 413). This check could not run. |
![lore-agent[bot]](https://avatars.githubusercontent.com/u/141229953?s=60&v=4){kind=small}
There was a problem hiding this comment.
Lore review — Approved
Clean, well-structured implementation of the Utopia issue and pull request tracker mirror under ontology v5; one minor spec wording update suggested.
Must fix (0) · Consider (0) · Nits (1)
Reviewed by gemini-4-argon
Submit a review that requests changes to have Lore push a fix for it. Comment @lore review to re-run the review.
Posted by floor, visit 9a52bb5d-3f9d-417f-8068-da0ad18b4b10.
| - FR5.4 — The key never enters `lore.repos.settings`, which read-scope routes serve whole: it lives in `lore.utopia_sources`, one row per repo matched without regard to case, overwritten by each setup that creates a source, readable by the `lore` role alone and answered by no route. ([validated by leaves no source_key and no utp_ key anywhere in the settings it writes](apps/lore-api/src/transport/routes/repos/utopia-setup.test.ts#L189), [validated by reads the source saved for it-utopia/Lore back as it-utopia/lore, and a second save overwrites the key](apps/lore-api/src/integration-tests/utopia-sources.test.ts#L73), [validated by grants lore every privilege and lore_ui none](apps/lore-api/src/integration-tests/utopia-sources.test.ts#L92)) | ||
|
|
||
| - FR5.5 — The setup calls carry any bearer Utopia accepts as a person: a session JWT or a personal access token with `write` scope (`utp_pat_…`, deeplethe/utopia#1244). The requests are the same; only the bearer differs. The route keeps taking a person's session token, for a deployment that holds no token of its own. ([validated by sends a utp_pat_1 write token as the bearer of the same request](libs/shared/src/outbound/utopia/utopia-client.test.ts#L241)) | ||
|
|
There was a problem hiding this comment.
nit: update FR5.1 prose and coverage link label from fourteen to sixteen kind words to match FR1.1, FR8.1, and utopia-setup.test.ts
| - FR5.1 — `POST /api/repos/{owner}/{repo}/utopia/setup` (admin scope) takes a person's Utopia session token and a workspace id. It finds the knowledge base named `owner/repo` in that workspace or creates it, imports the ontology, binds the sixteen kind words, creates a `statements` source named `lore`, saves the source's key to `lore.utopia_sources`, and writes `settings.utopia` with the knowledge base, the source and the ontology version. It answers the same three, never the key. ([validated by creates the knowledge base, imports the ontology, binds the sixteen kind words, creates the lore source, saves its key to lore.utopia_sources and its ids to settings](apps/lore-api/src/transport/routes/repos/utopia-setup.test.ts#L153), [validated by lists a workspace's knowledge bases under the session token](libs/shared/src/outbound/utopia/utopia-client.test.ts#L230), [validated by creates a knowledge base of kind knowledge with no ontology packs](libs/shared/src/outbound/utopia/utopia-client.test.ts#L251), [validated by imports the ontology as a multipart turtle file named lore-ontology.ttl](libs/shared/src/outbound/utopia/utopia-client.test.ts#L262), [validated by binds a kind word to its class key](libs/shared/src/outbound/utopia/utopia-client.test.ts#L310), [validated by creates a statements source and returns its id with the ingest token as the key](libs/shared/src/outbound/utopia/utopia-client.test.ts#L321)) |
5a974ca to
1dc79d8
Compare
There was a problem hiding this comment.
Lore review — Changes suggested
Ontology v5 test changes in lore-ontology.test.ts broke re-lint/require-spec-link in CI and left lore-api setup/mirror tests asserting version 4 and 14 kind words
Must fix (2) · Consider (0) · Nits (0)
Reviewed by gemini-4-argon
Submit a review that requests changes to have Lore push a fix for it. Comment @lore review to re-run the review.
Posted by floor, visit 94b7542a-e8eb-459f-8a4d-e69084473b5b.
| }); | ||
|
|
||
| it("declares text on Section, Statement and Commit and title on Document and Commit, both strings, so a pushed statement's words and a document's title are attributes the base knows", async () => { | ||
| it("declares text on Section, Statement, Commit, PullRequest, Issue and Release and title on Document, Commit, PullRequest, Issue and Release, both strings, so a pushed statement's words, a pull request's body and a document's title are attributes the base knows", async () => { |
There was a problem hiding this comment.
issue (blocking): Renamed and added ontology v5 tests have no matching spec links in specs/utopia-graph-mirror/spec.md (failing CI format job with re-lint/require-spec-link)
CI's format job failed with two re-lint/require-spec-link errors on libs/shared/src/domain/lore-ontology.test.ts (lines 47 and 151), and lines 9 and 35 were also renamed from 'fourteen class labels' / 'version 4' to 'sixteen class labels' / 'version 5' while specs/utopia-graph-mirror/spec.md (FR1.1, FR1.3, FR12.1) still links the old titles and has no statement linking the new v5 test at line 151.
|
|
||
| /** Bumped when lore-ontology.ttl changes shape; a repo whose settings hold an older number is re-set up. */ | ||
| export const LORE_ONTOLOGY_VERSION = 4; | ||
| export const LORE_ONTOLOGY_VERSION = 5; |
There was a problem hiding this comment.
issue (blocking): Bumping LORE_ONTOLOGY_VERSION to 5 and LORE_KIND_WORDS to 16 entries breaks apps/lore-api Utopia setup and mirror tests
setUpRepoInUtopia, installOntology, and sourceToMirror read LORE_ONTOLOGY_VERSION and LORE_KIND_WORDS directly from @re-cinq/lore-shared/lore-ontology.js. Neither apps/lore-api/src/work/utopia/utopia-mirror.test.ts (which fixtures SETTINGS at ontology_version: 4 and asserts bound: 14 and ontology_version: 4) nor apps/lore-api/src/transport/routes/repos/utopia-setup.test.ts (which asserts bound: 14 and ontology_version: 4) was updated in this commit; those suites were skipped in CI only because the format job failed first.
GitHub's issues listing (which carries pull requests with their merge time) is walked oldest update first from settings.utopia.tracker_since into pull:<n> / issue:<n> items: title, state, author, labels, times, description, and a pull request's closes/refers_to/delivers links, stated through the stub module the commit mirror now shares. Tags (tag:<name>, points_to Commit) and published releases (release:<tag>, publishes Tag) are pushed whole, with no cursor. Both run in the weekly backfill, on POST /api/repos/{owner}/{repo}/utopia/tracker and /releases, and on github.tracker.changed / github.release.changed, which a changed issue or pull request, a published release or a created tag emits under its delivery's :utopia dedupe key; a repo's hook now carries release and create. The commit, tracker and release walks share one cursor loop and one per-repo queue. Spec FR14 and FR15, ADR-052 amendment.
1dc79d8 to
8af31d2
Compare
gedaiu
changed the title
There was a problem hiding this comment.
Lore review — Approved
Ontology v5 tracker and release mirrors, shared cursor-walk/stub abstractions, and webhook/station handlers are clean and well-tested.
Must fix (0) · Consider (0) · Nits (0)
Reviewed by gemini-4-argon
Submit a review that requests changes to have Lore push a fix for it. Comment @lore review to re-run the review.
Posted by floor, visit 4c526fe2-c9d7-48b5-b3fb-bb28d7bfe75b.
The first tracker walk on lore, run right after #2802 deployed, pushed nothing. Its first item, `pull:1`, timed out after 30 seconds: Utopia's database was at its 4-core limit, with connections waiting 2 to 6 seconds. One walk pushes up to 2,000 items, and each is a job that database has to work through, so a first walk of a repo with a long history piles onto an already busy queue. This PR caps a walk at 200 entries, as #2801 did for commits. A long backlog now arrives in batches of 200, each resuming from `tracker_since`. A walk runs for every changed issue or pull request and once in the weekly backfill, so a busy repo like lore catches up within a day, and a quiet one over a few Sundays. Spec FR14.4 and the ADR-052 amendment now say 200. Generated with [Claude Code](https://claude.com/claude-code)
Utopia held each repo's default-branch commits, but a pull request or an issue appeared only as a number and a link, taken from a squash-merge's commit message, and releases and tags not at all. This PR mirrors GitHub's own record of every issue, pull request, tag and published release into the repo's knowledge base (
specs/utopia-graph-mirrorFR14 and FR15, ADR-052 amendment of 2026-10-17).Issues and pull requests. GitHub's issues listing returns pull requests too, with their merge time, so a single walk covers both. It reads oldest update first from a new cursor,
settings.utopia.tracker_since, and pushespull:<n>andissue:<n>items. Each item holds title, state (open, closed or merged), the login that opened it, labels, the open, close and merge times, and the description. A pull request'sCloses/Refs/Lore-Task:lines link it to its issues and task, stated through a module the commit mirror now shares, so both mirrors say the same thing about the same entity. The walk takes up to 2,000 entries per run, moves the cursor past each one that lands, and stops at the first failure.Tags and releases. A
tag:<name>item points to the commit the commit mirror names, and arelease:<tag>item (title, author, publish time, notes) publishes its tag. That way a release reaches the pull requests behind it. A repo has few of these, so every push sends them all, with no cursor: an unchanged item answers unchanged, and a refused one is sent again next time. Drafts are skipped.When it runs.
github.tracker.changed; a published or edited release, or a created tag, emitsgithub.release.changed. Both come from the same delivery under its own dedupe key (github:<id>:utopia).POST /api/repos/{owner}/{repo}/utopia/trackeror/releases.releaseandcreateevents. An existing hook gets them when the ensure step runs for that repo, aspushdid in Webhook: a repo's hook carries push events #2796.cursor-walk.ts).This branch is rebased onto #2801: commit items carry no file names, and the commit walk takes the newest 200.
Ontology version 5. Version 5 adds
ReleaseandTag,points_toandpublishes, and the tracker attributes. Once it deploys, every base below 5 re-imports the ontology on its next push. If re.8 is not live by then, those bases store 5 with every relation typed as a state, so after the apply the reset is5 → 4, not4 → 3.Out of scope: reviews, review comments, conversation comments, and tombstoning deleted issues, releases or tags.
Checks run locally: vitest across libs/shared, apps/lore-api and apps/stations;
tsc --noEmiton every package that imports shared;eslint .with 0 errors;prettier --check.Generated with Claude Code