Skip to content

feat: Answer dataset listings in JSON-LD when asked - #220

Merged
NathanCummings merged 1 commit into
mainfrom
feat/dataset-listing-jsonld
Nov 6, 2026
Merged

NathanCummings merged 1 commit into
mainfrom
feat/dataset-listing-jsonld

Conversation

@NathanCummings

@NathanCummings NathanCummings commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

What this changes

The dataset listings answer in JSON-LD when asked. Until now they returned plain JSON whatever the request asked for, so linked data could only be had one dataset at a time.

GET /v1/datasets
GET /v1/devices/{device}/datasets
GET /v1/devices/{device}/shots/{shot_id}/datasets
Accept: application/ld+json

They choose between JSON and JSON-LD by the rule the single records follow (#255).

A page is one document: FDS's context, and an @graph holding a dcat:Dataset node for each dataset on the page, in listing order. Each node is the one in the dataset's own document, with its Activity, distributions and derivations. FDS's catalogue records are left out, as the device catalogue already leaves them out of its listing. The filters, read access and offset/limit paging are those of the JSON listing, so another catalogue can, for example, ask for the simulated datasets on one shot:

GET /v1/devices/mast/shots/30421/datasets?activity_type=simulation
Accept: application/ld+json

Plain JSON stays the default.

Versions. Each node carries the dataset's replacements, dct:replaces and dct:isReplacedBy, as its own document does. A replaced dataset is left out unless include_replaced=true, as in the JSON listing.

Statements per page. A dataset's node follows its Activity, distributions and derivations, and the Activity's agents, instruments and inputs, through ORM relationships. Mapped one at a time, each would cost a statement per dataset. The listing loads them for the whole page with selectinload before mapping, so the JSON-LD listings issue the same statements whatever the page size, as the JSON ones do. The Activity inputs on a page are checked for read access together, and each Activity names only those the caller may read, as on a dataset's own document.

Code. map_dataset_to_dcat is split as the device mapping is: dataset_node builds the node alone, and map_dataset_to_dcat adds the context and FDS's record. DatasetService.to_dcat_listing builds a page. SQLModel types a relationship as the objects it holds, which pyright will not accept as a loader option, so a small _attribute helper narrows it with an isinstance check, as sqlmodel's col() does for columns.

The JSON-LD page of the docs gains a "Dataset listings" section.

How it was tested

  • Router tests: the shot listing answers in JSON-LD with the activity filter applied and no catalogue records; the root listing's Activity names only the inputs an anonymous caller may read (it fails with the read check removed); a replaced dataset is left out of the JSON-LD page, and with include_replaced=true both datasets carry their replacement links (the links fail without the change).
  • test_list_statement_counts.py covers the JSON-LD listings at the root, under a device (plain and with each include_*) and under a shot: each issues the same statements at page sizes 2 and 4. Without the up-front load all six fail.
  • uv run --all-extras pytest: 977 passed, with fix: Choose JSON or JSON-LD by the Accept header's preferences #255 underneath.
  • An earlier version of this branch, run as a second FDS in containers beside a prototype of federated search, answered GET /v1/devices/mast/shots/30420/datasets?activity_type=simulation in JSON-LD with three dcat:Dataset nodes, which the prototype listed on its Simulations page.

Checklist

  • uv run ruff check . and uv run ruff format --check . pass
  • uv run pyright passes
  • uv run --all-extras pytest passes
  • Tests cover the change
  • Documentation updated, if the change is user-visible
  • Authorisation checks, if any, live in the service layer rather than the router (the Activity-input check is in DatasetService; the listings' own read access is unchanged)

🤖 Generated with Claude Code

@NathanCummings
NathanCummings force-pushed the feat/dataset-listing-jsonld branch from 2c3ed83 to fe775c7 Compare November 6, 2026 07:54
@NathanCummings
NathanCummings changed the base branch from main to fix/content-negotiation November 6, 2026 07:54
@NathanCummings
NathanCummings marked this pull request as ready for review November 6, 2026 07:54
Base automatically changed from fix/content-negotiation to main November 6, 2026 07:57
The root, device and shot dataset listings answer in JSON-LD when the
Accept header prefers it, by the rule the single records follow. A page
is one document: FDS's context and an @graph holding each dataset's
dcat:Dataset node in listing order, without FDS's catalogue records.
The filters, read access and paging are those of the JSON listing.

Each node carries the dataset's replacements as dct:replaces and
dct:isReplacedBy, and a replaced dataset is left out unless
include_replaced is set, as in the JSON listing. Each Activity names
only the inputs the caller may read. The page's Activities,
distributions and derivations are loaded for the whole page at once, so
a JSON-LD listing issues the same statements whatever the page size.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@NathanCummings
NathanCummings force-pushed the feat/dataset-listing-jsonld branch from fe775c7 to a277efb Compare November 6, 2026 08:00
@NathanCummings
NathanCummings added this pull request to the merge queue Nov 6, 2026
Merged via the queue into main with commit e75a668 Nov 6, 2026
14 checks passed
@NathanCummings
NathanCummings deleted the feat/dataset-listing-jsonld branch November 6, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant