Repository navigation
feat: Answer dataset listings in JSON-LD when asked - #220
Merged
Merged
Conversation
6 tasks done
NathanCummings
force-pushed
the
feat/dataset-listing-jsonld
branch
from
November 6, 2026 07:54
2c3ed83 to
fe775c7
Compare
NathanCummings
marked this pull request as ready for review
November 6, 2026 07:54
The root, device and shot dataset listings answer in JSON-LD when the Accept header prefers it, by the rule the single records follow. A page is one document: FDS's context and an @graph holding each dataset's dcat:Dataset node in listing order, without FDS's catalogue records. The filters, read access and paging are those of the JSON listing. Each node carries the dataset's replacements as dct:replaces and dct:isReplacedBy, and a replaced dataset is left out unless include_replaced is set, as in the JSON listing. Each Activity names only the inputs the caller may read. The page's Activities, distributions and derivations are loaded for the whole page at once, so a JSON-LD listing issues the same statements whatever the page size. Co-Authored-By: Claude Opus 5.5 <[email protected]>
NathanCummings
force-pushed
the
feat/dataset-listing-jsonld
branch
from
November 6, 2026 08:00
fe775c7 to
a277efb
Compare
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The dataset listings answer in JSON-LD when asked. Until now they returned plain JSON whatever the request asked for, so linked data could only be had one dataset at a time.
They choose between JSON and JSON-LD by the rule the single records follow (#255).
A page is one document: FDS's context, and an
@graphholding adcat:Datasetnode for each dataset on the page, in listing order. Each node is the one in the dataset's own document, with its Activity, distributions and derivations. FDS's catalogue records are left out, as the device catalogue already leaves them out of its listing. The filters, read access andoffset/limitpaging are those of the JSON listing, so another catalogue can, for example, ask for the simulated datasets on one shot:Plain JSON stays the default.
Versions. Each node carries the dataset's replacements,
dct:replacesanddct:isReplacedBy, as its own document does. A replaced dataset is left out unlessinclude_replaced=true, as in the JSON listing.Statements per page. A dataset's node follows its Activity, distributions and derivations, and the Activity's agents, instruments and inputs, through ORM relationships. Mapped one at a time, each would cost a statement per dataset. The listing loads them for the whole page with
selectinloadbefore mapping, so the JSON-LD listings issue the same statements whatever the page size, as the JSON ones do. The Activity inputs on a page are checked for read access together, and each Activity names only those the caller may read, as on a dataset's own document.Code.
map_dataset_to_dcatis split as the device mapping is:dataset_nodebuilds the node alone, andmap_dataset_to_dcatadds the context and FDS's record.DatasetService.to_dcat_listingbuilds a page. SQLModel types a relationship as the objects it holds, which pyright will not accept as a loader option, so a small_attributehelper narrows it with anisinstancecheck, as sqlmodel'scol()does for columns.The JSON-LD page of the docs gains a "Dataset listings" section.
How it was tested
include_replaced=trueboth datasets carry their replacement links (the links fail without the change).test_list_statement_counts.pycovers the JSON-LD listings at the root, under a device (plain and with eachinclude_*) and under a shot: each issues the same statements at page sizes 2 and 4. Without the up-front load all six fail.uv run --all-extras pytest: 977 passed, with fix: Choose JSON or JSON-LD by the Accept header's preferences #255 underneath.GET /v1/devices/mast/shots/30420/datasets?activity_type=simulationin JSON-LD with threedcat:Datasetnodes, which the prototype listed on its Simulations page.Checklist
uv run ruff check .anduv run ruff format --check .passuv run pyrightpassesuv run --all-extras pytestpassesDatasetService; the listings' own read access is unchanged)🤖 Generated with Claude Code