Read .zarr folders and plain SpatialData stores in the serverless viewer - #5
Merged
Merged
Conversation
The serverless reader opens a `.zarr/` folder as well as a `.zarr.zip`. A URL whose path ends in `/` names a folder. Under an embed host whose bucket needs a presigned URL per object, the viewer lists the folder once and asks the host to sign keys in batches (new `sign-keys` / `list-keys` messages). A key missing from the listing reads as absent without a request, so S3's 403 for a missing key can't pass for an expired signature. A store without the `viewer/` sidecar, such as nf-core/sopa or spatialdata-io output, is read as plain SpatialData. `deriveSidecar` picks the first table with `obsm/spatial` and builds each image's manifest from its OME metadata, placing it against the cells as `imaging.pixel_to_world` does. It then generates the displays `manager.auto_displays` would. A store with nothing to show (no such table, Zarr v2, no consolidated metadata, not zarr) fails with a message that says which. Also reads AnnData's nullable string and integer columns and indexes, and skips dataframe-valued obsm entries. The embed `ready` inventory now reports each image's default `contrastLimits` and the drawable boundary `shapes`. Bumps the manifests to 1.1.0. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The serverless viewer can now open a
.zarr/folder as well as a.zarr.zip, and it opens SpatialData stores this app did not save. The first user is Cirro: the portal offers View on any.zarrfolder in a dataset, for example nf-core/sopa output, and on every.zarr.zip. The viewer either renders the store or says why it can't.Changes
Folder stores. A checkpoint URL whose path ends in
/names a folder. Outside embed mode it is read with zarrita'sFetchStore. Under an embed host,HostSignedFolderStore(packages/viewer/src/data/folderStore.ts) reads it through the host, because an S3 bucket needs a presigned URL per object:list-keys) and then signs keys in batches (sign-keys);The new messages are in
docs/EMBED_PROTOCOL.mdunder "Folder stores".Plain SpatialData. A store with no
viewer/sidecar no longer fails with "saved before the serverless viewer existed".plainSpatialData.deriveSidecarbuilds the sidecar from the consolidated metadata:obsm/spatial._channel_normcomputes them.imaging.pixel_to_worlddoes it. That is, for each coordinate system, try identity and every shapes/labels transform as the spots-to-system map, and keep the best IoU.manager.auto_displays, with stable ids so a saved view can name them.Clear failures. A store with nothing to draw raises
UnrenderableStoreErrorsaying which of these applies: no table withobsm/spatial, Zarr v2, no consolidated metadata, or not zarr at all. In embed mode that message is theerrorevent the host shows.AnnData encodings sopa writes. Nullable string and integer columns and indexes (
values+maskgroups) read as categorical or numeric. Dataframe-valuedobsmentries such asintensitiesare skipped. The default color is the first real pandas Categorical rather than any string column, since a string column is as likely a per-cell id.Embed inventory.
readynow reports each image's defaultcontrastLimitsand the drawable boundaryshapes, so a host's contrast sliders and boundary picker start where the canvas does.Version bumped to
1.1.0in all three manifests and the lockfile.What a plain store does not get: boundaries, because plain shapes parquet has no
bboxcovering index. A gene color also reads the whole CSR matrix, since there is no CSC mirror. Both are called out in the user guide.Screenshots
Verification
Run in the Cirro portal against dev.cirro.bio, with this branch's build served as the
spatialdatatool:.zarrfolder (237 objects)regiondefault color;cell_typeandleidencolor.zarrfolder (1,902 objects, 2 GB)cells.zarr.zip(Zarr v2).sdata.zarr.zip)nucleus_boundariesand a DAPI default contrast of 0–1919Checked locally against the (untracked)
test-data/plain stores:npm run lintshows 0 errors (8 existing warnings). The viewer typecheck, the frontendtsc,npm run test -w spatial-data-studio-frontend(68/68) andnpm run buildare all clean. New tests:plainSpatialData.test.ts: transforms, reconciliation, sidecar derivation, auto displays, and an end-to-endopenCheckpointof an in-memory store served as a host-signed folder.folderStore.test.ts: batching, absent keys, re-sign on 403, range reads.The Playwright e2e suite did not run.
After merge, this needs a
v1.1.0tag frommainso the release asset exists. The Cirro-toolsspatialdatatool is then re-pinned to it.🤖 Generated with Claude Code