Skip to content

Read .zarr folders and plain SpatialData stores in the serverless viewer - #5

Merged
sminot merged 1 commit into
mainfrom
claude/zarr-folder-stores
Sep 29, 2026
Merged

sminot merged 1 commit into
mainfrom
claude/zarr-folder-stores

Conversation

@sminot

@sminot sminot commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

The serverless viewer can now open a .zarr/ folder as well as a .zarr.zip, and it opens SpatialData stores this app did not save. The first user is Cirro: the portal offers View on any .zarr folder in a dataset, for example nf-core/sopa output, and on every .zarr.zip. The viewer either renders the store or says why it can't.

Changes

  • Folder stores. A checkpoint URL whose path ends in / names a folder. Outside embed mode it is read with zarrita's FetchStore. Under an embed host, HostSignedFolderStore (packages/viewer/src/data/folderStore.ts) reads it through the host, because an S3 bucket needs a presigned URL per object:

    • it lists the folder once (list-keys) and then signs keys in batches (sign-keys);
    • a key missing from the listing reads as absent without a request, so S3's 403 for a missing key can't be mistaken for an expired signature;
    • a listed key that answers 403 is re-signed once.

    The new messages are in docs/EMBED_PROTOCOL.md under "Folder stores".

  • Plain SpatialData. A store with no viewer/ sidecar no longer fails with "saved before the serverless viewer existed". plainSpatialData.deriveSidecar builds the sidecar from the consolidated metadata:

    • Table: the first one with obsm/spatial.
    • Image manifests: built from the OME metadata. Contrast defaults are the 99.9th percentile of the coarsest level, as _channel_norm computes them.
    • Placement: each image is placed against the spots the way imaging.pixel_to_world does it. That is, for each coordinate system, try identity and every shapes/labels transform as the spots-to-system map, and keep the best IoU.
    • Default displays: generated like manager.auto_displays, with stable ids so a saved view can name them.
  • Clear failures. A store with nothing to draw raises UnrenderableStoreError saying which of these applies: no table with obsm/spatial, Zarr v2, no consolidated metadata, or not zarr at all. In embed mode that message is the error event the host shows.

  • AnnData encodings sopa writes. Nullable string and integer columns and indexes (values + mask groups) read as categorical or numeric. Dataframe-valued obsm entries such as intensities are skipped. The default color is the first real pandas Categorical rather than any string column, since a string column is as likely a per-cell id.

  • Embed inventory. ready now reports each image's default contrastLimits and the drawable boundary shapes, so a host's contrast sliders and boundary picker start where the canvas does.

  • Version bumped to 1.1.0 in all three manifests and the lockfile.

What a plain store does not get: boundaries, because plain shapes parquet has no bbox covering index. A gene color also reads the whole CSR matrix, since there is no CSC mirror. Both are called out in the user guide.

Screenshots

Screenshot placeholder: the sopa toy .zarr folder open in the Cirro portal's file viewer (image, cells, settings panel)

Screenshot placeholder: the same viewer colored by the nullable-string cell_type column (Tumoral / B / T cell)

Screenshot placeholder: the error state for a Zarr v2 store (a Xenium Explorer cells.zarr.zip)

Verification

Run in the Cirro portal against dev.cirro.bio, with this branch's build served as the spatialdata tool:

Store Result
sopa toy run, .zarr folder (237 objects) Opens; H&E image, cells, region default color; cell_type and leiden color
sopa Xenium bone marrow, .zarr folder (1,902 objects, 2 GB) Opens in ~5 s, 29 requests; gene search works; CD34 colors in ~2–3 s
Xenium Explorer cells.zarr.zip (Zarr v2) "This is a Zarr v2 store…" error, no empty view
Existing checkpoints (Visium, Xenium .sdata.zarr.zip) Unchanged; the Xenium one now reports nucleus_boundaries and a DAPI default contrast of 0–1919

Checked locally against the (untracked) test-data/ plain stores:

  • Xenium: the morphology image lands at 0.2125 µm/px, matching the backend, even though the table's region is pixel-space labels.
  • Visium H&E: read as RGB at identity.

npm run lint shows 0 errors (8 existing warnings). The viewer typecheck, the frontend tsc, npm run test -w spatial-data-studio-frontend (68/68) and npm run build are all clean. New tests:

  • plainSpatialData.test.ts: transforms, reconciliation, sidecar derivation, auto displays, and an end-to-end openCheckpoint of an in-memory store served as a host-signed folder.
  • folderStore.test.ts: batching, absent keys, re-sign on 403, range reads.

The Playwright e2e suite did not run.

After merge, this needs a v1.1.0 tag from main so the release asset exists. The Cirro-tools spatialdata tool is then re-pinned to it.

🤖 Generated with Claude Code

The serverless reader opens a `.zarr/` folder as well as a `.zarr.zip`. A URL
whose path ends in `/` names a folder. Under an embed host whose bucket needs a
presigned URL per object, the viewer lists the folder once and asks the host to
sign keys in batches (new `sign-keys` / `list-keys` messages). A key missing
from the listing reads as absent without a request, so S3's 403 for a missing
key can't pass for an expired signature.

A store without the `viewer/` sidecar, such as nf-core/sopa or spatialdata-io
output, is read as plain SpatialData. `deriveSidecar` picks the first table
with `obsm/spatial` and builds each image's manifest from its OME metadata,
placing it against the cells as `imaging.pixel_to_world` does. It then
generates the displays `manager.auto_displays` would. A store with nothing to
show (no such table, Zarr v2, no consolidated metadata, not zarr) fails with a
message that says which.

Also reads AnnData's nullable string and integer columns and indexes, and skips
dataframe-valued obsm entries. The embed `ready` inventory now reports each
image's default `contrastLimits` and the drawable boundary `shapes`.

Bumps the manifests to 1.1.0.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@sminot
sminot marked this pull request as ready for review September 29, 2026 17:06
@sminot
sminot merged commit ca87f2f into main Sep 29, 2026
6 of 8 checks passed
@sminot
sminot deleted the claude/zarr-folder-stores branch September 29, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant