Repository navigation
Description
At least some of the test cases could benefit from more of some free text with an explanation of either the feature they're exercising or why they're invalid.
Previously, the parse_attributes cases had a _conformance object which had some obsolete information in. The coordinate transformation tests in https://github.com/clbarnes/ome_zarr_transformations_conformance have a sidecar file containing required parts of the test, and also has a description field. I think for the test cases with a zarr hierarchy, such a sidecar file is better than stashing it in the zarr.json. In the oztc repo above, the sidecar is in TOML, but for the sake of consistency I was leaning towards making it JSON.
So, for the different test kinds:
- parse_attributes
"_conformance"key in the root JSON with an object value
- validate_zarr
_conformance.jsonfile in the root of the Zarr hierarchy (maybe.conformance.json?)
- transform_coordinates
_conformance.jsonfile in the root of the Zarr hierarchy- also includes the required information for the test case - source and target coordinate systems, input coordinates, expected output coordinates
These objects could look something like
| field | requirement | type | description |
|---|---|---|---|
| description | MAY | string | description of the test, why it's valid or invalid |
| transform_coordinates | MUST for transform_coordinates tests | object | source and target coordinate systems, tolerances, starting coordinates and expected ending coordinates |
We could potentially add a field for added_on: date. I also considered encoding the test kind/ version/ profile/ validity/ path, but that duplicates information with the path of the test and means more things to update when you copy a test for e.g. a new version. Also, we want to consider users who want to use the test cases from this repo but not the oztest tool (using their own language's preferred test harness); putting more required information into this sidecar means those implementations become more dependent on it.
Activity
tlambert03 commentedon Oct 31, 2026
👋
fwiw, I opened ome/ngff#646 about stable codes for spec rules, which seems somewhat related to this (and mentioned by @kevinyamauchi in #50 (review)). I ran your cases through my branch, and 344 of the 462 invalid ones come back with a code, so there's decent overlap already. Going the other way, 99 of my 163 spec rules with examples never fired on any of those invalid cases, so where a case really is missing, I'm happy to turn my example into a proper one.
clbarnes commentedon Nov 1, 2026
That's a useful Venn diagram to have.
I don't think the conformance tests necessarily need to have a separate case for every spec rule. I think that would quickly become quite onerous, producing several cases for "omit this field", "this field but it's a string" for all combinations of fields. I'm more concerned with the valid cases ("you must be able to parse this") than the invalid ones, because the set of invalid data is infinite and allowing some level of permissive reading is better than encouraging all readers to reject as much data as possible.
If we had test cases for write/ mutation operations, the invalid cases would get a lot more important, because conservative writing is very desirable. But there are other problems with that.