Record what each backend holds in its metadata - #7
Merged
Merged
Conversation
Every backend faba writes now says which batch it holds and which
command wrote it (data-beans 0.6.23 metadata): `sample` is the batch,
`producer` is `faba {command}`. `faba count` also sets `content` to
`gene_count`, its rows being `{gene}/count/{spliced|unspliced}`.
Readers take the sample from here rather than from the file name.
`ToBackend::to_backend` takes the metadata, so no writer can leave it
out. QC keeps the input's metadata through its filtered matrices; the
gene-level repool keeps `sample` and `producer` and drops `content`.
CI now lints and tests with the `hdf5` feature too: clippy runs with
and without it, and the HDF5 job runs the tests instead of only
checking.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every backend faba writes now records which batch it holds and which command wrote it, using the backend metadata added in data-beans 0.6.23. Readers such as senna can take the sample id from the file itself instead of parsing faba's file names.
Metadata written
sampleproducerfaba {command}:count,dartseq,atoi,apa,snp,depthcontentgene_countfromfaba count(rows{gene}/count/{spliced|unspliced})ToBackend::to_backendtakes the metadata, so every writer states it.backend_meta(command, batch)builds the common part.sampleandproducerand dropscontent, since its rows change.CI
--features hdf5.cargo test --features hdf5instead ofcargo check. clippy with--all-targets --features hdf5covers compiling every target.Tests
to_backendwritessample,producerandcontentas given, and they read back from the stored backend.Version 0.15.4, with data-beans 0.6.23.