Skip to content

Reproducibility & data-transparency evaluation β€” feedback welcomeΒ #3

Description

@fractastical

Hi πŸ‘‹ β€” I'm building BioEval, an open evaluation of publicly available insect colony / swarm simulations, scoring each on data transparency, code availability, and computational reproducibility. Your project was included, and I wanted to share the results directly and β€” more importantly β€” ask for your feedback. If I've gotten something wrong or missed context, I'd genuinely like to correct it.

Overall: 72.2/100

Dimension Score
Data Disclosure 72
Dataset Resolvability 68
Code Availability 92
Traceability 62
Simulation Clarity 66
Reproducibility Package 82

(Scores include a documented +20 baseline calibration, capped at 100; the overall is the weighted mean of the six dimensions.)

Ways to improve the score:

  • Add an explicit open-source license (e.g., MIT or Apache 2.0) to the repository
  • Provide concrete repository URLs / DOIs for cited inspirations (FORMINDEX, MetaInformAnt, ActiveInferAnts) and any external datasets, and deposit the current repo state to Zenodo for a citable DOI
  • General: pin dependency versions + add a deterministic seed; add a parameters table mapping each constant to its source; tag a release and archive to Zenodo for a DOI; commit one example output with a checksum.

Methodology and full rubric: https://github.com/fractastical/bioinformatics-eval

Thanks for building and open-sourcing this β€” any corrections welcome!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions