Skip to content

Add reproducible chronological validation support - #157

Open
sergioald wants to merge 4 commits into
GeoOcean:developfrom
sergioald:add-reproducible-validation-support
Open

Add reproducible chronological validation support#157
sergioald wants to merge 4 commits into
GeoOcean:developfrom
sergioald:add-reproducible-validation-support

Conversation

@sergioald

Copy link
Copy Markdown
Collaborator

Summary

This PR adds reproducible chronological train/validation/test splitting support for time-dependent datasets.

The implementation is designed to make temporal validation explicit, deterministic, replayable, and resistant to accidental leakage across split boundaries.

What is included

  • deterministic chronological train/validation/test splits
  • fraction-based and explicit split boundaries
  • numeric and datetime coordinates
  • NumPy datetime64 support
  • timezone-aware pandas timestamps normalized to UTC
  • support for point samples and [start_time, end_time] intervals
  • strict exclusion of intervals crossing split boundaries
  • optional sample-count gaps / embargoes between partitions
  • immutable NumPy index arrays
  • deterministic validation manifests
  • SHA-256 fingerprints of ordered time coordinates
  • manifest replay with dataset-order/content verification
  • validation of malformed, overlapping, or inconsistent boundaries
  • documentation and regression coverage

Reproducibility

ValidationSplitManifest records the split definition together with a fingerprint of the ordered temporal coordinates.

apply_split_manifest() verifies the fingerprint before reproducing the split, so reordered or changed datasets are rejected rather than silently producing a different validation partition.

Scope

This PR intentionally focuses on a small reusable chronological-validation primitive.

It does not add:

  • rolling-origin or walk-forward validation
  • expanding-window validation
  • grouped/station-level splitting
  • benchmark execution
  • dataset-specific validation logic

Those can be considered separately if useful.

Validation

The implementation was checked locally before submission.

  • focused validation suite: 112 passed
  • complete repository test suite: 674 passed
  • git diff --check: passed
  • Ruff/static checks: passed

No unrelated changes are included.

@sergioald
sergioald requested a review from tausiaj August 7, 2026 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant