Use ac.util.dataset.should_simulate for the 22 auto-simulate guards - #17
Merged
Merged
Conversation
Migrate all 22 dataset auto-simulation guards (21 scripts) from the raw 'if not path.exists(dataset_path):' idiom to 'if ac.util.dataset.should_simulate(dataset_path):'. should_simulate restores the PYAUTO_SMALL_DATASETS=1 force-regeneration behaviour: it deletes an existing dataset directory so the simulator re-runs at capped resolution, instead of silently reusing a full-resolution dataset. This depends on the PyAutoCTI ac.util.dataset export (alias of autoarray's dataset_util) landing on the same-named branch. Also: - fix simulate_datasets_missing.sh, which referenced the nonexistent simulators/overview/non_uniform_charge_injection.py (real file: non_uniform_cosmic_rays.py) - drop the dead first dataset_name assignments (immediately overwritten) in imaging_ci/extract.py and imaging_ci/data_preparation/examples/cosmic_ray_flagging.py Notebooks are not regenerated here: autocti is not registered in PyAutoHands COLAB_PROJECTS, so regeneration happens via the PyAutoHands release pipeline. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Vu1p9rcDQGRrYddB6VWcxN
This was referenced Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pending release — library-first merge gate applies. Depends on PyAutoLabs/PyAutoCTI#106 (
ac.util.datasetexport); do not merge before that lands and the release floor covers it.Migrates all auto-simulate guards from the raw
if not path.exists(dataset_path):idiom toif ac.util.dataset.should_simulate(dataset_path):, restoringPYAUTO_SMALL_DATASETS=1force-regeneration (an existing dataset dir is deleted so the simulator re-creates it at capped resolution). The 2026-07-27 census counted 21 sites; the sweep found 22 sites across 21 files (data_preparation/start_here.pyhas two guards) — all migrated, each confirmed to guard a directory path with a paired simulator subprocess. One line changed per site; pre-existing CRLF endings preserved in the 8 files that have them.Also folds in the leg-6 side findings from the same PyAutoMind prompt (
draft/maintenance/pyauto_cti/autocti_util_dataset_export.md):simulate_datasets_missing.sh: fixed the reference to the nonexistentsimulators/overview/non_uniform_charge_injection.py→non_uniform_cosmic_rays.py(the other referenced path was verified to exist).dataset_name =assignment inscripts/imaging_ci/extract.pyandscripts/imaging_ci/data_preparation/examples/cosmic_ray_flagging.py.Out of scope (left for a sibling task, per the prompt): purging the tracked
dataset/overview/anddataset/dataset_1d/trees.Scripts touched
All under
scripts/:imaging_ci/advanced/chaining/pipelines/{parallel,serial,parallel_x2__serial_x2}.py,imaging_ci/advanced/chaining/x1_species_to_x2_species.py,imaging_ci/{correction,modeling,data_preparation}/start_here.py(data_preparation ×2 sites),imaging_ci/modeling/features/{cosmic_rays,non_uniform,serial_cti,visualize_full}.py,imaging_ci/data_preparation/examples/{bias_subtraction,cosmic_ray_flagging,pre_cti,pre_cti_and_cosmics,serial_cti}.py,imaging_ci/extract.py,plot/plotters/{ImagingCIPlotter,FitImagingCIPlotter,MultiFigurePlotter}.py,plot/diagnostics/imaging_ci/well_filling.py.Validation
Against the sibling PyAutoCTI branch (Python 3.12, arcticpy 2.6):
python -m py_compilepasses on all 21 edited files;grepconfirms zero raw guards remain inscripts/.scripts/imaging_ci/extract.pyrun end-to-end with the dataset absent: the new guard fired, simulateddataset/imaging_ci/simple, script exited 0.PYAUTO_SMALL_DATASETS=1rerun (dir deleted + re-simulated), script exited 0.scripts/imaging_ci/data_preparation/examples/cosmic_ray_flagging.py(guard + dead-assignment removal) exited 0.Notebooks
Not regenerated:
autoctiis not a registered PyAutoHands generate target (absent fromCOLAB_PROJECTSandworkspaces.yaml), and per this workspace's AGENTS.md notebook regeneration runs through the PyAutoHands release pipeline. The 22 matching guard occurrences across 21 notebooks will pick up the change at the next pipeline regen.Could not update
None — all 22 census sites migrated.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Vu1p9rcDQGRrYddB6VWcxN
Generated by Claude Code