[mlakolar]

Summary

The simulation scripts in exper/ were written for Julia 1.5, JLD, PyPlot and an older package API, and several no longer ran. This updates all of them for the current package and adds an environment and a README for exper/. Package code is unchanged.

Environment and cluster jobs

• exper/Project.toml: DiffPrecTest from this repository and CovSel from GitHub via [sources] (Julia 1.11+; the README gives the Julia 1.10 steps)
• exper/README.md: the simulations, and how to run, collect and summarize them
• JLD2 replaces JLD and Plots replaces PyPlot; each script loads only what it uses
• sbatch_*: JULIA and SCRATCH come from the environment instead of Midway2 paths. Julia runs with --project, the environment is instantiated and precompiled once before the workers start, and BLAS uses one thread per worker. The partition is no longer hard-coded: ./start_sbatch.sh -p <partition> passes its arguments to sbatch
• start_sbatch.sh for sims 8–11 submits both estimated (simN) and oracle (simN_o) runs, with different job names so their GNU parallel job logs don't collide
• collect.sh loops over the coverage simulations; collect_results.jl and aggregate_results.jl skip missing repetitions with a warning
• simN_YinXia*.jl read NUM_REP from the environment for short runs

Fixes

• The oracle scripts of sims 8–12 passed supports as Bool matrices, which bootstrap no longer accepts; they now pass svec indices, one copy per entry (bootstrap reorders each support)
• sim8_YinXia.jl looped over four dimensions with three defined; sim12_YinXia_power.jl skipped α = 1
• The coverage tables threw an InexactError whenever coverage × 1000 was not an exact integer, and LaTeXTabulars 1.0 escapes raw \multirow strings; the tables now use MultiRow cells
• sim5/summarize.jl rebuilt a p = 20 model that matched neither sim5a (p = 60) nor sim5b (p = 100) and read from a hard-coded home folder. It now takes the results folder and variant, averages ROC curves for the normal, bootstrap and D-trace methods, and reports the D-trace rates at the validated penalties. The model is shared with sim5a.jl and sim5b.jl in sim5/model.jl
• The summaries of sims 1–4, 6 and 7 share summary.jl

Changes to review

• sim8.jl and sim10.jl use estimated supports again. The 2019 "oracle sim" commit, which created the _o copies, also switched these two to oracle supports, while sim9.jl and sim11.jl kept estimated ones. sim8.jl now uses p = 50, 100, 150, the dimensions its start script ran
• Removed sim1/test.jl, sim2/sim2_test.jl and sim3/test.jl: scratch scripts that called functions that no longer exist or used undefined variables
• In the tables, numbers are in math mode (LaTeXTabulars 1.0 default) and the true values of Δ are rounded to three digits

Testing

• One repetition of every simulation script ran without errors on Julia 1.13: the coverage simulations at full size, sims 8–12 at p = 50, sim5 and the YinXia scripts at p = 20
• collect.sh, all summaries, plot_results.jl and the sim8 coverage scripts ran on those results
• The refactored model code reproduces the original models exactly (sim5 for p = 60 and 100; the true Δ in the summaries of sims 3, 4, 6 and 7)
• No Slurm cluster was available: the sbatch files and start scripts were run with stand-in sbatch, srun and parallel commands, which checked the script names, arguments and output folders of all 93 submissions

Julia's default random number generator changed in Julia 1.7, so the same seeds give different data than the 2019–2020 runs.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.64%. Comparing base (c6acff9) to head (744acfb).

Additional details and impacted files

@@           Coverage Diff           @@
##           master       #3   +/-   ##
=======================================
  Coverage   98.64%   98.64%           
=======================================
  Files          12       12           
  Lines        1403     1403           
=======================================
  Hits         1384     1384           
  Misses         19       19           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.