Prepare v0.9.2 release metadata and complete documentation

CITATION.cff

@@ -1,8 +1,8 @@
 cff-version: 1.2.0
 message: "If you use this software, please cite it as below."
 title: "Instance Space Analysis: A toolkit for the assessment of algorithmic power"
-version: 0.9.1
-date-released: 2026-08-20
+version: 0.9.2
+date-released: 2026-10-17
 doi: 10.5281/zenodo.4484107
 authors:
   - family-names: "Muñoz"

Contents.m

@@ -1,7 +1,8 @@
 % Instance Space Analysis Toolkit
-% Version 0.9.1 (R2025a) 20-Aug-2026
+% Version 0.9.2 (R2025a) 17-Oct-2026
 %
 % Standalone pipeline functions
+%   INIT        - Read metadata and validate training/evaluation inputs
 %   PRELIM      - Data preprocessing (outlier bounding, normalisation)
 %   SIFTED      - Feature selection (correlation + GA clustering)
 %   PILOT       - 2D/3D projection (PBLDR algorithm)
@@ -21,6 +22,7 @@
 %   exploreIS   - Thin wrapper around InstanceSpace.explore()
 %
 % Utilities
+%   ISAfootprintContains  - Closed-boundary footprint membership with distance tolerance
 %   ISAdefaults          - Fill in missing opts fields with default values
 %   ISAvalidateOpts      - Validate user-supplied opts fields
 %   ISAsubsetData         - Subset rows (and optionally feature columns) of a data struct

README.md

@@ -43,6 +43,8 @@ Or if you specifically use [MATILDA](http://matilda.unimelb.edu.au/matilda/), pl
 
 The reference documentation is at **<https://andremun.github.io/InstanceSpace/>**: a page for every function and for the `InstanceSpace` class, with examples, a getting-started guide, a stage-by-stage walkthrough, the metadata file format, and the full options reference. The same pages open inside MATLAB: with the repository root on the path, open the Help browser and choose **Supplemental Software → Instance Space Analysis Toolbox**.
 
+For v0.9.2 changes, upgrade guidance and remaining limitations, see [What's New](https://andremun.github.io/InstanceSpace/WhatsNew.html) and the [release notes](RELEASE_NOTES.md). Rebuild existing experiments from their original metadata to obtain the corrected fitted results.
+
 ## Installation Instructions
 
 The main requirement for the software to run is to have MATLAB R2025a or later, with the [Global Optimization](https://au.mathworks.com/help/gads/index.html), [Parallel Computing](https://www.mathworks.com/products/parallel-computing.html), [Optimization](https://au.mathworks.com/products/optimization.html), [Statistics and Machine Learning](https://au.mathworks.com/help/stats/index.html), and [Financial](https://au.mathworks.com/products/finance.html) toolboxes installed. The LIBSVM MEX-files (`svmpredict`/`svmtrain`), used for legacy models, are **not bundled with this repository** (no build source is available for them). They're only relevant for evaluating a pre-v0.9.0 model whose classifiers `ISAmigrateModel` couldn't retrain (missing original training data); `ISAmigrateModel` prefers retraining from scratch with the current registry whenever the training data is available, which needs no LIBSVM dependency. If you do hit that path, obtain LIBSVM yourself from [the official project](https://www.csie.ntu.edu.tw/~cjlin/libsvm/) and add its MEX-files to the MATLAB path — `PYTHIA`'s eval mode raises a clear `ISA:PYTHIA:noLibsvm` error naming the missing dependency if you don't.

RELEASE_NOTES.md

@@ -1,13 +1,15 @@
-# Instance Space Analysis Toolkit — v0.9.2 (in development)
+# Instance Space Analysis Toolkit — v0.9.2
 
-Bug fixes from the v0.9.2 backlog, serial GNU Octave support, a reference documentation site, and test infrastructure. It supports **MATLAB R2025a or later** and the bounded Octave workflow described below.
+Corrected training, exploration and reporting contracts, serial GNU Octave support, an independent INIT stage, a reference documentation site, and test infrastructure. It supports **MATLAB R2025a or later** and the bounded Octave workflow described below.
 
 ---
 
 ## New functionality
 
 **Independent INIT stage.** `InstanceSpace` now exposes `init` before `prelim`, including stage callbacks, completion tracking, option validation and saved input data. PRELIM reruns reuse `model.init.data`; explicitly rerunning INIT reloads metadata and invalidates downstream results. Existing staged calls beginning with PRELIM initialize automatically when the snapshot is absent, and legacy models remain usable for exploration. Full builds retain their numerical behaviour.
 
+**Explicit footprint boundary policy.** TRACE uses closed boundaries in 2D and 3D. `opts.trace.boundaryTolerance` is a finite nonnegative Euclidean distance in projected coordinates; its default is **0**, with no implicit numerical margin. The fitted value is saved in the model and reused during exploration and legacy contradiction counting. Older models without a saved tolerance use zero. MATLAB and Python share this policy; matching policies do not guarantee identical floating-point geometry.
+
 **Serial GNU Octave workflow.** Octave supports 2D/3D build → save/load → explore with normalization, feature selection, analytic/numerical/SIMPLS projection, KNN classifiers with none/Sobol tuning, TRACE3 footprints, CSV and PNG. The full workflow requires Octave **11.1 or later**, Datatypes **1.5.0** and Statistics **2.0.0**; validation uses Octave **11.3.0**. Bayesian tuning, parallel execution, other classifiers, legacy TRACE, FIG files and web palette export remain unsupported. 3D PNG requires Qt and a display or virtual display; `outputs.fig` defaults to false in Octave. Build and explore reject unsupported output options before processing or replacing outputs.
 
 Octave saves versioned MAT v7 archives that preserve trained KNN models without retraining; these classifier archives are runtime-specific, while MATLAB retains native MAT v7.3 persistence. Schema 2 protects ordinary structs from collisions with internal type markers and reads existing schema-1 archives; ambiguous marker-shaped user structs in old archives cannot be distinguished retrospectively. CSV is the supported data interchange format. Equal seeds do not promise identical optimizer, Sobol or geometry results across runtimes. See the [Octave compatibility guide](docs/octave-compatibility.md) for setup and the complete support boundary.
@@ -29,11 +31,31 @@ Octave saves versioned MAT v7 archives that preserve trained KNN models without
 ## Bug fixes
 
 - **PYTHIA evaluation scored algorithms against data that was never observed.** In `PYTHIA`'s evaluation mode, a trained algorithm absent from `metadata_test.csv` has an all-`NaN` performance column, which `PRELIM` turns into all-false labels; accuracy, precision and recall were then computed against those labels. Only instances with observed performance are now scored, and an algorithm with none reports `NaN`. Accuracy is divided by the number of scored instances, and the summary's per-algorithm `Probability_of_good` is also computed over observed instances only (blank when there are none) ([#58](https://github.com/andremun/InstanceSpace/issues/58)).
-- **Oracle probability of good performance was always 1.** The Oracle row of `PYTHIA`'s summary now reports `mean(any(Ybin,2))`, the fraction of instances on which any algorithm is good. This is below 1 when `opts.perf.AbsPerf = true` and no algorithm meets the threshold on some instances ([#59](https://github.com/andremun/InstanceSpace/issues/59)).
+- **Oracle probability of good performance was always 1.** The Oracle row of `PYTHIA`'s summary now reports the fraction of instances with any observed performance on which at least one observed algorithm is good (equivalent to `mean(any(Ybin,2))` when all outcomes are observed). Rows with no observations are excluded; no observations gives `NaN`. This is below 1 when `opts.perf.AbsPerf = true` and no algorithm meets the threshold on some instances ([#59](https://github.com/andremun/InstanceSpace/issues/59)).
 - **`TRACE` failed when called without PYTHIA predictions.** `TRACE`'s documentation allows `Yhat = []`, falling back to the true labels, but the footprint loop indexed `Yhat(:,i)` inside a `parfor`, which slices it for every iteration and so failed with an out-of-bounds index. The pipeline itself never passed `[]`, so only direct callers were affected. Predictions are now split per algorithm before the loop.
 - **`ISArecallView` on a figure with no axes** failed inside `ismember` instead of raising its own `ISA:ISArecallView:noAxes` error.
 - **Footprint holes were dropped from the traced boundary.** `traceOneRegion` followed only the boundary cycle that contained its start vertex, so the boundary of a hole inside a footprint region was omitted (with the `ISA:scriptfcn:boundaryHoleOmitted` warning). Every cycle is now traced, and cycles are separated by a `NaN` row as regions already were; the warning is removed ([#52](https://github.com/andremun/InstanceSpace/issues/52)).
-- **[Behaviour-changing] `PYTHIA`'s SVM posterior probabilities were not reproducible.** In `fitOneClassifier`'s `'svm'` case, `fitSVMPosterior`'s own sigmoid-calibration draws depended on however many random numbers `fitcsvm`'s SMO solver had just consumed, so `Pr0hat`/`Pr0sub` for `opts.classifier = 'svm'` could differ between two runs with the same `opts.seed`. `fitOneClassifier` now reseeds to the same per-algorithm/per-fold seed immediately before `fitSVMPosterior`, mirroring the #41 fix for PILOT/SIFTED/PILOTviewpoint. Existing saved SVM models are unaffected; rerunning `PYTHIA` training with `opts.classifier = 'svm'` will change `Pr0hat`/`Pr0sub` (not `Yhat`/`Ysub`, which threshold at 0.5 and were already reproducible) ([#61](https://github.com/andremun/InstanceSpace/issues/61)).
+- **[Behaviour-changing] `PYTHIA`'s SVM posterior probabilities were not reproducible.** In `fitOneClassifier`'s `'svm'` case, `fitSVMPosterior`'s own sigmoid-calibration draws depended on however many random numbers `fitcsvm`'s SMO solver had just consumed, so `Pr0hat`/`Pr0sub` for `opts.classifier = 'svm'` could differ between two runs with the same `opts.seed`. `fitOneClassifier` now reseeds to the same per-algorithm/per-fold seed immediately before `fitSVMPosterior`, mirroring the #41 fix for PILOT/SIFTED/PILOTviewpoint. Existing saved SVM models are unaffected; rerunning `PYTHIA` training with `opts.classifier = 'svm'` may change `Pr0hat`/`Pr0sub` and can also affect predicted labels, tuning decisions and downstream footprints through the calibrated classifier ([#61](https://github.com/andremun/InstanceSpace/issues/61)).
+
+## Further correctness fixes
+
+- **Training data and portfolio consistency.** After algorithms that are never good are pruned, PRELIM recomputes labels, winners, transforms and portfolio indices from the retained raw outcomes. Constant columns use a unit scale. Training inputs remaining incomplete after ingestion filtering raise errors identifying the instances; test outcomes can remain unobserved. Raw performance must be nonnegative, with zero allowed; the original relative ratio formulas and documented zero substitution are retained.
+- **Repeatable feature selection and projection.** SIFTED resets its fitness caches between runs, including workers, and retains pre-selection inputs for reruns. PLS persists its fitted feature mean and applies it during exploration and CLOISTER projection. Analytic PILOT solves least squares directly, and standalone numerical fallbacks receive complete defaults. Seed bounds and per-stage seeding are validated.
+- **Training/test separation and honest summaries.** Recommendations use saved training fallback choices and precision weights, regardless of test outcomes. Selector summaries use out-of-fold selections for training and explicitly labelled test metrics for exploration. Selector recall counts disjoint successes and missed opportunities; cost-sensitive weights use per-instance regret. Invalid classifier CV searches fail explicitly. Score metadata distinguishes posterior probabilities from raw scores and mixed fold outputs. These CV summaries remain conditional on fitted preprocessing, feature selection, projection and tuning; they are not outer cross-validation of the complete pipeline.
+- **Staged builds and persistence.** Options consumed by each stage are saved with provenance. Partial models can be saved, loaded and resumed; incompatible option edits cannot silently retain stale fitted artifacts. Training feature exclusions are replayed during exploration. Builds respect serial settings and close only parallel pools they created.
+- **Footprint evaluation and exports.** Exploration reuses trained space geometry and accepts single, repeated or lower-dimensional test batches. TRACE records whether the purity target was achieved and why its search stopped; a retained candidate below the target is not labelled accepted. CSV exports include 3D vertices and triangle faces, explicit empty geometry and `geometry_manifest.json`. Export replaces obsolete toolkit-owned geometry and plot files; user files must use other names. Viewpoint generation and recall use the same camera convention and equal data-axis scales.
+
+## Upgrading from v0.9.1
+
+Keep a backup of existing models and result directories. Rebuild experiments from the original metadata to obtain the corrected preprocessing, feature selection, PLS coordinates, classifiers, selector summaries and footprints; loading an old model does not retroactively apply training fixes. Rebuild CLOISTER to obtain a valid stored 3D boundary. Models predating the saved pre-SIFTED snapshot must rebuild preprocessing before rerunning feature selection. `ISAmigrateModel` remains available for pre-v0.9.0 layouts, but migration is not a substitute for rebuilding corrected experiments.
+
+Options now reject ambiguous subsetting modes, invalid index files, unsupported seed values and negative raw performance earlier. Consumers of exported geometry should read the manifest, preserve NaN-separated 2D rings and use one-based 3D face indices. Saved Octave classifiers are runtime-specific; use CSV for data interchange.
+
+## Known limitations
+
+Numerical PILOT restart selection can amplify small floating-point differences across BLAS implementations or thread settings ([#63](https://github.com/andremun/InstanceSpace/issues/63)). A fixed seed is not a guarantee of identical numerical projections across environments. Keep runtime, libraries and thread settings fixed for reproducibility-sensitive work. This issue remains open. PILOT does not implement the optional canonical rotation proposed in closed issue #30; projections can have different orientations. The proposed rotation feature is excluded from this release.
+
+`opts.pilot.topoWeight` remains reserved and disabled. Octave support is limited to the workflow above. Geometric degeneracies and optimizer differences can produce different MATLAB/Python/Octave results even when the same boundary policy is used.
 
 ---
 

doc/generate.py

@@ -83,6 +83,7 @@
     ("Utilities", "FunctionList", [
         ("ISAdefaults", "ISAdefaults"),
         ("ISAvalidateOpts", "ISAvalidateOpts"),
+        ("ISAfootprintContains", "ISAfootprintContains"),
         ("ISAgetClassifierFcn", "ISAgetClassifierFcn"),
         ("ISAmigrateModel", "ISAmigrateModel"),
         ("ISAsubsetData", "ISAsubsetData"),

doc/html/404.html

@@ -13,7 +13,7 @@
 <header class="topbar">
   <button class="menu" type="button" aria-label="Show contents" aria-expanded="false">&#9776;</button>
   <a class="brand" href="index.html">Instance Space Analysis Toolbox</a>
-  <span class="version">v0.9.1</span>
+  <span class="version">v0.9.2</span>
   <div class="search" role="search">
     <input type="search" id="search" placeholder="Search documentation" aria-label="Search documentation" autocomplete="off">
     <ul id="search-results" hidden></ul>
@@ -49,6 +49,7 @@
 <li><details><summary>Utilities</summary><ul>
 <li><a href="ISAdefaults.html">ISAdefaults</a></li>
 <li><a href="ISAvalidateOpts.html">ISAvalidateOpts</a></li>
+<li><a href="ISAfootprintContains.html">ISAfootprintContains</a></li>
 <li><a href="ISAgetClassifierFcn.html">ISAgetClassifierFcn</a></li>
 <li><a href="ISAmigrateModel.html">ISAmigrateModel</a></li>
 <li><a href="ISAsubsetData.html">ISAsubsetData</a></li>
@@ -127,6 +128,7 @@ <h1>Instance Space Analysis Toolbox</h1>
 addpath(pwd)     <span class="com">% the class, the wrappers, and info.xml for the Help browser</span>
 startup          <span class="com">% adds core/, utils/, output/ and deprecated/</span></code></pre></div>
 <p>Required products: MATLAB R2025a or later with the Statistics and Machine Learning, Optimization, Global Optimization, Parallel Computing, and Financial toolboxes.</p>
+<p>A bounded serial GNU Octave workflow is also supported; see <a href="WhatsNew.html">What's New</a> for supported classifiers, outputs and persistence limits.</p>
 <p>With the repository root on the path, this documentation also appears in the MATLAB Help browser under <strong>Supplemental Software</strong>.</p></section>
 <section class="refsect" id="references"><h2>References</h2><ul>
 <li>Smith-Miles, K. &amp; Muñoz, M.A. (2023). Instance Space Analysis for Algorithm Testing. <em>ACM Computing Surveys</em>, 55(12), Article 255. <a href="https://doi.org/10.1145/3572895">https://doi.org/10.1145/3572895</a></li>

doc/html/CLOISTER.html

@@ -13,7 +13,7 @@
 <header class="topbar">
   <button class="menu" type="button" aria-label="Show contents" aria-expanded="false">&#9776;</button>
   <a class="brand" href="index.html">Instance Space Analysis Toolbox</a>
-  <span class="version">v0.9.1</span>
+  <span class="version">v0.9.2</span>
   <div class="search" role="search">
     <input type="search" id="search" placeholder="Search documentation" aria-label="Search documentation" autocomplete="off">
     <ul id="search-results" hidden></ul>
@@ -49,6 +49,7 @@
 <li><details><summary>Utilities</summary><ul>
 <li><a href="ISAdefaults.html">ISAdefaults</a></li>
 <li><a href="ISAvalidateOpts.html">ISAvalidateOpts</a></li>
+<li><a href="ISAfootprintContains.html">ISAfootprintContains</a></li>
 <li><a href="ISAgetClassifierFcn.html">ISAgetClassifierFcn</a></li>
 <li><a href="ISAmigrateModel.html">ISAmigrateModel</a></li>
 <li><a href="ISAsubsetData.html">ISAsubsetData</a></li>

doc/html/Deprecated.html

@@ -13,7 +13,7 @@
 <header class="topbar">
   <button class="menu" type="button" aria-label="Show contents" aria-expanded="false">&#9776;</button>
   <a class="brand" href="index.html">Instance Space Analysis Toolbox</a>
-  <span class="version">v0.9.1</span>
+  <span class="version">v0.9.2</span>
   <div class="search" role="search">
     <input type="search" id="search" placeholder="Search documentation" aria-label="Search documentation" autocomplete="off">
     <ul id="search-results" hidden></ul>
@@ -49,6 +49,7 @@
 <li><details><summary>Utilities</summary><ul>
 <li><a href="ISAdefaults.html">ISAdefaults</a></li>
 <li><a href="ISAvalidateOpts.html">ISAvalidateOpts</a></li>
+<li><a href="ISAfootprintContains.html">ISAfootprintContains</a></li>
 <li><a href="ISAgetClassifierFcn.html">ISAgetClassifierFcn</a></li>
 <li><a href="ISAmigrateModel.html">ISAmigrateModel</a></li>
 <li><a href="ISAsubsetData.html">ISAsubsetData</a></li>

doc/html/FILTER.html

@@ -13,7 +13,7 @@
 <header class="topbar">
   <button class="menu" type="button" aria-label="Show contents" aria-expanded="false">&#9776;</button>
   <a class="brand" href="index.html">Instance Space Analysis Toolbox</a>
-  <span class="version">v0.9.1</span>
+  <span class="version">v0.9.2</span>
   <div class="search" role="search">
     <input type="search" id="search" placeholder="Search documentation" aria-label="Search documentation" autocomplete="off">
     <ul id="search-results" hidden></ul>
@@ -49,6 +49,7 @@
 <li><details><summary>Utilities</summary><ul>
 <li><a href="ISAdefaults.html">ISAdefaults</a></li>
 <li><a href="ISAvalidateOpts.html">ISAvalidateOpts</a></li>
+<li><a href="ISAfootprintContains.html">ISAfootprintContains</a></li>
 <li><a href="ISAgetClassifierFcn.html">ISAgetClassifierFcn</a></li>
 <li><a href="ISAmigrateModel.html">ISAmigrateModel</a></li>
 <li><a href="ISAsubsetData.html">ISAsubsetData</a></li>