RLT-RSI Experiment

This project trains and compares computer models that reuse layers to classify whether an on/off sequence contains an odd or even number of on values.

Note

Status: Experimental research prototype (v0.3.0).

• Compares standard models with models that repeat the same layer 1, 2, or 4 times.
• Checks whether models trained on short sequences work on longer ones.
• Keeps training, validation, and test examples separate.
• Includes a bounded search that adjusts how many times a layer runs.
• Offers a quick NumPy check and full PyTorch training.

What the committed results show

In the committed runs, no configuration learns parity. In the recorded 80-epoch PyTorch runs (MPS, CPU), mean held-out accuracy is 0.464-0.521 and mean train accuracy is 0.49-0.59 for every configuration, and held-out loss is about ln 2 (0.693), the loss of always predicting 0.5. Differences between the standard model and the looped models are differences between numbers near chance, with N=3 seeds and 128 held-out examples. The "improvement", "flat" and "regression" labels and the H1/H2 verdicts in those reports follow an operational rule and are not evidence that looping helps or that the models generalize to longer sequences. No RSI-adaptation result is committed yet: the adaptation code has tests, but no adaptation run output is stored under results/.

Quick start

You need Git and Python 3.9+ (Python downloads). The NumPy path runs on CPU without model weights or an API key. Package installation needs network access.

git clone https://github.com/rustfuture/rlt-rsi-experiment.git
cd rlt-rsi-experiment

# Setup (macOS/Linux Bash or zsh)
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[dev]'

# NumPy smoke pipeline (frozen features; fast CPU check)
python3 -m rlt_rsi.train --backend numpy --seeds 7,42,123 \
    --train-size 64 --dev-size 32 --heldout-size 32 --epochs 4 \
    --output runs/quickstart/smoke.json

More commands

# Run test suite
python -m pytest -q

# Adaptation run (NumPy backend); writes a separate local run record
python3 -m rlt_rsi.train_rsi --backend numpy --seeds 7,42,123 \
    --train-size 64 --dev-size 32 --heldout-size 32 --epochs 4 \
    --rsi-generations 2 --rsi-epochs-per-gen 2 \
    --output runs/quickstart/rsi_smoke.json

On Windows PowerShell, replace python3 with py -3 for environment creation and activate with .\.venv\Scripts\Activate.ps1; after activation use python for the remaining commands. If activation is restricted, call .\.venv\Scripts\python.exe directly; no policy change is required. See the Python venv guide.

Each training command prints a JSON summary and writes JSON plus a Markdown report under runs/quickstart/. This directory is gitignored, so the examples do not replace committed research results. A smoke run exercises the pipeline; it does not demonstrate learned parity.

Optional end-to-end training

Install PyTorch separately before this longer run. Model and report artifacts stay under runs/quickstart/:

python -m pip install -e '.[torch]'
python -m rlt_rsi.train --backend torch --device auto --seeds 7,42,123 --epochs 80 \
    --artifacts-dir runs/quickstart/torch-run

To run in a browser without local setup, open notebooks/rlt_rsi_colab.ipynb in Google Colab.

The default run compares a single-pass baseline with looped models. See the reference notes for configurations and extended commands.

How it works

• The data code creates separate training and validation examples with lengths 4–8, then held-out examples with lengths 12–16.
• The model turns each input into a vector, processes all positions with one shared layer, then predicts whether the number of on values is odd or even.
• The baseline runs the block once; looped models repeat the same block with shared weights.
• Baseline and looped runs fit on training examples and evaluate on held-out examples; RSI additionally selects loop counts using validation data.
• RSI (recursive self-improvement style search) tests nearby loop counts and evaluates the final choice on held-out examples after selection. Details are in the adaptation notes.

Tests

# PyTorch CPU job
python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
python -m pip install -e '.[dev]'
python -c "import torch; print(torch.__version__)"
pytest -q

# NumPy test job
python -m pip install -e '.[dev]'
pytest -q
python -m rlt_rsi.train --backend numpy --seeds 7,42,123 --train-size 64 --dev-size 32 --heldout-size 32 --epochs 4 --output results/ci.json
python -m rlt_rsi.train_rsi --backend numpy --seeds 7,42,123 --train-size 64 --dev-size 32 --heldout-size 32 --epochs 4 --rsi-generations 2 --rsi-epochs-per-gen 2 --output results/rsi_ci.json

The NumPy CI job runs on Python 3.11; a separate job installs CPU PyTorch. Tests cover split isolation, model parameter counts, length generalization, reports, and training and adaptation on both backends.

RSI-Style Iterative Adaptation

rlt_rsi.train_rsi changes only the loop schedule, selects candidates on validation data, and uses held-out data once after search. The full adaptation flow and commands document this process.

Scope and Limitations

• The task covers sequence parity only; it does not test language modeling or general reasoning.
• Adaptation is a bounded heuristic over loop counts from 1–8, selected on validation data. It is not general Recursive Self-Improvement.
• The ±0.05 held-out accuracy threshold across N=3 seeds is an operational rule, not a statistical significance test. Reports include paired 95% t-intervals; with N=3 they are very wide (t=4.30), so pass more seeds (for example --seeds 1,2,3,4,5,6,7,8,9,10) before reading anything into a label.
• The NumPy and PyTorch backends use different architectures and parameter counts; see the backend comparison and its model test and PyTorch test.
• Every committed configuration is near chance (see What the committed results show), so the H2 "supported" verdict for looped-2 in the MPS run rests on differences around 0.47 and should not be read as evidence of better length generalization.
• The MPS run's in_distribution_accuracy (and so H2) used pooled train+dev accuracy; the current code and the CPU reference run use dev only. See the dated note at the end of the MPS run record.
• Looped-4 held-out accuracy is 0.5052 (flat, paired delta +0.0417) in the committed MPS run and 0.5208 (improvement, paired delta +0.0573) in the committed --device cpu run (results/torch-cpu-dev-reference). The two runs have the same seeds and settings but different devices. No repeated run on either device is committed, and bit-exact reproducibility on --device cpu is unverified: no test or artifact checks it.
• The hypotheses were specified before analysis, but are not described as preregistered because DESIGN.md and initial results were committed together (161752a).

Checkpoint .pt files are machine-specific and gitignored; only JSON manifests with SHA-256 hashes are tracked. More interpretation rules and artifact details are in the reference notes.

Documentation and Artifacts

Path	Contents
docs/reference.md	Relocated backend tables, operational rules, artifact notes, and citations
DESIGN.md	Architectural controls, research questions (DESIGN.md#research-questions), and hypotheses

Repository Map

Repository map

Path	Contents
rlt_rsi/train.py	Baseline and looped training, backends, evaluation
rlt_rsi/train_rsi.py	RSI-style iterative adaptation CLI and reporting
rlt_rsi/rsi.py	RSI adaptation loop (run_rsi_numpy, run_rsi_torch)
rlt_rsi/data.py	Example-disjoint split generation and manifests
rlt_rsi/model.py	NumPy model and configuration
rlt_rsi/diagnostics.py	Overfit diagnostic routines for optimization checks
tests/	Data, model, generalization, RSI, and torch tests
results/	Committed experiment artifacts
notebooks/rlt_rsi_colab.ipynb	CPU smoke and GPU walkthrough
DESIGN.md	Architectural controls and research questions

License

MIT — see LICENSE.