Repository navigation
Train models from the web UI and streamline the train command - #111
Conversation
Adds a Train page to the web UI that trains a model from a dataset in a background thread, shows progress, can be stopped early, and offers the .keras/.tflite files for download. The Logs page now accepts log uploads so data from IMPSYpi, the AUv3 app or IMPSY Web can be trained on. `impsy train` now takes a .npz file as an argument and reads the dimension from it, and Ctrl-C (or a stopping callback) still saves the model with its latest weights. generate_dataset creates its destination folder so dataset generation works in a fresh workspace. Part of #97. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016oUxTsghWZWfnCnnkKcuNv
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved critical and moderate issues remain in the web training workflow.
Review effort: Lite
Findings: 1
Open (3)
What changed in this PR
Adds web-based model training, improved CLI dataset handling, interruption support, and related documentation/tests.
Changes:
- Added log uploads, dataset dimensions, background training, progress polling, stopping, and downloads.
- Added dataset autodetection, callback support, and interrupted-training saves.
- Updated dataset generation, documentation, and test coverage.
Open findings include one critical stop-state race, four moderate validation/performance issues, and one minor patience-validation issue.
| File | Reviewed changes |
|---|---|
tests/test_webui.py |
Web UI and background training tests |
tests/test_trained_model.py |
Training and CLI tests |
tests/test_data.py |
Dataset destination creation test |
README.md |
Updated training documentation |
impsy/web_interface.py |
Uploads, dataset metadata, validation, and training jobs |
impsy/train.py |
Dataset resolution and callback-aware training |
impsy/templates/train.html |
Training controls and progress display |
impsy/templates/models.html |
Training page navigation |
impsy/templates/logs.html |
Log upload interface |
impsy/templates/datasets.html |
Dataset dimensions and Train links |
impsy/templates/base.html |
Training navigation |
impsy/mdrnn.py |
Additional Keras callback support |
impsy/dataset.py |
Automatic destination directory creation |
docs/impsypi-workshop.md |
Updated workshop guidance |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| def stop(self): | ||
| if self.running: | ||
| self._stop_requested = True | ||
| self.status = "stopping" |
There was a problem hiding this comment.
Fixed in 204866b: stop() now takes the lock and only moves running → stopping, and the thread sets its final status/files/error under the same lock. Added a test that stopping a finished job leaves it finished.
| if not name.endswith("d-mdrnn.log"): | ||
| return None | ||
| dim = name[: -len("d-mdrnn.log")].rsplit("-", 1)[-1] | ||
| return int(dim) if dim.isdigit() else None |
There was a problem hiding this comment.
Fixed in 204866b: log_file_dimension now uses -(\d+)d-mdrnn\.log$, matching generate_dataset, with a regression case for 2d-mdrnn.log.
| def get_dataset_file_info(filepath): | ||
| """Get metadata for a dataset file, including the dimension of its data.""" | ||
| info = get_file_info(filepath) | ||
| try: | ||
| from impsy.train import dataset_dimension | ||
|
|
||
| info["dimension"] = dataset_dimension(filepath) | ||
| except Exception: | ||
| info["dimension"] = None |
There was a problem hiding this comment.
Fixed in 204866b: the dimension now comes from standard training-dataset-{N}d.npz names without loading the file, and only other names get loaded. Tested with a fake file that couldn't be loaded.
- Make stopping a training job and finishing it atomic so a late stop
can't leave a finished job stuck as "stopping".
- Require the "-" in -{dimension}d-mdrnn.log when checking uploads,
matching what generate_dataset accepts.
- Read dataset dimensions from standard training-dataset-{N}d.npz names
instead of decompressing every dataset on each page load.
- Give patience its own error message.
- Explain when no performance is long enough to train on, instead of
failing inside Keras.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016oUxTsghWZWfnCnnkKcuNv
Part of #97 (option 2 from the discussion).
Web UI
.tflite/.kerasdownload buttons appear when it's done..logfiles as uploads, so logs from IMPSYpi, the AUv3 app or IMPSY Web can be trained on. It rejects names that don't end in-{N}d-mdrnn.log, sincedatasetwould skip them.The web UI runs on the user's own device, e.g. a laptop running the Docker image or a Raspberry Pi. A Pi 4 or 5 should be able to train small models; the docs recommend against training on a Pi Zero 2 W.
impsy trainimpsy train datasets/foo.npzworks: the dimension is read from the dataset.-Dis optional and gives an error if it doesn't match the dataset. With no arguments, it uses the only.npzindatasets/..kerasand.tflitewith the latest weights.train_mdrnn()andPredictiveMusicMDRNN.train()accept extra Keras callbacks. The web UI uses this for progress and stopping.Fixes
generate_datasetnow creates its destination folder. Before this, generating a dataset in the web UI failed on a fresh workspace.train_mdrnn(save_model=False, save_tflite=True)no longer crashes with an undefinedmodel_keras_file.Docs
Testing
impsy webuion a fresh workspace: uploaded 6 logs, made a 4d dataset, trainedxsfor 30 epochs (~25 s), downloaded the.tflite, checked that a second job is refused, and stopped a large run, which saved a model after 4 epochs.Open questions
-ckpt.kerascheckpoint instead?🤖 Generated with Claude Code
https://claude.ai/code/session_016oUxTsghWZWfnCnnkKcuNv