Skip to content

Add a Colab notebook for training models from log files - #112

Merged
cpmpercussion merged 3 commits into
mainfrom
colab-notebook
Sep 26, 2026
Merged

cpmpercussion merged 3 commits into
mainfrom
colab-notebook

Conversation

@cpmpercussion

@cpmpercussion cpmpercussion commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Part of #97 (option 1 from the discussion).

Adds notebooks/impsy_training.ipynb, which can be opened straight from GitHub in Colab: open in Colab. That link will work once this is merged to main.

Steps

  1. %pip install impsy
  2. Upload .log files or a .zip of them.
  3. Count logs by dimension, taken from the -{N}d-mdrnn.log filename.
  4. Choose settings with Colab form fields: model size, max epochs, patience, and dimension (0 means the most common dimension).
  5. generate_dataset
  6. train_mdrnn with early stopping.
  7. Download the .tflite, with notes on using it in IMPSY/IMPSYpi, the AUv3 app and IMPSY Web.

The notebook only uses generate_dataset/train_mdrnn arguments that exist in the released 1.1.0, so it works with the PyPI version now and doesn't depend on #111. Outside Colab, it uses a local logs/ folder and prints the model path instead of downloading it.

The README training section and docs/get-started.md now link to the notebook.

Testing

  • Ran the notebook with jupyter nbconvert --execute in a clean Python 3.12 venv with pip install impsy (1.1.0) and 6 synthetic 4d logs. Every cell ran and it trained an s model to .tflite.
  • Not tested in Colab itself: the files.upload()/files.download() cells, the #@param form fields, and whether Colab asks for a session restart after the install.

Dependency fix

This also allows psutil 7 (>=6.0.0,<8, lock updated to 7.2.2). The <7 cap wasn't deliberate: it came from the original Poetry ^6.0.0 constraint, which the PEP 621 migration kept as <7. It clashed with current IPython (psutil>=7) when installing impsy in Colab. IMPSY only calls cpu_count() and virtual_memory(), which are unchanged in 7. The full test suite passes on 7.2.2.

🤖 Generated with Claude Code

https://claude.ai/code/session_016oUxTsghWZWfnCnnkKcuNv

notebooks/impsy_training.ipynb installs impsy from PyPI, takes uploaded
.log files (or a .zip), makes a dataset, trains a model and downloads
the .tflite. It uses the train_mdrnn/generate_dataset APIs from the
released 1.1.0 so it works before the next release, and runs outside
Colab too using a local logs folder. Linked from the README and the
get-started page.

Part of #97.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016oUxTsghWZWfnCnnkKcuNv
Copilot AI lite review requested due to automatic review settings September 26, 2026 08:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The notebook has a critical Colab dependency conflict and additional validation issues remain unresolved.

Review effort: Lite
Findings: 1 High severity

Open (1)
What changed in this PR

Adds a Google Colab workflow for training IMPSY models from log files and links it from the documentation.

Changes:

  • Adds upload, dataset generation, training, and .tflite download steps.
  • Adds configurable model and training settings.
  • Links the notebook from README.md and docs/get-started.md.
File Summary Findings
README.md Links to the Colab training notebook. None.
notebooks/​impsy_training.ipynb Implements the Colab training workflow. Critical psutil dependency conflict (4 votes); moderate short-dataset and invalid-dimension validation issues (1 vote each).
docs/​get-started.md Documents the Colab training option. Nit: clarify the “no cloud component” statement (1 vote).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

"metadata": {},
"outputs": [],
"source": [
"%pip install -q impsy"

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constraint and lock are fixed in d99cabc (psutil>=6.0.0,<8, locked at 7.2.2). The notebook installs from PyPI, so this reaches Colab with the next impsy release, which should go out soon after this merges.

cpmpercussion and others added 2 commits September 26, 2026 18:31
psutil was capped at <7 only because the original Poetry caret
constraint (^6.0.0) was carried over in the PEP 621 migration. IMPSY
only uses cpu_count() and virtual_memory(), which are unchanged in 7.
The cap conflicts with current IPython (psutil>=7), e.g., when
installing impsy in Colab.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016oUxTsghWZWfnCnnkKcuNv
The notebook now says which dimensions are available when DIMENSION has
no logs, and stops before training when no log is longer than the
training sequence, instead of failing inside Keras. The get-started page
no longer says training has no cloud component, now that it links to
Colab.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016oUxTsghWZWfnCnnkKcuNv
@cpmpercussion
cpmpercussion merged commit dcf0170 into main Sep 26, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants