Skip to content

how are we going to manage remote training #97

Closed

Description

@cpmpercussion

it would be handy for users to submit make a dataset and train a model remotely (e.g., for classroom or collaboration use).

how can we do that?

Activity

  1. cpmpercussion commented on Sep 26, 2026

    @cpmpercussion
    OwnerAuthor

    Some notes on options for this.

    Training is small enough that it doesn't need a GPU. On my laptop (Apple Silicon CPU) the default s model trains at about 5 seconds per epoch on 10,000 events (roughly 10 minutes of knob data), so a full run with early stopping takes a few minutes. Data can already come from IMPSYpi, the AUv3 app and IMPSY Web, since they all write the same .log format. So "remote training" mostly means: somewhere with Python and TensorFlow that can take .log files and hand back a .tflite.

    Options, roughly from least to most effort:

    1. A Colab notebook. pip install impsy, upload .log files, run dataset and train, download the .tflite. Cheap to make and works for a class straight away, but everyone needs a Google account.
    2. Training in the web UI, run on a shared machine. The Flask web UI already handles logs, datasets and model upload/download; only training is missing (the workshop page says "You can't train new models from the web UI (yet!)"). With a train page, a teacher could run impsy webui on one decent machine and students could upload logs from any frontend and download models. It needs a background job (training shouldn't block a request) and some access control if it's not on a trusted network. It would also help people training locally.
    3. A GitHub template repo with Actions. Push .log files, and a workflow runs impsy dataset and impsy train and attaches the .tflite to a release. Free for public repos, CPU is enough, and it keeps a history of data and models for collaboration. The catch is that everyone needs GitHub and git, and logs in a public repo are public.
    4. A hosted service (e.g., a small upload form on Hugging Face Spaces). Easiest for students, but someone has to run it and look after other people's performance data. Probably not worth it yet.
    5. Training in the browser in IMPSY Web (TF.js). The models are small enough, but it would need a TF.js LSTM+MDN and a way to get a .tflite out, so it's a big job.

    I'd start with 2, since it closes the web UI gap for everyone and makes classroom use simple, and add 1 as a quick option for people without a machine to run IMPSY on. Things to decide for a shared setup: limits on model size and epochs, checking the dimension of uploaded logs, and how to keep each student's logs and models separate.

  2. cpmpercussion commented on Sep 26, 2026

    @cpmpercussion
    OwnerAuthor

    I agree starting with 2.

    For 1, If there was a notebook in this repo we could have a magic colab link to open it up directly - convenient!

    Part of 2 could be making sure our training command is super streamlined. So ideally user can just call "train" with a .npz files and it figures the rest out ending with a tflite model.

    Normal training checkpoints every epoch so if training is interrupted maybe we could convert the latest or best to tflite.

  3. cpmpercussion commented on Sep 26, 2026

    @cpmpercussion
    OwnerAuthor

    I'm not sure how 4 would work would have to look at hf spaces. 5 would be amazing but a task for the impsy-web repo and clearly work.

    3 I think is not an ideal solution. Someone familiar with GitHub to drive it would probably be able to do this on their own machine.

  4. cpmpercussion commented on Sep 26, 2026

    @cpmpercussion
    OwnerAuthor

    Option 2 (training in the web UI, plus a streamlined train command) landed in #111, and option 1 (Colab notebook) in #112. The rest is split out:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions