Train models from the web UI and streamline the train command

README.md

@@ -133,12 +133,16 @@ Use the `dataset` command:
 
 This command collates all logs of dimension N+1 from the logs directory and saves the data in a compressed `.npz` file in the datasets directory. It will also print out some information about your dataset, in particular the total number of individual interactions. To have a useful dataset, it's good to start with more than 10,000 individual interactions but YMMV.
 
-To train the model, use the `train` command---this can take a while on a normal computer, so be prepared to let your computer sit and think for a few hours! You'll have to decide what _size_ model to try to train: `xxs`, `xs`, `s`, `m`, `l`, `xl`. The size refers to the number of LSTM units in each layer of your model and roughly corresponds to "learning capacity" at a cost of slower training and predictions.
+To train the model, use the `train` command. You don't need a GPU: an `s` model trained on a laptop CPU with around 10,000 interactions takes a few minutes. You'll have to decide what _size_ model to try to train: `xxs`, `xs`, `s`, `m`, `l`, `xl`. The size refers to the number of LSTM units in each layer of your model and roughly corresponds to "learning capacity" at a cost of slower training and predictions.
 It's a good idea to start with an `xs` or `s` model, and the larger models may work better for quite large datasets (e.g., >1M individual interactions).
 
-    poetry run ./start_impsy.py train --dimension (N+1) --modelsize s
+    poetry run ./start_impsy.py train datasets/training-dataset-(N+1)d.npz --modelsize s
 
-Early stopping is enabled by default and will stop training after the model stops improving for 10 epochs (configurable with `--patience`). You can disable it with `--no-earlystopping`.
+The model's dimension is read from the dataset. You can also leave out the dataset file and give `--dimension (N+1)` to use `datasets/training-dataset-(N+1)d.npz`.
+
+Early stopping is enabled by default and will stop training after the model stops improving for 10 epochs (configurable with `--patience`). You can disable it with `--no-earlystopping`. If you press `Ctrl-C` during training, IMPSY stops and saves the model as it is.
+
+You can also upload logs, make a dataset and train a model from the web UI (`impsy webui`, see the Train page) on whatever device is running IMPSY, e.g., a laptop running the Docker image or a Raspberry Pi. A Raspberry Pi 4 or 5 should be able to train small models, but we wouldn't recommend training on a Pi Zero 2 W.
 
 By default, your trained model will be saved in the `models` directory in `.keras` and `.tflite` format.
 

docs/impsypi-workshop.md

@@ -123,7 +123,7 @@ Finally, the SD card is configured to allow ethernet over USB so you can plug yo
 
 Connecting to the Raspberry Pi should be straightforward: if your computer knows about it over USB-ethernet, you can just type <http://impsypi.local:4000> into a web browser and see the web UI. The Pi gives itself the IP address 169.254.1.107 so you might try <http://169.254.1.107:4000> as well if the above doesn't work.
 
-Now you can poke around the web UI and see what you can do. Most importantly, you can update the configuration file directly, download log files and generate or download datasets. You can't train new models from the web UI (yet!) and it's possibly a bad idea to train models on the slower Raspberry Pis anyway.
+Now you can poke around the web UI and see what you can do. Most importantly, you can update the configuration file directly, upload or download log files and generate or download datasets. The web UI can also train models on the Train page. A Raspberry Pi 4 or 5 should be able to train a small model (try `xs` or `s`), but we wouldn't recommend training on a Pi Zero 2 W; instead, train on your laptop (e.g., with the Docker image) and upload the `.tflite` model to your Pi.
 
 > N.B.: I'm new to this whole ethernet over USB and SD card building business. I'm probably doing something wrong so if there are issues here please be patient and if you know a better way to set something up, let me know!
 
@@ -308,7 +308,7 @@ docker run --rm \
   --volume ./datasets:/impsy/datasets \
   --volume ./models:/impsy/models \
   charlepm/impsy \
-  poetry run ./start_impsy.py train -D 4 -S datasets/training-dataset-4d.npz
+  poetry run ./start_impsy.py train datasets/training-dataset-4d.npz
 ```
 
 to train a 4D model. This command makes sure that the model files that are saved by the train command get saved out of the docker container and in your regular computer filesystem.

impsy/dataset.py

@@ -70,6 +70,7 @@ def generate_dataset(
 
     # Save Performance Data in a compressed numpy file.
     dataset_name = filename if filename else f"training-dataset-{dimension}d.npz"
+    Path(destination).mkdir(parents=True, exist_ok=True)
     dataset_file = Path(destination) / dataset_name
 
     # Input format is:

impsy/mdrnn.py

@@ -227,8 +227,11 @@ def train(
         save_location="models",
         validation_split=0.1,
         patience=10,
+        callbacks=None,
     ):
-        """Train the network for a number of epochs with a specific dataset."""
+        """Train the network for a number of epochs with a specific dataset.
+
+        Extra Keras callbacks can be given in callbacks."""
         save_location = Path(save_location)
         checkpoint_path = save_location / f"{self.model_name}-ckpt.keras"
         checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(
@@ -242,11 +245,13 @@ def train(
         early_stopping_callback = tf.keras.callbacks.EarlyStopping(
             monitor="val_loss", mode="min", verbose=1, patience=patience
         )
+        extra_callbacks = list(callbacks or [])
         callbacks = [terminateOnNaN]
         if checkpointing:
             callbacks.append(checkpoint_callback)
         if early_stopping:
             callbacks.append(early_stopping_callback)
+        callbacks += extra_callbacks
 
         # Do the data scaling in here.
         X = np.array(X, dtype=np.float32) * SCALE_FACTOR

impsy/templates/base.html

@@ -34,6 +34,9 @@
                             Datasets {% if workflow.datasets %}<span class="badge bg-success rounded-pill">&#10003;</span>{% endif %}
                         </a>
                     </li>
+                    <li class="nav-item">
+                        <a class="nav-link {% if active_page == 'train' %}active{% endif %}" href="{{ url_for('train') }}">Train</a>
+                    </li>
                     <li class="nav-item">
                         <a class="nav-link {% if active_page == 'models' %}active{% endif %}" href="{{ url_for('models') }}">
                             Models {% if workflow.models %}<span class="badge bg-success rounded-pill">&#10003;</span>{% endif %}

impsy/templates/datasets.html

@@ -31,6 +31,7 @@ <h5 class="card-title">Dataset Files</h5>
                     <th>File</th>
                     <th>Size</th>
                     <th>Date</th>
+                    <th>Dim</th>
                     <th class="text-end">Actions</th>
                 </tr>
             </thead>
@@ -43,7 +44,9 @@ <h5 class="card-title">Dataset Files</h5>
                     </td>
                     <td class="text-nowrap">{{ file.size }}</td>
                     <td class="text-nowrap">{{ file.date }}</td>
+                    <td>{% if file.dimension %}{{ file.dimension }}d{% else %}?{% endif %}</td>
                     <td class="text-end text-nowrap">
+                        <a href="{{ url_for('train', dataset=file.name) }}" class="btn btn-outline-primary btn-sm">Train</a>
                         <a href="{{ url_for('download_dataset', filename=file.name) }}" class="btn btn-outline-secondary btn-sm">Download</a>
                         <form method="POST" action="{{ url_for('delete_dataset', filename=file.name) }}" class="d-inline" onsubmit="return confirm('Delete {{ file.name }}?')">
                             <button type="submit" class="btn btn-outline-danger btn-sm">Delete</button>

impsy/templates/logs.html

@@ -2,7 +2,23 @@
 {% block title %}IMPSY - Log Files{% endblock %}
 {% block content %}
 <h1 class="mb-2">Log Files</h1>
-<p class="text-secondary mb-4">Interaction logs recorded during performances. Run <code>impsy run</code> to record new data.</p>
+<p class="text-secondary mb-4">Interaction logs recorded during performances. Run <code>impsy run</code> to record new data, or upload logs from another IMPSY device.</p>
+
+<div class="card mb-3">
+    <div class="card-body">
+        <h5 class="card-title">Upload Logs</h5>
+        <form method="POST" enctype="multipart/form-data" class="row g-2 align-items-end">
+            <div class="col-auto">
+                <label for="file" class="form-label">Log files (.log)</label>
+                <input type="file" class="form-control" name="file" id="file" accept=".log" multiple>
+            </div>
+            <div class="col-auto">
+                <button type="submit" class="btn btn-primary">Upload</button>
+            </div>
+        </form>
+        <div class="form-text mt-2">Log files from IMPSY, IMPSYpi, the AUv3 app and IMPSY Web all work. Names need to end in <code>-{dimension}d-mdrnn.log</code>.</div>
+    </div>
+</div>
 
 <div class="card">
     <div class="card-body">

impsy/templates/models.html

@@ -2,7 +2,7 @@
 {% block title %}IMPSY - Models{% endblock %}
 {% block content %}
 <h1 class="mb-2">Models</h1>
-<p class="text-secondary mb-4">Upload trained models or train new ones with <code>impsy train</code>.</p>
+<p class="text-secondary mb-4">Upload trained models or train new ones on the <a href="{{ url_for('train') }}">Train</a> page or with <code>impsy train</code>.</p>
 
 <div class="card mb-3">
     <div class="card-body">

impsy/templates/train.html

@@ -0,0 +1,126 @@
+{% extends "base.html" %}
+{% block title %}IMPSY - Train{% endblock %}
+{% block content %}
+<h1 class="mb-2">Train</h1>
+<p class="text-secondary mb-4">Train a model from a dataset. Training runs in the background and saves <code>.keras</code> and <code>.tflite</code> files to the models folder. A small model on a laptop CPU takes a few minutes.</p>
+
+<div class="card mb-3">
+    <div class="card-body">
+        <h5 class="card-title">Train a Model</h5>
+        {% if dataset_files %}
+        <form method="POST" action="{{ url_for('train') }}" class="row g-2 align-items-end">
+            <div class="col-12 col-md-auto">
+                <label for="dataset" class="form-label">Dataset</label>
+                <select class="form-select" id="dataset" name="dataset" required>
+                    {% for file in dataset_files %}
+                    <option value="{{ file.name }}" {% if file.name == selected_dataset %}selected{% endif %}>{{ file.name }}{% if file.dimension %} ({{ file.dimension }}d){% endif %}</option>
+                    {% endfor %}
+                </select>
+            </div>
+            <div class="col-auto">
+                <label for="model_size" class="form-label">Model size</label>
+                <select class="form-select" id="model_size" name="model_size">
+                    {% for size in sizes %}
+                    <option value="{{ size }}" {% if size == default_size %}selected{% endif %}>{{ size }}</option>
+                    {% endfor %}
+                </select>
+            </div>
+            <div class="col-auto">
+                <label for="max_epochs" class="form-label">Max epochs</label>
+                <input type="number" class="form-control" id="max_epochs" name="max_epochs" min="1" max="{{ max_epochs }}" value="100" style="width: 110px">
+            </div>
+            <div class="col-auto">
+                <label for="patience" class="form-label">Patience</label>
+                <input type="number" class="form-control" id="patience" name="patience" min="1" value="10" style="width: 100px">
+            </div>
+            <div class="col-auto">
+                <button type="submit" class="btn btn-primary" {% if job.status in ['running', 'stopping'] %}disabled{% endif %}>Start Training</button>
+            </div>
+        </form>
+        <div class="form-text mt-2">
+            Training stops early when the validation loss hasn't improved for <em>patience</em> epochs.
+            {% if config_dimension %}Your config uses dimension {{ config_dimension }}; the model's dimension comes from the dataset.{% endif %}
+        </div>
+        {% else %}
+        <p class="text-secondary mb-0">No datasets yet. <a href="{{ url_for('datasets') }}">Generate a dataset</a> from your log files first.</p>
+        {% endif %}
+    </div>
+</div>
+
+<div class="card" id="job-card" {% if job.status == 'idle' %}hidden{% endif %}>
+    <div class="card-body">
+        <div class="d-flex justify-content-between align-items-start">
+            <h5 class="card-title">Training Status</h5>
+            <form method="POST" action="{{ url_for('train_stop') }}" id="stop-form" {% if job.status != 'running' %}hidden{% endif %}>
+                <button type="submit" class="btn btn-outline-danger btn-sm">Stop and Save</button>
+            </form>
+        </div>
+        <dl class="row mb-0">
+            <dt class="col-sm-3">Status</dt><dd class="col-sm-9" id="job-status">{{ job.status }}</dd>
+            <dt class="col-sm-3">Dataset</dt><dd class="col-sm-9"><code id="job-dataset">{{ job.dataset }}</code></dd>
+            <dt class="col-sm-3">Model size</dt><dd class="col-sm-9" id="job-size">{{ job.model_size }}</dd>
+            <dt class="col-sm-3">Epoch</dt><dd class="col-sm-9" id="job-epoch">{{ job.epoch }} / {{ job.max_epochs }}</dd>
+            <dt class="col-sm-3">Loss (val)</dt><dd class="col-sm-9" id="job-loss">&ndash;</dd>
+            <dt class="col-sm-3">Time</dt><dd class="col-sm-9" id="job-elapsed">{{ job.elapsed }} s</dd>
+        </dl>
+        <div class="alert alert-danger mt-3 mb-0" id="job-error" {% if not job.error %}hidden{% endif %}>{{ job.error }}</div>
+        <div class="mt-3" id="job-files" {% if not job.model_files %}hidden{% endif %}>
+            {% for name in job.model_files %}
+            <a href="{{ url_for('download_model', filename=name) }}" class="btn btn-outline-secondary btn-sm me-1">Download {{ name }}</a>
+            {% endfor %}
+        </div>
+    </div>
+</div>
+{% endblock %}
+
+{% block scripts %}
+<script>
+(function () {
+    const statusUrl = "{{ url_for('train_status') }}";
+    const downloadUrl = "{{ url_for('download_model', filename='FILENAME') }}";
+    const fmt = (v) => (v === null || v === undefined) ? "–" : v.toFixed(3);
+
+    function render(job) {
+        document.getElementById("job-card").hidden = job.status === "idle";
+        document.getElementById("job-status").textContent = job.status;
+        document.getElementById("job-dataset").textContent = job.dataset || "";
+        document.getElementById("job-size").textContent = job.model_size || "";
+        document.getElementById("job-epoch").textContent = job.epoch + " / " + job.max_epochs;
+        const n = job.loss.length;
+        document.getElementById("job-loss").textContent =
+            n ? fmt(job.loss[n - 1]) + " (" + fmt(job.val_loss[n - 1]) + ")" : "–";
+        document.getElementById("job-elapsed").textContent = job.elapsed + " s";
+        document.getElementById("stop-form").hidden = job.status !== "running";
+        const err = document.getElementById("job-error");
+        err.hidden = !job.error;
+        err.textContent = job.error || "";
+        const files = document.getElementById("job-files");
+        files.hidden = !job.model_files.length;
+        files.replaceChildren(...job.model_files.map((name) => {
+            const a = document.createElement("a");
+            a.href = downloadUrl.replace("FILENAME", encodeURIComponent(name));
+            a.className = "btn btn-outline-secondary btn-sm me-1";
+            a.textContent = "Download " + name;
+            return a;
+        }));
+    }
+
+    let wasRunning = {{ 'true' if job.status in ['running', 'stopping'] else 'false' }};
+    async function poll() {
+        try {
+            const job = await (await fetch(statusUrl)).json();
+            render(job);
+            const running = job.status === "running" || job.status === "stopping";
+            if (wasRunning && !running) {
+                // Re-enable the form now training has finished.
+                location.reload();
+                return;
+            }
+            wasRunning = running;
+        } catch (e) { /* try again next time */ }
+        setTimeout(poll, 2000);
+    }
+    poll();
+})();
+</script>
+{% endblock %}