Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions documentation/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ v1.2.0 | December xx, 2026

New features
-------------
* Create automations of the types that installed plugins add from the *New automation* form on an asset's Automations page, not only from the CLI or the API [see `PR #2871 <https://www.github.com/FlexMeasures/flexmeasures/pull/2871>`_]

Infrastructure / Support
-------------------------
Expand Down
5 changes: 3 additions & 2 deletions documentation/features/automations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,9 @@ Plugin schemas validate configuration and parameters, and unknown fields are rej
Run a worker for the queue declared by the handler, in addition to the automation dispatcher.

The UI lists registered plugin types in their own tabs and shows their parameters, data source, input sensors, output sensors and recent jobs.
Existing actions, including editing recurrence, activation, deletion and *Run now*, also apply to these automations.
Creation forms for plugin types are a follow-up: use the CLI or API to create them for now.
Existing actions, including editing recurrence, activation, deletion, *Copy* and *Run now*, also apply to these automations.
The *New automation* form offers each installed plugin type under *Plugin types*.
Fill in its configuration and parameters as JSON, or pick an existing data source of that type; the plugin's generator is used, so the form asks for no data generator.

API-created automations remember the user who created them.
The dispatcher and worker recheck that the user is active and can write to the output sensors.
Expand Down
5 changes: 4 additions & 1 deletion flexmeasures/cli/tests/test_plugin_automations.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,12 @@ def test_ui_inspects_custom_and_unavailable_type(
assert "Measurements &lt;script&gt;" in response
assert '<script>alert("label")</script>' not in response
assert r"\u003cscript\u003e" in response
# The creation form offers the installed type, under the escaped name its tab uses.
assert '<option value="mock-ingestion">Measurements &lt;script&gt;' in response
else:
assert "mock-ingestion (plugin unavailable)" in response
assert "Create plugin-defined automation types with the CLI" in response
# A type whose plugin is gone keeps its tab, but cannot be created.
assert '<optgroup label="Plugin types">' not in response


def test_the_page_has_a_tab_for_a_type_it_can_come_to_list(
Expand Down
38 changes: 29 additions & 9 deletions flexmeasures/ui/templates/assets/asset_automations.html
Original file line number Diff line number Diff line change
Expand Up @@ -170,12 +170,19 @@ <h5 class="modal-title">New automation for {{ asset.name }}</h5>
<div class="mb-3">
<label for="automationType" class="form-label">Type
<i class="fa fa-info-circle ps-2" data-bs-toggle="tooltip" data-bs-placement="right"
title="Create plugin-defined automation types with the CLI. They can be viewed and managed here." style="font-size: 16px"></i>
title="What the automation computes. Besides forecasts, schedules and reports, installed plugins can add types of their own, listed under Plugin types." style="font-size: 16px"></i>
</label>
<select class="form-select" id="automationType">
<option value="forecasting" selected>forecasting</option>
<option value="scheduling">scheduling</option>
<option value="reporting">reporting</option>
{% if plugin_automation_types %}
<optgroup label="Plugin types">
{% for plugin_type in plugin_automation_types %}
<option value="{{ plugin_type.type_id }}">{{ plugin_type.display_name }}</option>
{% endfor %}
</optgroup>
{% endif %}
</select>
</div>
<div class="mb-3">
Expand Down Expand Up @@ -213,7 +220,8 @@ <h6 class="mb-1">Configuration</h6>
</div>
</div>
</div>
<div class="mb-3 chooses-generator">
<!-- A plugin type declares its own data generator, so only a forecast or report automation names one. -->
<div class="mb-3 chooses-generator names-generator-class">
<label for="automationGenerator" class="form-label">Data generator
<i class="fa fa-info-circle ps-2" data-bs-toggle="tooltip" data-bs-placement="right"
title="Forecaster class (defaults to TrainPredictPipeline) or reporter class (required for type reporting, e.g. PandasReporter)." style="font-size: 16px"></i>
Expand All @@ -231,10 +239,11 @@ <h6 class="mb-1">Configuration</h6>
<div class="mb-3">
<label for="automationParameters" class="form-label">Parameters (JSON)
<i class="fa fa-info-circle ps-2" data-bs-toggle="tooltip" data-bs-placement="right"
title="Forecast parameters (for type forecasting), a schedule trigger message (for type scheduling), or report parameters (for type reporting). A fixed start, end or prior is refused, as every run would share that moment. Leave the timing out to start a forecast or schedule at the run time, and to continue a report from the last successful one." style="font-size: 16px"></i>
title="Forecast parameters (for type forecasting), a schedule trigger message (for type scheduling), report parameters (for type reporting), or the parameters a plugin type defines. A fixed start, end or prior is refused, as every run would share that moment. Leave the timing out to start a forecast or schedule at the run time, and to continue a report from the last successful one." style="font-size: 16px"></i>
</label>
<textarea class="form-control" id="automationParameters" rows="4" placeholder='{"sensor": 2092}'></textarea>
<div class="form-text">
<!-- The run period options belong to the built-in types; a plugin type's parameters follow its own schema. -->
<div class="form-text builtin-type-only">
Say the period each run covers with two of "start-offset", "end-offset" and "duration",
such as <code>{"start-offset": "1D,DB", "duration": "P1D"}</code> for the whole of the next day.
</div>
Expand Down Expand Up @@ -391,11 +400,17 @@ <h5 class="modal-title">Edit automation</h5>

// A data source stores a data generator and the configuration it runs under, and the results computed under it are recorded on it,
// so picking one here is an alternative to setting the two fields below by hand.
// Only forecast and report automations can name one: a schedule automation works its source out from the asset and the flex config on every run.
// Forecast, report and plugin automations can name one: a schedule automation works its source out from the asset and the flex config on every run.
const SOURCE_TYPE_PER_AUTOMATION_TYPE = {
forecasting: "forecaster",
reporting: "reporter",
};
for (const pluginType of {{ plugin_automation_types | tojson }}) {
SOURCE_TYPE_PER_AUTOMATION_TYPE[pluginType.type_id] = pluginType.source_type;
}

// The types FlexMeasures itself defines; any other type comes from a plugin, which declares its own data generator.
const BUILTIN_AUTOMATION_TYPES = ["forecasting", "scheduling", "reporting"];

// How many search results the list shows at once, so that a broad search stays readable.
const MAX_SOURCE_RESULTS = 25;
Expand Down Expand Up @@ -485,8 +500,15 @@ <h5 class="modal-title">Edit automation</h5>
return $("#automationType").val() !== "scheduling";
}

// A plugin type declares its data generator, so only a forecast or a report automation names the generator class.
function typeNamesGeneratorClass() {
return typeChoosesGenerator() && BUILTIN_AUTOMATION_TYPES.includes($("#automationType").val());
}

function showGeneratorFields() {
$(".chooses-generator").toggle(typeChoosesGenerator());
$(".names-generator-class").toggle(typeNamesGeneratorClass());
$(".builtin-type-only").toggle(BUILTIN_AUTOMATION_TYPES.includes($("#automationType").val()));
}

/* Empty the creation form, so that the next automation starts from the defaults the form was rendered with.
Expand Down Expand Up @@ -797,9 +819,7 @@ <h6>Recently created jobs
order: [[createdAtColumn, "desc"]], // most recently created first
language: {
emptyTable: `No automations of type '${esc(automationTypes[automationType] || automationType)}' have been defined ${scope} yet.`
+ (userCanManage ? (["forecasting", "scheduling", "reporting"].includes(automationType)
? " Create one with the 'New automation' button, or with the CLI command `flexmeasures add automation`."
: " Create one with the CLI command `flexmeasures add automation`.") : ""),
+ (userCanManage ? " Create one with the 'New automation' button, or with the CLI command `flexmeasures add automation`." : ""),
},
columns: columns,
data: automations.map(AutomationRow),
Expand Down Expand Up @@ -1178,7 +1198,7 @@ <h6>Recently created jobs
$("#newAutomationErr").removeClass("d-none").text(error.message);
return;
}
const generator = choosesGenerator && !reusesSource ? $("#automationGenerator").val().trim() : "";
const generator = typeNamesGeneratorClass() && !reusesSource ? $("#automationGenerator").val().trim() : "";
const automationData = {
name: $("#automationName").val(),
type: $("#automationType").val(),
Expand Down
10 changes: 10 additions & 0 deletions flexmeasures/ui/tests/js/test_automation_actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,15 @@
Path(__file__).resolve().parents[2] / "templates/assets/asset_automations.html"
)

# The installed plugin types, as the view hands them to the page: one type, recorded under the generator base its plugin declares.
PLUGIN_AUTOMATION_TYPES = [
{
"type_id": "mock-ingestion",
"display_name": "Mock ingestion",
"source_type": "ingestor",
}
]


@pytest.fixture(scope="module", autouse=True)
def setup_ui_test_data():
Expand Down Expand Up @@ -42,6 +51,7 @@ def automation_script(
"scheduling": "Schedules",
"mock-ingestion": "Mock ingestion",
},
plugin_automation_types=PLUGIN_AUTOMATION_TYPES,
config={"FLEXMEASURES_JOB_TTL": timedelta(days=7)},
)
# An unrendered value would reach the browser as a syntax error, which reads as every check failing at once.
Expand Down
55 changes: 53 additions & 2 deletions flexmeasures/ui/tests/js/test_automation_copy.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,15 @@
import re

import pytest
from jinja2 import Environment, StrictUndefined

# The page's script is rendered by one helper, which asserts that it leaves no Jinja value behind.
# Sharing it keeps these checks from breaking whenever the page reads another value.
from test_automation_actions import TEMPLATE, automation_script
from test_automation_actions import (
PLUGIN_AUTOMATION_TYPES,
TEMPLATE,
automation_script,
)


@pytest.fixture(scope="module", autouse=True)
Expand All @@ -26,7 +31,11 @@ def creation_form_html() -> str:
r'(<form id="newAutomationForm">.*?</form>)', TEMPLATE.read_text(), re.DOTALL
)
assert match is not None
return match.group(1).replace("{{ asset.timezone }}", "Europe/Amsterdam")
environment = Environment(autoescape=True, undefined=StrictUndefined)
return environment.from_string(match.group(1)).render(
asset={"timezone": "Europe/Amsterdam"},
plugin_automation_types=PLUGIN_AUTOMATION_TYPES,
)


# A stand-in for the handful of jQuery calls the form helpers make, backed by the real DOM.
Expand Down Expand Up @@ -121,6 +130,11 @@ def test_a_copy_only_reuses_a_data_source_where_one_can_be_named(assert_js):
const sourceless = page.automationCopyValues(
{{id: 10, name: "Orphan", type: "forecasting", "cron": "0 0 * * *", timezone: "UTC"}},
{{parameters: {{}}}});
const pluginCopy = page.automationCopyValues(
{{id: 11, name: "Site import", type: "mock-ingestion", "cron": "*/15 * * * *", timezone: "UTC"}},
{{parameters: {{}}, source: source}});
eq("a plugin copy reuses the original's data source", pluginCopy.sourceId, 6);

eq("an automation whose details carry no source names none", sourceless.sourceId, null);
eq("and its parameters still default to an empty object", JSON.parse(sourceless.parameters), {{}});
""")
Expand Down Expand Up @@ -216,3 +230,40 @@ def test_copy_is_offered_to_managers_only(assert_js):
check("someone who may not manage automations is not offered Copy",
!viewerRow.actions.includes("automation-copy"), viewerRow.actions);
""")


def test_the_form_offers_installed_plugin_types_and_asks_them_for_no_generator_class(
assert_js,
):
"""A plugin type declares its own data generator, so the form takes its config and parameters but no generator class."""
assert_js(f"""
const holder = document.createElement("div");
holder.innerHTML = {json.dumps(creation_form_html())};
document.body.appendChild(holder);
{JQUERY_STUB}
const page = new Function({json.dumps(copy_script())}
+ "\\nreturn {{ prefillNewAutomationForm, resetNewAutomationForm, automationCopyValues, typeNamesGeneratorClass }};")();

const pluginOption = document.querySelector('#automationType optgroup[label="Plugin types"] option[value="mock-ingestion"]');
check("an installed plugin type is offered under Plugin types", pluginOption !== null, document.getElementById("automationType").innerHTML);
eq("and named as its tab names it", pluginOption.textContent.trim(), "Mock ingestion");

page.prefillNewAutomationForm(page.automationCopyValues(
{{id: 11, name: "Site import", type: "mock-ingestion", "cron": "*/15 * * * *", timezone: "UTC"}},
{{parameters: {{}}, source: {{id: 6}}}}));
const shown = selector => [...document.querySelectorAll(selector)].every(node => node.style.display !== "none");
const hidden = selector => [...document.querySelectorAll(selector)].every(node => node.style.display === "none");
eq("a plugin copy keeps its type", document.getElementById("automationType").value, "mock-ingestion");
check("a plugin type still takes a config and a data source",
shown("#automationConfig") && document.getElementById("automationConfig").closest(".chooses-generator").style.display !== "none"
&& document.getElementById("automationSourceSearch").closest(".chooses-generator").style.display !== "none",
"config or source picker hidden");
check("but asks for no data generator class", hidden(".names-generator-class"), "generator class field shown");
check("nor for the run period options of the built-in types", hidden(".builtin-type-only"), "run period hint shown");
eq("and sends none", page.typeNamesGeneratorClass(), false);

page.resetNewAutomationForm();
check("a forecast names its data generator class again", shown(".names-generator-class"), "generator class field still hidden");
check("and gets the run period options back", shown(".builtin-type-only"), "run period hint still hidden");
eq("and sends its class", page.typeNamesGeneratorClass(), true);
""")
3 changes: 3 additions & 0 deletions flexmeasures/ui/tests/js/test_automation_source_picker.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ def test_source_search_asks_for_the_sources_of_the_automations_own_type(assert_j

const scheduleUrl = new URL(page.sourceSearchUrl("scheduling", "x"), "http://x");
eq("a schedule automation narrows to no source type", scheduleUrl.searchParams.get("type"), null);

const pluginUrl = new URL(page.sourceSearchUrl("mock-ingestion", ""), "http://x");
eq("a plugin automation searches the sources its generator records under", pluginUrl.searchParams.get("type"), "ingestor");
""")


Expand Down
24 changes: 24 additions & 0 deletions flexmeasures/ui/tests/test_asset_crud.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,30 @@ def test_automations_page_manager_can_set_timezones(client, setup_assets, as_adm
assert b'timezone: $("#editAutomationTimezone").val()' in response.data


def test_automations_page_offers_installed_plugin_types_for_creation(
app, client, setup_assets, as_admin, monkeypatch
):
"""An installed plugin type can be created from the form, and its source picker searches the sources its generator records under."""
from flexmeasures.data.automations import register_automation_handler
from flexmeasures.data.tests.plugins.mock_ingestion import __automation_types__

asset = setup_assets["wind-asset-1"]
response = client.get(url_for("AssetCrudUI:automations", id=asset.id))
assert b'<optgroup label="Plugin types">' not in response.data

monkeypatch.setattr(app, "automation_handlers", dict(app.automation_handlers))
register_automation_handler(app, __automation_types__[0])
response = client.get(url_for("AssetCrudUI:automations", id=asset.id))

assert response.status_code == 200
assert b'<optgroup label="Plugin types">' in response.data
assert b'<option value="mock-ingestion">Mock ingestion</option>' in response.data
assert (
b'{"display_name": "Mock ingestion", "source_type": "ingestor", "type_id": "mock-ingestion"}'
in response.data
)


@pytest.mark.parametrize(
"args, error",
[
Expand Down
11 changes: 11 additions & 0 deletions flexmeasures/ui/views/assets/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,17 @@ def automations(self, id: str):
for automation in asset_it_can_list.automations
if automation.type not in registered_types
},
# Only an installed plugin type can be created, so a type whose plugin is gone gets a tab but no place in the creation form.
# Its data sources are recorded under the generator's base, which is what narrows the form's source picker to sources of this type.
plugin_automation_types=[
dict(
type_id=type_id,
display_name=handler.display_name,
source_type=handler.generator_class.__data_generator_base__,
)
for type_id, handler in registered_types.items()
if handler.generator_class is not None
],
# Managing an automation is gated like running one, so both follow create-children.
user_can_manage_automations=user_can_create_children(asset),
user_can_create_children=user_can_create_children(asset),
Expand Down
Loading