Feature/configurable solver

docs/impulse/docs/config/configuration.md

@@ -135,21 +135,23 @@ Two independent filter families:
 
 | Field                   | Type           | Default                  | Description                                                                                                                 |
 |-------------------------|----------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------|
-| `solver`                | `str`          | `"DefaultSolver"`        | `"DefaultSolver"` adapts to the silver layer: it selects channels from a narrow EAV `channel_tags` table when `source.channel_tags_table` is set and otherwise from columns on `channel_metrics`; it filters containers via a narrow EAV `container_tags` table or, when `source.container_tags_table` is omitted, a wide-only `container_metrics`. `"DeltaSolver"` and `"KeyValueStoreSolver"` are **deprecated aliases** that resolve to `DefaultSolver`. |
+| `solver`                | `str`          | `"DefaultSolver"`        | Registered name of the solver to use. `"DefaultSolver"` adapts to the silver layer: it selects channels from a narrow EAV `channel_tags` table when `source.channel_tags_table` is set and otherwise from columns on `channel_metrics`; it filters containers via a narrow EAV `container_tags` table or, when `source.container_tags_table` is omitted, a wide-only `container_metrics`. `"DeltaSolver"` and `"KeyValueStoreSolver"` are **deprecated aliases** that resolve to `DefaultSolver`. A custom solver is selected by the name it registered; see [Custom solver](../extensibility/custom_solver.md). |
 | `data_type`             | `str`          | `"RLE"`                  | `"RLE"` (intervals `[tstart, tend)`) or `"RAW"` (raw timestamps; converted to intervals at query time by the configured `raw_encoder`). |
 | `raw_encoder`           | `str`          | `null` (resolves to `"RLE"` when `data_type = "RAW"`) | How RAW point data is converted to intervals. `"RLE"` (the default) run-length encodes on the fly to reduce memory consumption: consecutive samples with the same value collapse into a single `[tstart, tend)` interval per run. `"INTERVAL"` keeps every original sample, only deriving `tend` from the next sample's timestamp and dropping exact duplicate points — use it when downstream analysis needs all original timestamps. Only takes effect with `data_type = "RAW"`; ignored for `"RLE"` input. See [How Impulse interprets intervals](../data_model/silver_layer_schema.md#raw-format) for validity semantics and what counts as a duplicate point. |
 | `drop_implausible_data` | `bool`         | `false`                  | When `true`, drops `channels` rows where `is_plausible = false`. Requires `data_type = "RAW"`; combining with `"RLE"` raises a validation error. |
 | `max_channels_per_batch` | `int`         | `500`                    | Caps how many distinct channels are read in one solve batch, to bound per-solve memory. Counts channel selections (each distinct `channel(...)` / `poi_channel(...)` in your expressions). Applies to events, aggregations, and calculated channels; for calculated channels it counts the *input* channels they read, not the number of calculated channels. The cap is a packing target, not a hard limit: an expression that on its own reads more channels than the cap still runs, in a batch of its own. Must be `>= 1`. The old name `batch_size` is a **deprecated alias** (still accepted, warns, removed in a future release); setting both raises a validation error. |
 | `max_containers_per_batch` | `int`       | `null`                   | Caps how many containers are solved at once, to bound per-solve memory. `null` disables the cap; when set it must be `>= 1`. On an **incremental** run, `Report.run()` commits at most this many upserted containers per iteration and loops until the population is drained (see [incremental](#incremental-optional)). On a **full** run, and when a changed definition recomputes over all containers, the population is split into chunks of about this size and solved chunk by chunk. Chunk sizes are approximate (the cap is a memory heuristic, not an exact bound). |
-| `solver_config`         | `SolverConfig` | `null`                   | Per-table column mappings, per-table equality filters, and project scoping. Set `project_id` to scope reads by project — it is applied to `container_tags` (if configured), `container_metrics`, and `channel_mapping` (if configured), so it works in both narrow EAV and wide-only data models. Omit it when you don't need project scoping. See [Solver column mappings and filters](#solver-column-mappings-and-filters). |
+| `solver_config`         | `SolverConfig` | `null`                   | Per-table column mappings, per-table equality filters, and project scoping. Set `project_id` to scope reads by project — it is applied to `container_tags` (if configured), `container_metrics`, and `channel_mapping` (if configured), so it works in both narrow EAV and wide-only data models. It is validated with the config class of the selected `solver`: with `DefaultSolver` you can omit it when you need none of these, while a [custom solver](../extensibility/custom_solver.md) can add its own settings here and make them required. See [Solver column mappings and filters](#solver-column-mappings-and-filters). |
 | `measurement_db`        | `str`          | `"MeasurementDB"`        | Registered name of the MeasurementDB implementation to use. The default is the built-in reader; a custom implementation can reshape a different source schema at read time. See [Custom measurement DB](../extensibility/custom_measurement_db.md). |
 | `measurement_db_config` | `dict`         | `null`                   | Keyword arguments for that implementation's config class, merged with the `source` tables (e.g. extra source tables, prefilter keys). |
 
 If `query_engine` is omitted, the default is `DefaultSolver` with
 `data_type = "RLE"`.
 
-Unknown keys under `query_engine` are rejected when the config is loaded: a misspelled key such as
-`"measurment_db"` fails with `Extra inputs are not permitted` instead of being silently ignored.
+Unknown keys at the top level of the config, under `query_engine` and directly under
+`query_engine.solver_config` are rejected when the config is loaded: a misspelled key such as
+`"measurment_db"`, or `"solver"` placed outside `query_engine`, fails with `Extra inputs are not
+permitted` instead of being silently ignored.
 
 For a source schema that needs reshaping beyond column renames, select your own read seam — see
 [Custom measurement DB](../extensibility/custom_measurement_db.md).

docs/impulse/docs/data_model/index.md

@@ -15,7 +15,8 @@ tables are **fully optional** add-ons, used only when configured — see the
 The rest of this page documents the full shape. Landing your data in it during
 ingest is the simplest path (see the [Ingestion guide](ingestion.md)); if you
 can't reshape, a [`SolverConfig`](ingestion.md#adapting-to-existing-data-layouts)
-can remap column names, or you can implement a custom solver.
+can remap column names, or a [custom MeasurementDB](../extensibility/custom_measurement_db.md)
+can reshape your tables when they are read.
 
 :::
 

docs/impulse/docs/data_model/ingestion.md

@@ -223,28 +223,30 @@ key names (`container_id`, `channel_id`) themselves are fixed
 constants. Their column *types* are not fixed, though — `container_id`
 in particular may be a `long`, `int`, or `string`; the engine adopts
 whatever type your tables use, as long as it is consistent across them.
-For different relationships or composite keys, see custom solvers below.
+For different relationships or composite keys, see the next section.
 
-### Custom solvers
+### Layouts `SolverConfig` can't express
 
 For physical layouts that do not match the silver-layer relationships at
 all — no EAV tag tables, alias lookup tables instead of `channel_tags`,
 computed-column joins, JSON-encoded values, multi-column composite keys
-that need pre-processing, etc. — you can implement a custom solver by
-subclassing
-[`QuerySolver`](../references/api/impulse_query_engine/analyze/query/solvers/query_solver.md)
-(or one of the existing solvers) and registering it in your report config.
-
-This is significantly more invested than the `SolverConfig` path: you take
-on responsibility for the four solver pipeline stages
-(`filter_container_tags`, `filter_container_metrics`, `filter_channel_tags`,
-`filter_channel_metrics`) and the `solve` method. Some advanced deployments
-do this — e.g. when the customer's silver layer pre-dates Impulse and
-synthesises Impulse-shaped views via SQL CTEs at query time. If you find
-yourself heading down this path, it is usually worth first asking whether
-a one-time ETL job to produce the standard silver-layer shape would be
-cheaper.
-
-The general rule: **`SolverConfig` for naming differences, custom solver
-for structural differences, ETL into the standard shape for everything
-else.**
+that need pre-processing, etc. — implement a
+[custom MeasurementDB](../extensibility/custom_measurement_db.md) that reshapes
+your tables into the silver model when they are read. Every reader (the solve,
+incremental detection, ad-hoc queries) then sees the same silver-shaped data,
+and the stock solver runs unchanged.
+
+A [custom solver](../extensibility/custom_solver.md) is for the other kind of
+difference: when the engine should do something different with silver-shaped
+data, such as custom candidate selection, extra pruning, or post-processing the
+solve output. Both are registered under a name and selected in the report
+config (`query_engine.measurement_db` / `query_engine.solver`).
+
+Either path is significantly more invested than `SolverConfig`: you take on
+responsibility for returning the shapes the engine expects. If you find
+yourself heading down this path, it is usually worth first asking whether a
+one-time ETL job to produce the standard silver-layer shape would be cheaper.
+
+The general rule: **`SolverConfig` for naming differences, a custom
+MeasurementDB for structural differences, a custom solver for engine
+behaviour, ETL into the standard shape for everything else.**

docs/impulse/docs/extensibility/custom_measurement_db.md

@@ -22,7 +22,7 @@ used and behaviour is unchanged.
 |---|---|
 | Column names, or narrow-EAV vs. wide layout | `DefaultSolver` + [`solver_config`](../config/configuration.md#solver-column-mappings-and-filters) — no code |
 | Different tables, unions, joins, grain changes, EAV unpivots, or a pre-reshape row filter | **A custom MeasurementDB** (this page) |
-| Query planning / execution the config cannot express | Implementation of a custom solver (see [Custom solvers](../data_model/ingestion.md#custom-solvers)) |
+| Query planning / execution the config cannot express | Implementation of a custom solver (see [Custom solver](custom_solver.md)) |
 
 ## Select it in the report config
 

docs/impulse/docs/extensibility/custom_solver.md

@@ -0,0 +1,117 @@
+---
+sidebar_position: 2
+title: Custom solver
+---
+
+# Custom solver
+
+The **solver** turns a query into results. It runs a fixed pipeline over the
+[silver model](../data_model/silver_layer_schema.md): filter containers by tags and metrics, find the
+candidate channels for each selection, pick the best candidate, and evaluate the TSAL expressions
+(`solve`). Impulse ships `DefaultSolver`, which adapts to narrow or wide silver layouts through
+[`solver_config`](../config/configuration.md#solver-column-mappings-and-filters). When you need the
+engine to *behave* differently, you can register your own solver and select it by name in the report
+config.
+
+With nothing configured, `DefaultSolver` is used and behaviour is unchanged.
+
+## Custom solver or custom MeasurementDB?
+
+Impulse has two extension points with different jobs. The **MeasurementDB** decides *where and how*
+the silver tables are read; the **solver** decides *what the engine does* with them.
+
+| Your problem | Use |
+|---|---|
+| Column names differ, or narrow-EAV vs. wide layout | `DefaultSolver` + `solver_config`, no code |
+| Your tables have a different shape: other tables, unions, joins, grain changes, EAV unpivots, or row filters that must run before a reshape | A [custom MeasurementDB](custom_measurement_db.md), or producing the silver shape upstream (ETL job or views) |
+| The engine should do something different with silver-shaped data: custom candidate selection, extra pruning, post-processing the solve output, a different time-series encoding or storage | **A custom solver** (this page) |
+
+Rule of thumb: if the problem is **how the data looks**, change how it's read; if it's **what the
+engine does with silver data**, change the solver.
+
+Prefer not to reshape data inside a solver. The solver isn't the only reader of the silver tables:
+incremental reporting detects new and updated containers by running only the solver's container
+stages (`filter_container_tags`, `filter_container_metrics`), outside `solve()`, and anything that
+queries the MeasurementDB directly doesn't go through the solver at all. A reshape in a solver has to
+be repeated in every stage that reads a table, and the other readers still see the original tables.
+Reshape in a [custom MeasurementDB](custom_measurement_db.md) instead: every reader goes through it,
+so they all see the same data.
+
+## Implement one
+
+A custom solver is a subclass of `DefaultSolver`, plus an optional config class for its own
+settings, registered under a name:
+
+```python
+import pyspark.sql.functions as F
+
+from impulse_query_engine.analyze.query.solvers import DefaultSolver, SolverConfig, register_solver
+
+
+class ExcludeContainersConfig(SolverConfig):
+    excluded_container_ids: list[int]  # required: no default
+
+
+@register_solver("ExcludeContainersSolver", ExcludeContainersConfig)
+class ExcludeContainersSolver(DefaultSolver):
+    def solve(self, query, channels_df, selections, dtypes):
+        solved = super().solve(query, channels_df, selections, dtypes)
+        return solved.filter(
+            ~F.col(self.config.container_id_col).isin(self.config.excluded_container_ids)
+        )
+```
+
+- **Override only the stages you want to change** and call `super()` for the rest. Subclass
+  `DefaultSolver`. Subclassing `QuerySolver` directly is possible but means implementing all
+  stages, including the alias stages (`filter_aliased_channel_metrics`,
+  `resolve_channel_selections`), and overriding `from_config` if you need RAW data.
+- **Put your settings on a `SolverConfig` subclass.** It keeps the per-table `column_name_mapping`,
+  `filters` and `project_id`, and `self.config` is an instance of your class.
+- **`register_solver(name, config_cls=SolverConfig, *, aliases=())`** registers the class under the
+  name and each alias. Re-registering a name replaces the previous entry.
+- **Impulse builds the solver with `from_config(ctx)`.** `DefaultSolver` subclasses inherit a version
+  that passes the Spark session, `solver_config` and the RAW-data settings (`data_type`,
+  `drop_implausible_data`, `raw_encoder`). Override `from_config` only if your constructor takes
+  different arguments.
+
+## Select it in the report config
+
+```json
+"query_engine": {
+  "solver": "ExcludeContainersSolver",
+  "solver_config": { "excluded_container_ids": [3] }
+}
+```
+
+- `solver` is the registered name. The default is `"DefaultSolver"`; `"DeltaSolver"` and
+  `"KeyValueStoreSolver"` are deprecated aliases for it.
+- `solver_config` is validated with the config class the selected solver registered.
+- In Python, the same keys go on `QueryEngine`:
+  `QueryEngine(solver="ExcludeContainersSolver", solver_config={"excluded_container_ids": [3]})`.
+
+For ad-hoc queries you don't need the registry: pass an instance,
+`query.solve(spark, solver=ExcludeContainersSolver(spark, config=ExcludeContainersConfig(excluded_container_ids=[3])))`.
+
+## Parse-time validation
+
+The config is checked when it is loaded, so these fail as a validation error rather than at run time:
+
+- an unregistered `solver` name (the message lists the registered names);
+- a missing required setting of your config class, or a setting with the wrong type, including when
+  `solver_config` is left out entirely;
+- an unknown key in `solver_config`: a misspelled setting, or your settings without `"solver"`
+  selecting your solver. `SolverConfig` forbids unknown keys and your config class inherits that, so
+  neither is silently dropped.
+
+## Rules
+
+- **Import the package that registers the solver before the report config is parsed**, including in
+  tools that only generate or edit configs. Selection is by name only: config data never triggers an
+  import.
+- **Keep each stage's output contract.** `filter_container_metrics` returns all container columns
+  needed downstream (container dimensions, event bounds), and `solve` returns one row per container:
+  `container_id` plus one column per selection.
+- **Use the internal column names from `self.config`** (e.g. `self.config.container_id_col`), not
+  string literals: column mapping has already been applied when a stage runs.
+- **Calculated channels need `solve_calculated_channels`**, which `DefaultSolver` implements. The
+  `QuerySolver` base raises, so subclass `DefaultSolver` if your reports use calculated channels.

docs/impulse/docs/references/api/impulse_query_engine/analyze/query/solvers/default_solver.md

@@ -49,6 +49,15 @@ silver layer.
 ``RawEncoder.INTERVAL`` only derives ``tend`` and drops exact
 duplicates.  Only consulted when ``is_raw_data`` is ``True``.
 
+#### from\_config
+
+```python
+def from_config(cls, ctx: SolverBuildContext) -> DefaultSolver
+```
+
+Build the solver with the Spark session and RAW-data flags from the report config.
+
+
 #### filter\_container\_tags
 
 ```python

docs/impulse/docs/references/api/impulse_query_engine/analyze/query/solvers/query_solver.md

@@ -3,6 +3,21 @@ sidebar_label: query_solver
 title: impulse_query_engine.analyze.query.solvers.query_solver
 ---
 
+## SolverBuildContext
+
+```python
+class SolverBuildContext()
+```
+
+Everything a report passes to a solver when it builds one (see ``QuerySolver.from_config``).
+
+The fields copy the ``query_engine`` settings a solver needs rather than passing the report's
+``QueryEngine`` config, because that model lives in ``impulse_reporting`` and the query engine
+must not depend on the reporting layer (``config_parser`` imports this package, so the reverse
+import would also be circular). When a solver needs another ``query_engine`` setting, add a
+field here and set it in ``Report.create_solver``.
+
+
 ## QuerySolver
 
 ```python
@@ -21,6 +36,18 @@ columns** needed for container dimensions and event bounds (e.g.
 not only ``container_id``.
 
 
+#### from\_config
+
+```python
+def from_config(cls, ctx: SolverBuildContext) -> QuerySolver
+```
+
+Build the solver from what a report provides (see ``register_solver``).
+
+The base implementation passes only ``ctx.solver_config``. Override it when the
+constructor needs more, e.g. the Spark session or the RAW-data flags.
+
+
 #### scoped\_container\_metrics
 
 ```python

docs/impulse/docs/references/api/impulse_query_engine/analyze/query/solvers/solver_registry.md

@@ -0,0 +1,36 @@
+---
+sidebar_label: solver_registry
+title: impulse_query_engine.analyze.query.solvers.solver_registry
+---
+
+Name registry for query solvers, so a report config can select a solver by name.
+
+A solver registers under a name with ``@register_solver(...)`` and is selected with
+``query_engine.solver`` in the report config. Selection is by name only: the package that
+registers a solver must be imported before the report config is parsed.
+
+
+#### register\_solver
+
+```python
+def register_solver(name: str,
+                    config_cls: type[SolverConfig] = SolverConfig,
+                    *,
+                    aliases: tuple[str, ...] = ())
+```
+
+Register a ``QuerySolver`` subclass under *name* (and *aliases*), configured by *config_cls*.
+
+``config_cls`` validates ``query_engine.solver_config`` when this solver is selected.
+Re-registering a name replaces the previous entry.
+
+
+#### resolve\_solver
+
+```python
+def resolve_solver(name: str) -> Registration[QuerySolver, SolverConfig]
+```
+
+Return the registration for *name*; raises ``KeyError`` listing the registered names.
+
+