Skip to content

AgentiGrid: keep "};" on MATPOWER cell arrays so ExaGO reads genfuel #137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ahajdar wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

AgentiGrid: keep "};" on MATPOWER cell arrays so ExaGO reads genfuel #137

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
368cd6a
AgentiGrid: keep "};" on MATPOWER cell arrays so ExaGO reads genfuel
ahajdar Oct 3, 2026
67f7d6f
AgentiGrid tests: OPFLOW fixture with real fuel types; live test no l…
ahajdar Oct 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions agentigrid/agentigrid/parsers/matpower_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,11 @@ def parse_matpower(path: Path) -> MATNetwork:

if bracket == "{":
# Cell array (bus_name, gentype, genfuel, etc.) → store as raw
raw_text = comment + "\n" + text[match.start():block_start] + block_content + ("}" if bracket == "{" else "]")
# Keep the MATLAB terminator "};" -- ExaGO's reader finds the end of
# mpc.genfuel by searching for "};", so a bare "}" makes it skip the
# fuel types (and with them the default ramp rates and the
# renewable Pmin=0 relaxation).
raw_text = comment + "\n" + text[match.start():block_start] + block_content + "};"
extra_sections[section_name] = raw_text
continue

Expand Down Expand Up @@ -263,7 +267,7 @@ def parse_matpower(path: Path) -> MATNetwork:
logger.debug("Parsed %d gencost entries", len(gencost))
else:
# Unknown numeric section → store as raw
raw_text = comment + "\n" + text[match.start():block_start] + block_content + "]"
raw_text = comment + "\n" + text[match.start():block_start] + block_content + "];"
extra_sections[section_name] = raw_text
logger.debug("Stored unknown section mpc.%s as raw text", section_name)

Expand Down
15 changes: 14 additions & 1 deletion agentigrid/agentigrid/parsers/matpower_writer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,19 @@ def _write_gencost_row(gc: GenCost) -> str:
return "\t" + "\t".join(parts) + ";"


def _terminate(raw: str) -> str:
"""Ensure a raw section ends with "};" / "];".

ExaGO locates the end of ``mpc.genfuel`` by the literal "};". Networks
parsed by older versions stored the block without the semicolon; writing
that back made ExaGO ignore the fuel types, leaving every ramp rate at 0
(SCOPFLOW cannot redispatch in contingencies) and wind/solar fixed at Pmax.
"""
if raw.endswith("}") or raw.endswith("]"):
return raw + ";"
return raw


def write_matpower(network: MATNetwork, path: Path) -> None:
"""Write a MATNetwork object to a MATPOWER .m file.

Expand Down Expand Up @@ -137,7 +150,7 @@ def write_matpower(network: MATNetwork, path: Path) -> None:
# --- Extra sections (gentype, genfuel, bus_name, etc.) ---
for section_name, raw_text in network.extra_sections.items():
lines.append("")
lines.append(raw_text.rstrip())
lines.append(_terminate(raw_text.rstrip()))

lines.append("")

Expand Down
24 changes: 24 additions & 0 deletions agentigrid/claude_code_changes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,30 @@ This document records significant changes made by Claude Code, grouped by the pr

---

## MATPOWER cell arrays keep "};" — ExaGO reads genfuel again (2026-10-03)

**Symptom.** SCOPFLOW reported local infeasibility on AgentiGrid-rewritten ACTIVSg200 at base load, while the original file converged. The rewritten file had `mpc.genfuel = { ... }` with a bare `}`.

**Cause.** `src/ps/psreaddata.cpp` ends `mpc.genfuel` at the first line containing `};`. The parser stored cell arrays (`gentype`, `genfuel`, `bus_name`) without the trailing `;` and the writer wrote them back as `}`, so for every case AgentiGrid rewrote, ExaGO ignored the fuel types: all units took the default fuel, ramp rates were 0, and wind/solar stayed at Pmin = Pmax.

**Evidence in the repository.** `tests/fixtures/sample_opflow_output.txt` (OPFLOW on AgentiGrid-rewritten ACTIVSg200) listed all 49 generators as COAL; `case_ACTIVSg200.m` has 25 coal, 17 ng, 6 wind and 1 nuclear (bus 189). `test_gen_189` asserted COAL, so it encoded the bug.

### Changes

| File | Change |
|---|---|
| `agentigrid/parsers/matpower_parser.py` | Keep `};` / `];` on stored cell-array sections |
| `agentigrid/parsers/matpower_writer.py` | Terminate cell arrays with `};`, including raw sections from networks parsed before the fix |
| `tests/grid_files/test_matpower_cell_terminators.py` | New (3 tests): mimics ExaGO's genfuel detection on a round-tripped ACTIVSg200; all fail without the fix |
| `tests/fixtures/sample_opflow_output.txt` | Regenerated with the fixed writer (Ipopt 3.14.14): real fuel mix |
| `tests/results/test_opflow_parser.py` | Expected values for the new fixture (21 iterations, 0.021 s, bus 189 NUCLEAR); new `test_fuel_mix_matches_case_file` |
| `tests/runner/test_executor.py` | `TestExecutorLive` skips without the `opflow` binary (like `TestMPIGuard`) and no longer overwrites the tracked fixture on every run; regenerate with `AGENTIGRID_REGENERATE_FIXTURES=1` |
| `tests/README.md` | How to regenerate the fixture |

Fast suite (`python -m pytest -m "not slow"`), without ExaGO binaries: **1083 passed, 13 skipped**. Before this change the same run had 2 failures: `test_solve_time` (the committed fixture no longer matched its expected value) and `test_opflow_run` (no skip without the binary).

---

## C.2/C.3 corrections — metric gating, summary aggregation, predicate audit, sweep dedup (2026-06-23)

Four contained fixes from the prompt-13/15/17 runs. No capability behavior changed (capacity numbers, dispatch results, metric/predicate definitions, and the C.1 binding identifier are untouched) — only how results are gated, summarized, audited, and deduplicated.
Expand Down
6 changes: 6 additions & 0 deletions agentigrid/tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ Tests are grouped by the part of AgentiGrid they cover:
| `e2e/` | End-to-end runs through the agent loop |
| `fixtures/` | Shared sample files (e.g. `sample_opflow_output.txt`) |

`fixtures/sample_opflow_output.txt` is OPFLOW output for `case_ACTIVSg200.m` as
rewritten by AgentiGrid; `results/test_opflow_parser.py` checks values from it.
To regenerate it (needs the `opflow` binary), run the live executor test with
`AGENTIGRID_REGENERATE_FIXTURES=1 python -m pytest tests/runner/test_executor.py -k test_opflow_run`
and update the expected values in `results/test_opflow_parser.py`.

Run the fast tests from the AgentiGrid root:

```bash
Expand Down
134 changes: 66 additions & 68 deletions agentigrid/tests/fixtures/sample_opflow_output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ This program contains Ipopt, a library for large-scale nonlinear optimization.
For more information visit https://github.com/coin-or/Ipopt
******************************************************************************

This is Ipopt version 3.14.20, running with linear solver ma27.
This is Ipopt version 3.14.14, running with linear solver ma27.

Number of nonzeros in equality constraint Jacobian...: 2826
Number of nonzeros in equality constraint Jacobian...: 2832
Number of nonzeros in inequality constraint Jacobian.: 1958
Number of nonzeros in Lagrangian Hessian.............: 1956
Number of nonzeros in Lagrangian Hessian.............: 1992

Total number of variables............................: 469
Total number of variables............................: 475
variables with only lower bounds: 0
variables with lower and upper bounds: 270
variables with lower and upper bounds: 276
variables with only upper bounds: 0
Total number of equality constraints.................: 400
Total number of inequality constraints...............: 490
Expand All @@ -22,52 +22,50 @@ Total number of inequality constraints...............: 490
inequality constraints with only upper bounds: 490

iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 4.0417248e+04 3.70e+00 4.65e+01 -1.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 3.9034986e+04 3.40e+00 4.25e+01 -1.0 2.83e+00 - 1.39e-01 8.08e-02f 1
2 3.6673124e+04 2.88e+00 3.56e+01 -1.0 7.20e+00 - 4.19e-01 1.48e-01f 1
3 3.1578046e+04 1.80e+00 1.23e+02 -1.0 2.11e+01 - 8.80e-01 3.49e-01f 1
4 3.0599807e+04 1.52e+00 1.10e+02 -1.0 2.12e+01 - 4.46e-01 1.53e-01f 1
5 2.9699291e+04 1.26e+00 9.78e+01 -1.0 1.84e+01 - 4.37e-01 1.71e-01f 1
6 2.9309885e+04 1.15e+00 9.04e+01 -1.0 1.59e+01 - 6.85e-01 8.97e-02f 1
7 2.8645960e+04 8.46e-01 7.05e+01 -1.0 3.67e+00 - 2.11e-03 2.64e-01f 1
8 2.7638522e+04 2.53e-02 7.27e+01 -1.0 3.56e+00 - 9.30e-01 1.00e+00f 1
9 2.7640850e+04 8.98e-04 7.46e-01 -1.0 7.87e-01 - 1.00e+00 1.00e+00h 1
0 4.0417248e+04 3.70e+00 6.61e+01 -1.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 3.8873587e+04 3.57e+00 6.36e+01 -1.0 4.46e+00 - 4.92e-02 3.53e-02f 1
2 3.6809680e+04 3.38e+00 5.88e+01 -1.0 4.50e+00 - 1.55e-01 5.19e-02f 1
3 3.4170482e+04 3.08e+00 4.94e+01 -1.0 3.95e+00 - 2.60e-01 8.73e-02f 1
4 3.2530838e+04 2.69e+00 2.99e+01 -1.0 9.39e+00 - 3.74e-01 1.23e-01f 1
5 2.9302664e+04 1.78e+00 8.68e+01 -1.0 2.07e+01 - 9.47e-01 3.17e-01f 1
6 2.8914676e+04 1.56e+00 7.90e+01 -1.0 1.48e+01 - 3.97e-01 1.26e-01f 1
7 2.8108655e+04 6.74e-01 5.46e+01 -1.0 5.25e+00 - 9.65e-01 5.69e-01f 1
8 2.7648421e+04 1.63e-02 2.21e+01 -1.0 1.66e+00 - 8.62e-01 1.00e+00f 1
9 2.7654216e+04 1.36e-03 1.97e-01 -1.0 3.99e-01 - 1.00e+00 1.00e+00h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
10 2.7585786e+04 1.54e-04 9.33e-02 -1.7 3.31e-01 - 1.00e+00 1.00e+00f 1
11 2.7569878e+04 1.44e-03 5.82e-02 -2.5 6.72e-02 - 9.88e-01 1.00e+00f 1
12 2.7563905e+04 1.32e-02 2.35e-02 -2.5 1.04e-01 - 1.00e+00 1.00e+00h 1
13 2.7564551e+04 3.76e-05 1.44e-04 -2.5 2.12e-02 - 1.00e+00 1.00e+00h 1
14 2.7558777e+04 6.47e-03 6.79e-02 -3.8 7.82e-02 - 8.63e-01 1.00e+00f 1
15 2.7557880e+04 4.68e-03 1.05e-01 -3.8 7.97e-02 - 1.00e+00 3.39e-01h 1
16 2.7557754e+04 5.40e-04 4.26e-01 -3.8 1.03e-01 - 1.00e+00 8.88e-01h 1
17 2.7557736e+04 9.85e-06 2.13e-04 -3.8 2.55e-02 - 1.00e+00 1.00e+00h 1
18 2.7557602e+04 1.37e-05 1.54e-01 -5.7 3.45e-02 - 9.15e-01 7.81e-01h 1
19 2.7557572e+04 3.77e-06 1.09e-02 -5.7 9.25e-03 - 1.00e+00 9.65e-01h 1
10 2.7574726e+04 1.60e-04 2.22e+00 -2.5 2.75e-01 - 8.77e-01 1.00e+00f 1
11 2.7569519e+04 2.59e-03 1.09e-02 -2.5 7.33e-02 - 1.00e+00 1.00e+00f 1
12 2.7560217e+04 1.93e-02 1.30e-01 -3.8 1.26e-01 - 7.37e-01 1.00e+00h 1
13 2.7557763e+04 1.06e-02 2.11e-02 -3.8 1.04e-01 - 9.63e-01 6.25e-01h 1
14 2.7557837e+04 1.74e-03 6.51e-01 -3.8 9.82e-02 - 1.00e+00 8.38e-01h 1
15 2.7557760e+04 1.86e-05 4.99e-04 -3.8 2.51e-02 - 1.00e+00 1.00e+00h 1
16 2.7557607e+04 1.53e-05 1.57e-01 -5.7 3.40e-02 - 9.19e-01 7.80e-01h 1
17 2.7557573e+04 3.73e-06 1.16e-02 -5.7 9.12e-03 - 1.00e+00 9.63e-01h 1
18 2.7557573e+04 1.95e-06 9.06e-03 -5.7 1.36e-03 - 1.00e+00 5.00e-01h 2
19 2.7557573e+04 6.41e-07 2.58e-06 -5.7 1.07e-03 - 1.00e+00 1.00e+00h 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
20 2.7557572e+04 1.94e-06 8.78e-03 -5.7 1.36e-03 - 1.00e+00 5.00e-01h 2
21 2.7557572e+04 6.42e-07 2.58e-06 -5.7 1.07e-03 - 1.00e+00 1.00e+00h 1
22 2.7557571e+04 2.07e-07 1.75e-04 -7.0 6.18e-04 - 9.89e-01 1.00e+00h 1
23 2.7557571e+04 5.19e-08 2.16e-07 -7.0 3.05e-04 - 1.00e+00 1.00e+00h 1
20 2.7557571e+04 2.07e-07 1.73e-04 -7.0 6.18e-04 - 9.89e-01 1.00e+00h 1
21 2.7557571e+04 5.19e-08 2.16e-07 -7.0 3.05e-04 - 1.00e+00 1.00e+00h 1

Number of Iterations....: 23
Number of Iterations....: 21

(scaled) (unscaled)
Objective...............: 1.1868829499149242e+03 2.7557570559622669e+04
Dual infeasibility......: 2.1624874764514693e-07 5.0209585722739846e-06
Constraint violation....: 5.1921755382135226e-08 5.1921755382135226e-08
Variable bound violation: 1.1735202365414921e-08 1.1735202365414921e-08
Complementarity.........: 2.0827918551588903e-07 4.8359177721495380e-06
Overall NLP error.......: 2.1624874764514693e-07 5.0209585722739846e-06
Objective...............: 1.1868829484688854e+03 2.7557570526047901e+04
Dual infeasibility......: 2.1611411147490891e-07 5.0178325304334832e-06
Constraint violation....: 5.1889360219194158e-08 5.1889360219194158e-08
Variable bound violation: 1.2335941157459729e-08 1.2335941157459729e-08
Complementarity.........: 2.0818450105630093e-07 4.8337193467056586e-06
Overall NLP error.......: 2.1611411147490891e-07 5.0178325304334832e-06


Number of objective function evaluations = 25
Number of objective gradient evaluations = 24
Number of equality constraint evaluations = 25
Number of inequality constraint evaluations = 25
Number of equality constraint Jacobian evaluations = 24
Number of inequality constraint Jacobian evaluations = 24
Number of Lagrangian Hessian evaluations = 23
Total seconds in IPOPT = 0.063
Number of objective function evaluations = 23
Number of objective gradient evaluations = 22
Number of equality constraint evaluations = 23
Number of inequality constraint evaluations = 23
Number of equality constraint Jacobian evaluations = 22
Number of inequality constraint Jacobian evaluations = 22
Number of Lagrangian Hessian evaluations = 21
Total seconds in IPOPT = 0.021

EXIT: Optimal Solution Found.
=============================================================
Expand Down Expand Up @@ -551,47 +549,47 @@ Gen Status Fuel Pg Qg Pmin Pmax Qmin Qmax
51 1 COAL 1.36 0.53 1.36 4.53 -0.55 2.11
52 1 COAL 1.36 0.65 1.36 4.53 -0.55 2.11
53 1 COAL 2.72 0.70 2.72 9.07 -1.11 4.23
65 1 COAL 86.50 -4.97 86.50 86.50 -21.66 32.04
65 1 WIND 86.50 -4.97 0.00 86.50 -21.66 32.04
67 1 COAL 1.41 -0.57 1.41 4.70 -0.57 2.19
68 1 COAL 8.38 -3.41 8.38 27.92 -3.41 13.01
69 1 COAL 8.38 -3.41 8.38 27.92 -3.41 13.01
70 1 COAL 8.38 -3.41 8.38 27.92 -3.41 13.01
71 1 COAL 8.38 -3.41 8.38 27.92 -3.41 13.01
72 1 COAL 8.38 -3.41 8.38 27.92 -3.41 13.01
73 1 COAL 8.38 -3.41 8.38 27.92 -3.41 13.01
76 1 COAL 1.20 0.52 1.20 4.00 -0.44 2.04
77 1 COAL 0.72 0.26 0.72 2.40 -0.27 1.22
78 0 COAL 0.00 0.00 5.40 18.00 -2.00 9.16
79 0 COAL 0.00 0.00 5.40 18.00 -2.00 9.16
90 1 COAL 0.96 -0.36 0.96 3.20 -0.36 1.63
91 1 COAL 1.50 -0.55 1.50 5.00 -0.55 2.55
92 0 COAL 0.00 0.00 1.89 6.30 -0.70 3.21
76 1 NG 1.20 0.52 1.20 4.00 -0.44 2.04
77 1 NG 0.72 0.26 0.72 2.40 -0.27 1.22
78 0 NG 0.00 0.00 5.40 18.00 -2.00 9.16
79 0 NG 0.00 0.00 5.40 18.00 -2.00 9.16
90 1 NG 0.96 -0.36 0.96 3.20 -0.36 1.63
91 1 NG 1.50 -0.55 1.50 5.00 -0.55 2.55
92 0 NG 0.00 0.00 1.89 6.30 -0.70 3.21
94 1 COAL 5.40 -2.20 5.40 18.00 -2.20 8.39
104 1 COAL 67.60 0.80 67.60 67.60 -14.26 21.09
105 1 COAL 154.80 6.45 154.80 154.80 -28.51 42.17
114 1 COAL 1.40 0.30 1.40 1.40 -0.24 0.36
115 1 COAL 133.50 23.61 133.50 133.50 -21.60 31.95
104 1 WIND 67.60 0.80 0.00 67.60 -14.26 21.09
105 1 WIND 154.80 6.45 0.00 154.80 -28.51 42.17
114 1 WIND 1.40 0.30 0.00 1.40 -0.24 0.36
115 1 WIND 133.50 23.61 0.00 133.50 -21.60 31.95
125 1 COAL 39.02 6.39 39.02 130.05 -15.87 60.60
126 1 COAL 39.02 16.14 39.02 130.05 -15.87 60.60
127 1 COAL 39.02 22.98 39.02 130.05 -15.87 60.60
135 1 COAL 133.92 26.66 133.92 446.40 -54.46 208.02
136 1 COAL 133.92 25.72 133.92 446.40 -54.46 208.02
147 1 COAL 92.40 11.12 92.40 92.40 -14.47 21.41
147 1 WIND 92.40 11.12 0.00 92.40 -14.47 21.41
151 1 COAL 1.62 -0.61 1.62 5.40 -0.66 2.52
152 1 COAL 23.17 -5.34 23.17 77.22 -9.42 35.99
153 1 COAL 23.17 -1.87 23.17 77.22 -9.42 35.99
154 1 COAL 23.17 -5.54 23.17 77.22 -9.42 35.99
155 1 COAL 23.17 -5.55 23.17 77.22 -9.42 35.99
161 0 COAL 0.00 0.00 41.58 138.60 -15.38 70.55
164 0 COAL 0.00 0.00 3.60 12.00 -1.33 6.11
165 0 COAL 0.00 0.00 7.80 26.00 -2.89 13.23
166 0 COAL 0.00 0.00 2.82 9.40 -1.04 4.78
167 1 COAL 2.82 -1.04 2.82 9.40 -1.04 4.78
168 0 COAL 0.00 0.00 2.82 9.40 -1.04 4.78
169 0 COAL 0.00 0.00 2.82 9.40 -1.04 4.78
170 1 COAL 2.82 -1.04 2.82 9.40 -1.04 4.78
161 0 NG 0.00 0.00 41.58 138.60 -15.38 70.55
164 0 NG 0.00 0.00 3.60 12.00 -1.33 6.11
165 0 NG 0.00 0.00 7.80 26.00 -2.89 13.23
166 0 NG 0.00 0.00 2.82 9.40 -1.04 4.78
167 1 NG 2.82 -1.04 2.82 9.40 -1.04 4.78
168 0 NG 0.00 0.00 2.82 9.40 -1.04 4.78
169 0 NG 0.00 0.00 2.82 9.40 -1.04 4.78
170 1 NG 2.82 -1.04 2.82 9.40 -1.04 4.78
182 1 COAL 5.25 0.88 5.25 17.50 -2.14 8.16
183 1 COAL 7.98 1.99 7.98 26.60 -3.25 12.40
189 1 COAL 383.40 -46.67 170.75 569.15 -46.67 209.45
196 0 COAL 0.00 0.00 20.25 67.50 -7.49 34.36
197 0 COAL 0.00 0.00 20.25 67.50 -7.49 34.36
189 1 NUCLEAR 383.40 -46.67 170.75 569.15 -46.67 209.45
196 0 NG 0.00 0.00 20.25 67.50 -7.49 34.36
197 0 NG 0.00 0.00 20.25 67.50 -7.49 34.36
Loading