AgentiGrid: robust agent iterations, accurate UI timeline, LLM usage accounting

[ahajdar]

Replaces #139 (fork PR, closed so CI can run).

Merge request type

• New feature
• Resolves bug
• Documentation
• Other

Relates to

• OPFLOW
• SOPFLOW
• SCOPFLOW
• TCOPFLOW
• CMake build system
• Spack configuration
• Manual
• Web docs
• Other (AgentiGrid)

This MR updates

• Header files
• Source code
• CMake build system
• Spack configuration
• Web docs
• Manual
• Other (AgentiGrid UI, config templates, tests)

Summary

Makes AgentiGrid's agent loop robust to bad LLM output, fixes how the UI timeline reports such iterations, and adds LLM usage accounting. Found in long runs with local models (Ollama), where malformed or unparseable actions are common. All changes are in agentigrid/.

Agent loop (engine/agent_loop.py):

• An exception in an action handler no longer ends the run; the iteration is discarded with feedback to the model. Sweep, boundary and reserve handlers tolerate a non-dict feasibility / candidate_set.
• UI timeline bug: when an iteration recorded no entry, the callback re-sent the previous entry, and the end of the run re-sent the last entry again. The timeline then showed duplicates and stale, lower-numbered cards (e.g. "Iteration 6" after iteration 20). Now the callback fires only for a new entry, and a UI-only "discarded" card explains a rejected iteration. Same in resume.
• journal.discarded_actions records each iteration that produced no entry: rejected / api_error / truncated / internal, the feedback, stop_reason, and the raw response (capped at 4,000 characters). Entries and summary statistics are unchanged.
• Separate handling for output cut at max_tokens; a modify that applies no command says so; modify entries keep skipped_commands.
• Optional stall breaker: AGENTIGRID_STALL_LIMIT=N stops after N consecutive iterations without progress. Off by default; a sidebar checkbox sets it.

Backends (backends/):

• Anthropic prompt caching of the per-session system prompt (llm.prompt_cache, default on). Billing only; the model sees the same text. prompt_tokens remains the total input; cache writes and reads are reported separately.
• UsageMeter counts every LLM call, including post-search analysis; totals are exported in the journal as llm_usage.
• Models that reject temperature are retried once without it.
• LLMResponse gains api_error, stop_reason, content_types and cache token fields. AGENTIGRID_MAX_TOKENS overrides the output-token cap.

UI and reports:

• Iteration count shown as "N + base case" (a run with max 2 showed "Iterations 3"). summary_stats() adds llm_iterations and has_base_case; total_iterations is unchanged.
• FAILED is shown only for solves that did not converge, not for analysis / complete / sweep / contingency / explore entries or discarded proposals.
• Default Anthropic model claude-sonnet-4-20250514 (retired) → claude-sonnet-4-6.

Testing: new tests/llm/test_prompt_cache.py and tests/journal/test_iteration_count.py; 8 new tests in tests/agent_loop/test_agent_loop.py (4 of them fail on the previous agent loop). python -m pytest -m "not slow" from agentigrid/ without ExaGO binaries: 1103 passed, 12 skipped, 2 failed. The 2 failures are the OPFLOW fixture issues fixed by the genfuel PR (#137); with both applied: 1108 passed, 13 skipped. The changelog entry is in agentigrid/claude_code_changes.md.

AI assistance: prepared with Claude (Anthropic); the commit carries Co-Authored-By: Claude.

Linked Issue(s)

None filed. Independent of #137; both apply cleanly in either order.