Repository navigation
Phase 6: LLM layer (local Ollama, guarded, three-rung fallback) - #17
Merged
withrvr merged 1 commit intoAug 31, 2026
Merged
Conversation
…rung fallback Every run's narrative can now come from a real local LLM (phi4-mini via Ollama) -- but never has to, and never gets the final word: a numeric guard rejects any output that invents a number, falling through to the deterministic template built in Phase 5. Verified end to end against a real, unmocked Ollama instance, not just unit-tested in isolation. - ai/provider.py: call_ollama/call_cloud, the LiteLLM seam. Cloud is opt-in only (COLLECTIONS_CLOUD_LLM_MODEL), never required. - ai/guard.py: numbers_are_contained -- every number in an LLM output must appear in its input, or the output is rejected outright. - ai/roles/summary_narrator.py: narrate(), the three-rung chain (Ollama -> cloud -> ai/fallback.render_summary), numeric-guarded at every rung. Never raises -- the deterministic third rung always succeeds. - ai/context.py: anydoc workbook-to-Markdown for LLM context only -- not consumed by any Phase 6 role (summary_narrator narrates the already-computed metrics dict, never raw sheets); ready for Phase 7's exception_explainer and Phase 10's schema_mapper. - observability/metrics.py: llm_calls_total, a plain in-process counter for now -- Phase 6's own done-when, verified directly against real local Ollama calls. - models.py: Run.summary_source records which rung actually produced a run's narrative, surfaced via GET /collections/summary. - compose.override.yml: dev backend wired toward Ollama at host.docker.internal:11434, with the binding caveat documented (Ollama must listen on 0.0.0.0, not loopback-only, for a container to reach it -- verified the fallback chain degrades correctly either way). New dependencies: litellm, firecrawl-anydoc. Refs: docs/ARCHITECTURE.md's LLM layer section, docs/API.md
withrvr
merged commit Aug 31, 2026
b531205
into
claude/erp-collection-reporting-workflow-iycqto
3 of 7 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 6: LLM layer
What this adds
Three-rung narrative generation: local Ollama (
phi4-minivia LiteLLM), an optional cloud model (never required), then the Phase 5 deterministic Jinja template. Every rung's output is numeric-guarded -- an invented number is rejected outright, never surfaced.Run.summary_sourcerecords which rung actually produced a given run's narrative.Assessment requirement served
MASTER_PLAN.md Phase 6 done-when: "the pipeline completes with the network disconnected, and llm_calls_total shows local calls succeeding." Verified against a real, unmocked local Ollama instance (not just unit-tested) -- Ollama never touches the internet, so this holds literally.
Docs updated
Verification
Also verified through the actual Docker Compose stack:
docker compose exec backendreaches the real API end to end, correctly degrading tosummary_source: "fallback"since the container can't reach Windows-bound Ollama without an extraOLLAMA_HOST=0.0.0.0step (documented in ARCHITECTURE.md).Known gaps carried to next phase
ai/context.py's anydoc conversion -- not called by anything yet./metricsPrometheus scrape endpoint yet --llm_calls_totalis an in-process counter; Phase 12 formalizes it.