Repository navigation
Graph-engineering upgrades: model tiering, budget hard-stops, workflow diagrams - #12
Merged
Merged
Conversation
Distilled the actionable parts of a "graph engineering" writeup (our Workflow tool already is a graph runtime) into three concrete improvements: - Model tiering across all 6 workflows: high-volume fan-out nodes (critics, per-story evidence, per-dimension gaps, per-category E2E cases, meta-critics) run model:'sonnet'; the high-stakes gates/judgment (adversarial verifiers, status verifier, synthesis) run model:'opus'. Tiers are FANOUT_MODEL/ JUDGE_MODEL constants overridable via args.models. The loop-iteration implementer is the deliberate exception — it writes real code, so it inherits the session model rather than being silently downgraded. - Budget as a hard stop for the expensive fan-outs, guarded on budget.total (null => no cap): improve-skills skips the verify phase below a token floor and routes findings to needsDesign (unverified, never auto-applied); loop-iteration escalates all items below its floor instead of failing mid-fix. Fail-safe direction preserved — a budget stop never approves unverified work. - docs/workflow-graphs.md: Mermaid node/edge diagram for every workflow with the per-node model tier (the "see the graph before you run" view). Filed the tiering, soft/hard-budget, and diagram-drift conventions in the memory wiki. Syntax validated for all six scripts. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G
… drift checks Ran /improve-skills over the graph-engineering diff (on the new tiering itself); it confirmed 1 blocker + 7 important. Applied all: - BLOCKER (loop-iteration): the deterministic denylist gate trusted the implementer's self-reported changedFiles, but the caller applies the real diff. An under-reported file list touching a denylisted path (.claude/, .env) could bypass the gate. Now parse paths from the actual diff (parseDiffPaths) and gate on the UNION of diff-paths + changedFiles. Unit-tested: a diff that edits .claude/loop.md while changedFiles omits it now escalates. Same "give the checker the real artifact, not the self-report" rule — it applies to the deterministic gate, not just the LLM verifier. - Ported the JSON-string args normalization (previously only in improve-skills) to critic-panel, gap-analysis, e2e-design, scan-docs, loop-iteration, rewiring every args.X read to the normalized `a` so the args.models tiering override no longer silently no-ops when args arrive as a JSON string. - workflow-dsl lens now also checks model-tiering + budget-guard fail-safety; consistency-dry lens and /dream lint now check docs/workflow-graphs.md for drift vs the scripts; added docs/workflow-graphs.md to DEFAULT_TARGETS (the drift check was documented but not wired). - improve-skills.md now documents budgetStop and instructs disclosing a skipped-verification run (needsDesign items are UNVERIFIED, not design-pending). All six workflow scripts validated; the denylist union fix unit-tested. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G
Low-severity polish confirmed (recorded) by the /improve-skills gate:
- Fan-out agent labels now include the map index for consistency with the
project's own documented DSL rule ("always include the map index"): critic:,
meta:, gap:, e2e: labels switch from ${key} to ${key}-${i} in critic-panel,
improve-skills, gap-analysis, e2e-design. Harmless today (keys are unique),
but aligns every fan-out with the Verify-phase pattern in the same files.
- docs/workflow-graphs.md: note under the critic-panel diagram that
uiInScope:false drops the 5 UI-facing critics (the 10-node graph is the
default/full case).
- skills-index.md: document the args.models = { fanout, judge } tiering
override once, in the shared workflows note, so it's discoverable without
reading the .js source.
Syntax validated for all six scripts.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_014xUCrJdREvUMjoVzSX5T8G
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Distills the actionable parts of a "graph engineering" writeup (Slate / Random Labs) into our existing Workflow runtime — which already is a graph runtime (nodes = agents, edges = data hand-offs,
parallel/pipeline, scoped failure). Three real gaps were closed, then the change was run through our own/improve-skillsgate, which caught a security regression that is fixed here too.What changed
Model tiering (all 6 workflows)
model: 'sonnet'.model: 'opus'.FANOUT_MODEL/JUDGE_MODELconstants, overridable viaargs.models = { fanout, judge }.loop-iterationimplementer writes real code, so it inherits the session model rather than being silently downgraded.Budget as a hard stop (soft + hard control)
budget.total(null ⇒ no cap, so no accidentalInfinityloop).improve-skillsskips the verify fan-out below a token floor and routes findings toneedsDesign(unverified ⇒ never auto-applied).loop-iterationescalates all items below its floor instead of failing mid-fix.Diagram-first
docs/workflow-graphs.md: a Mermaid node/edge graph for every workflow, annotated with the per-node model tier — the "see the graph before you run it" view, version-controlled next to the scripts.Gate findings applied (from
/improve-skillsrun on this diff — 1 blocker + 7 important)loop-iterationdenylist bypass: the deterministic gate trusted the implementer's self-reportedchangedFiles, but the caller applies the realdiff. An under-reported file list touching a denylisted path (.claude/,.env) could slip past. Now parses paths from the actual diff (parseDiffPaths) and gates on the union of diff-paths +changedFiles. Unit-tested: a diff editing.claude/loop.mdwhilechangedFilesomits it now escalates.argsnormalization to the other 5 workflows (was only inimprove-skills), rewiring everyargs.Xread to the normalizedaso theargs.modelsoverride no longer silently no-ops.workflow-dsllens now also checks model-tiering + budget-guard fail-safety;consistency-drylens and/dream lintnow checkdocs/workflow-graphs.mdfor drift vs the scripts; added the diagram toimprove-skillsDEFAULT_TARGETS(the drift check was documented but not wired).improve-skills.mdnow documents thebudgetStopreturn field and instructs disclosing a skipped-verification run.Nice-to-haves
${key}-${i}) for consistency with the project's own DSL rule.uiInScope:falsedrops the 5 UI critics.skills-index.mddocuments theargs.modelsoverride once.Memory wiki
topics/workflow.md+log.md.Not adopted (out of scope)
The post's quant/hedge-fund framing and the standalone Slate runtime — we already have the graph runtime, and the trading claims overstate what orchestration solves (data quality, transaction costs, and overfitting remain the hard part).
Verification
node --checkwrapped in the async-function harness)./improve-skillson the new tiering itself (21 agents, Sonnet fan-out + Opus verify), and every confirmed finding is fixed in this branch.🤖 Generated with Claude Code
Generated by Claude Code