feat: Implement workflow generation command with YAML validation and LLM integration

[krishvsoni]

Implemented workflow generation command with YAML validation and LLM integration

Resolves #9

What changed

New files

File	Purpose
spec/schemas/workflow.schema.json	JSON Schema describing SkillFlow workflows. Single source of truth used by both the validator and the LLM system prompt.
src/utils/schemas.ts	loadWorkflowSchema(), getWorkflowSchemaText(), validateWorkflow(yamlText) → { valid, errors[], data? }. Hand-rolled validator (no new deps).
src/utils/workflow-generator.ts	generateWorkflow({ prompt, skills, previousWorkflow?, model?, apiKey?, llm? }) — builds the system prompt + two few-shot pairs (linear pipeline, approval step), calls the LLM, strips code fences, returns YAML. llm is injectable so tests don't hit the network.
src/commands/workflow.ts	CLI: gitclaw workflow generate with -d / -p / --refine / -m / --api-key / --dry-run. Runs the 2-retry validation loop and writes workflows/<slug>.yaml.
test/workflow-validator.test.ts	11 unit tests for the validator.
test/workflow-generator.test.ts	13 unit tests for prompt assembly, fence stripping, retry loop, dry-run, and refine mode (LLM mocked).
test/ts-resolve-hook.mjs	Tiny ESM resolve hook so node --experimental-strip-types can follow Node16-style .js internal imports during tests.

Modified files

File	Change
src/index.ts	Added import { handleWorkflowCommand } and an early-dispatch branch for gitclaw workflow … (mirrors the existing gitclaw plugin … pattern).
package.json	test script preloads the resolve hook so npm test works out of the box.

CLI surface

# Generate from scratch (writes workflows/<slug>.yaml)
gitclaw workflow generate -p "every morning summarize unread emails and post to Slack"

# Iteratively refine an existing workflow
gitclaw workflow generate \
  --refine workflows/morning-email-digest.yaml \
  -p "add a human approval step before the Slack post"

# Preview without writing
gitclaw workflow generate -p "..." --dry-run

# Override model / key
gitclaw workflow generate -p "..." -m anthropic:claude-sonnet-4-5 --api-key sk-...

---

Why these choices

1. Schema on disk, not inlined

Single source of truth. spec/schemas/workflow.schema.json is loaded at runtime and embedded verbatim into the LLM system prompt — the validator and the generator cannot drift.

2. Hand-rolled validator (no AJV)

Zero new deps. AJV isn't in package.json; rather than add ajv + ajv-formats, the validator implements only what this schema needs (type, required, additionalProperties, pattern, minItems, $ref, plus a depends_on cross-field check) and exposes the same { valid, errors[] } contract.

3. LLM via @mariozechner/pi-ai (injectable)

Reuse the existing stack. pi-ai is what gitclaw already uses everywhere — supports all providers, inherits telemetry and cost tracking. The llm? option makes it swappable, so tests stay offline.

4. Retry loop, not best-effort

Bounded self-healing. When the LLM emits invalid YAML, the CLI re-prompts up to 2x with the validator's errors appended; if still invalid it prints errors + the last YAML and exit(1) — no unbounded loop, no silent failure.

5. Argv dispatch, not Commander

Match the codebase. gitclaw has no Commander dep — it uses hand-rolled argv parsing with subcommand dispatch (see gitclaw plugin …). The workflow subcommand follows the same pattern.

6. Flat requires_approval, not nested compliance.*

Stay aligned with runtime. gitclaw's existing SkillFlow shape is flat (skill/prompt/channel). A nested compliance object would diverge from the loader — the flat field is what the runtime can act on today.

---

Proof

1. Tests pass — 24/24

2. Typecheck clean

$ npx tsc --noEmit
$ echo $?
0

3. Pre-existing tests not regressed

Existing test/telemetry.test.ts — 8/8 still pass under the new npm test flags.

test/sdk.test.ts fails in this environment, but the failure is pre-existing and unrelated: it imports dist/exports.js, which requires npm run build against @mariozechner/* packages that aren't installed locally. Nothing in this PR touches the SDK surface.

4. End-to-end retry behavior demonstrated in tests

The "retries on invalid YAML, succeeds on second attempt" test plumbs an LlmClient that returns invalid YAML once, then valid YAML — and asserts the retry prompt to the LLM included the words "schema validation". From the test output above:

Generating workflow...
Retry 1/2 — fixing validation errors...
✔ runGenerate retries when validation fails, then writes the file when the second attempt is valid (6.11ms)

The "give up after 2 retries" test:

Generating workflow...
Retry 1/2 — fixing validation errors...
Retry 2/2 — fixing validation errors...
Workflow validation failed after retries:
  - (root): missing required property "name"
✔ runGenerate gives up after MAX_RETRIES and throws (0.50ms)

5. Schema-driven safety: example violations the validator catches

Input violation	Reported error
YAML with no name	(root): missing required property "name"
name: MyWorkflow (not kebab-case)	name: value "MyWorkflow" does not match pattern ^[a-z0-9]+(-[a-z0-9]+)*$
steps: []	steps: array must have at least 1 item(s), got 0
Step missing prompt	steps[0]: missing required property "prompt"
Step with unknown field nonsense: true	steps[0]: unknown property "nonsense"
depends_on: [does_not_exist]	steps[1].depends_on: references unknown step id "does_not_exist"
Truncated YAML	YAML parse error: …

---

Issue → PR mapping (acceptance checklist)

• Issue: "user describes the desired workflow in plain English" — -p "<text>"
• Issue: "LLM generates a complete SkillsFlow workflow with the correct nodes, order, and parameters" — generateWorkflow + few-shot examples; depends_on expresses ordering
• Issue: "User can review, edit, and save the generated workflow" — written as plain YAML to workflows/<slug>.yaml
• Issue: "Support iterative refinement" — --refine <file> mode
• Issue (author's scope): "Component 1 — LLM prompt template (system + few-shot)" — src/utils/workflow-generator.ts
• Issue (author's scope): "Component 2 — schema validator that catches malformed output before saving" — src/utils/schemas.ts + retry loop in src/commands/workflow.ts
• Issue: "Component 3 — UI integration into the SkillsFlow builder page (Studio side)" — explicitly out of scope per the issue author

Notes for review

• No new npm dependencies were added.
• All new TS compiles under the existing strict tsconfig.json.
• The default LLM client (pi-agent-core + pi-ai) is lazy-imported, so it never loads in tests.
• OPENAI_API_KEY / <PROVIDER>_API_KEY resolution falls back through --api-key, then provider-specific env, then OPENAI_API_KEY. Missing key produces a clear error and exit(1).

[krishvsoni]

Hey @shreyas-lyzr, pinging you on this one since you opened #9.

Quick heads-up on scope: per your follow-up comment on the issue, this PR ships

• Components 1 and 2 :
  (the LLM prompt template + few-shot examples, and the schema validator with a 2-retry self-healing loop).

• Component 3: the SkillsFlow builder UI integration on the Studio side is intentionally out of scope here, matching what you flagged as "lives in Studio."

That said, if you'd prefer to land it all in one go, I'm happy to extend this PR with:

• a minimal chat input wired into the SkillsFlow page builder
• a "Generate" action that calls gitclaw workflow generate (or the underlying generateWorkflow() SDK export) and renders the YAML back into the visual builder
• the refine flow (--refine) surfaced as a follow-up prompt in the same chat

Just let me know which way you want it:

1. Merge as-is (OSS-only scope, Studio UI tracked separately), or
2. Extend this PR to cover the Studio UI too give me the rough shape you want
   (which page/component, where the chat lives, how the generated workflow should be applied to the canvas)
   & I'll push another commit.

Either way works for me, flagging early so we don't end up doing it twice.

[shreyas-lyzr]

Three blocking issues and a handful of medium/low concerns. The overall structure is solid — schema-on-disk, injectable LLM, retry loop, good test coverage — but the items below need to be addressed before merge.

[shreyas-lyzr]

Blocking: argv[++i] silently produces undefined when a flag is the last token with no value (e.g. gitclaw workflow generate --prompt). For --prompt the downstream guard catches it, but --dir causes an unguarded resolve(undefined) in Node, which throws a confusing TypeError rather than a user-friendly message. For --api-key and --model the undefined propagates into the LLM call and surfaces far from the parse site.

Add a bounds check after each ++i:

Suggested change

	case "-m":
	case "--refine":
	if (i + 1 >= argv.length) { console.error(RED("--refine requires a value")); process.exit(2); }
	flags.refine = argv[++i];
	break;

Apply the same pattern to -d/--dir, -p/--prompt, -m/--model, and --api-key.

[shreyas-lyzr]

Blocking: --refine accepts any path, including absolute paths and ../ traversals, and passes it directly to readFile without constraining it to the agent directory.

resolve(agentDir, '/etc/passwd') resolves to /etc/passwd (Node's path.resolve ignores the base when the second argument is absolute). A user could unintentionally (or intentionally) pass --refine /etc/hostname and have its content sent to the LLM.

Add a containment check before reading:

Suggested change

	const refinePath = resolve(agentDir, flags.refine);
	const refinePath = resolve(agentDir, flags.refine);
	if (!refinePath.startsWith(agentDir + path.sep) && refinePath !== agentDir) {
	throw new Error(`--refine path must be inside the agent directory: ${refinePath}`);
	}
	previousWorkflow = await readFile(refinePath, "utf-8");

[shreyas-lyzr]

Blocking: stripCodeFences only strips fences when the entire string is a single fenced block (the regex is anchored with ^ and $). When the LLM wraps the YAML in commentary — "Here is your workflow:\nyaml\n...\n\nLet me know if..." — the regex does not match and the raw prose+fence is passed to the YAML parser, which will almost certainly fail, burning one of the two retry slots.

A more robust extractor that pulls the first fenced block regardless of surrounding text, and falls back to the raw trimmed text, would handle this reliably:

Suggested change

	const FENCE_INNER_RE = /```(?:ya?ml)?\s*\n([\s\S]*?)\n```/i;
	export function stripCodeFences(raw: string): string {
	const m = raw.match(FENCE_INNER_RE);
	return m ? m[1].trim() : raw.trim();
	}

The existing tests still pass with this change. Adding a test with leading/trailing prose would cement the behaviour.

[shreyas-lyzr]

Silent overwrite: writeFile unconditionally overwrites an existing workflow file with the same slugified name. If the LLM returns the same name: value on a second run, the previous file is silently replaced.

Consider checking whether the file exists first and either erroring out or appending a counter suffix, unless overwrite-by-default is intentional design. At minimum this should be documented in the help text.

[shreyas-lyzr]

The cross-field depends_on check verifies that referenced ids exist but does not detect cycles. A step whose depends_on includes its own id, or a cycle like A→B→A, passes validation today and would deadlock or loop at runtime.

This is not blocking if the runtime handles cycles gracefully, but worth a note or a future-issue if cycle detection is deferred.

[shreyas-lyzr]

The import expression import("@mariozechner/pi-ai" as any) is a TypeScript-specific syntax (import(specifier as SomeType)). It works under tsc because TypeScript interprets it as a type assertion on the dynamic import, but it is not valid JavaScript and will break if the file is ever processed by a non-TypeScript tool (esbuild, rollup, swc in JS mode, node strip-types in a future version that tightens this).

The correct approach is to type the result after the import:

Suggested change

	import("@mariozechner/pi-ai" as any) as Promise<any>,
	const [piAi, piAgentCore] = await Promise.all([
	import("@mariozechner/pi-ai") as Promise<any>,
	import("@mariozechner/pi-agent-core") as Promise<any>,
	]);
	const { getModel } = piAi;
	const { Agent } = piAgentCore;

This compiles identically under tsc but is syntactically clean.

[shreyas-lyzr]

Additional findings in loadFlowDefinition (src/workflows.ts) and its call site in src/voice/server.ts. This function is unchanged in the PR diff but is consumed by the new executeFlow path this PR introduces, so the bugs are activated by this PR.

---

1. Path traversal — blocking

server.ts:827 constructs the path from user-controlled input:

const flowPath = join(resolve(opts.agentDir), "workflows", flowName + ".yaml");
const flow = await loadFlowDefinition(flowPath);

flowName comes from msg.text.match(/@([a-z0-9]+(?:-[a-z0-9]+)*)/) at server.ts:3136, which does constrain characters. But loadFlowDefinition accepts an arbitrary filePath: string and passes it straight to readFile with no bounds check. Any future caller that skips the regex — or any bypass in how chat input reaches this code — leads to arbitrary file read. The function should own its own safety: change its signature to (agentDir: string, flowName: string), validate flowName against the already-present KEBAB_RE, and construct the path internally. The call site becomes loadFlowDefinition(opts.agentDir, flowName).

2. yaml.load() returns non-object values — no explicit type guard

yaml.load() returns null for an empty file and can return a string or number on single-value YAML. The current guard:

if (!data?.name || !data?.steps || !Array.isArray(data.steps))

uses optional chaining, so when data is null the condition correctly triggers — but the error says "missing name or steps" rather than "file is empty or not a mapping". More importantly, when data is a string (valid YAML), data?.name is undefined and the same misleading message fires. Add an explicit structural check first:

if (data === null || typeof data !== "object" || Array.isArray(data)) {
    throw new Error("Invalid flow definition: file must be a YAML mapping");
}

3. description not coerced to string

description: data.description || "" passes through a raw number or boolean if the YAML has description: 42. Any downstream code calling .length or other string methods on it throws. Change to String(data.description || "").

4. Per-step skill not validated for emptiness

String(s.skill || "") maps a missing or null skill to "". executeFlow then builds the prompt:

Use the skill "" (load it with /skill:).

and fires an LLM call with no error. After the steps.map(...), add:

const badStep = steps.findIndex((s) => !s.skill.trim());
if (badStep !== -1) throw new Error(`step[${badStep}] has an empty skill`);

5. id / depends_on silently dropped — semantic mismatch with schema

The new workflow.schema.json and WorkflowStep in schemas.ts declare id and depends_on as supported fields. validateWorkflow accepts them. But loadFlowDefinition maps steps to SkillFlowStep which has neither field, so they are silently dropped. executeFlow then runs all steps in declaration order regardless of declared dependencies. A workflow author who writes depends_on based on the schema will get silent wrong behavior. Either: (a) add id?: string; depends_on?: string[] to SkillFlowStep and have executeFlow respect them, or (b) remove those fields from the schema so it doesn't advertise support for them.