Skip to content

Make the blueprint suite assert what the blueprints actually are - #7

Merged
CraftOS-dev merged 1 commit into
mainfrom
korivi-blueprint-specs
Sep 15, 2026
Merged

CraftOS-dev merged 1 commit into
mainfrom
korivi-blueprint-specs

Conversation

@korivi-CraftOS

Copy link
Copy Markdown
Collaborator

CI has been red on main for five commits. This turns it green.

failure  346c9f0a  fix serve command on windows
failure  d7855566  plugin for pi agent
failure  331719d9  Reference to blueprint for stack guidance
failure  d619baf5  Fix help entity not found issue
failure  9ac213ae  Update openclaw plugin with command as entry point

Three blueprints failed scaffold → canon → validate with 43 unfilled <REPLACE: …> placeholders in reference/requirements.md. The suite read that as three broken blueprints. It was one broken assertion and one genuinely missing spec.

The assertion was wrong for two of them

It said every blueprint scaffolds into an app that passes validate. That is true of react-node, which ships a finished to-do starter — a View, a model, operations. It is not true, and is not meant to be true, of two others:

Blueprint What it actually ships
blueprint-base framework files only, no runtime code: no entities, no operations, a pipeline of placeholder echoes
blueprint-pocketbase-react adapter hooks only; the tasks collection is created by hand in PocketBase, and install prints "download the pocketbase binary"

For those two, validate refusing a fresh scaffold is validate working. An unauthored spec must not pass, or the check means nothing on the apps that are finished.

The only way to satisfy the old assertion would have been to invent a spec for an app that does not exist — three fictional documents, to turn a truthful red into a decorative green. The check is inverted for them instead, and each carries the reason it is incomplete:

[ ok ] blueprint-base: scaffold → canon → validate refuses (starting point, not a finished app)
[ ok ] blueprint-pocketbase-react: scaffold → canon → validate refuses (starting point, not a finished app)

The reason is a string rather than a boolean on purpose: it should read wrongly the day someone finishes the blueprint and forgets this line.

One blueprint had a real gap

blueprint-python-fastapi ships a complete, runnable agent app — model, operations, adapter, working pipeline. It simply had no spec written.

It has one now, describing what it actually is rather than copying the other starter: an app whose interface is the agent, with no browser View. Q1–Q9 of the Quality Standard concern a View it does not ship, so they are recorded n/a with that reason rather than quietly skipped — an author who later adds a View knows those entries are theirs to fill in.

Its gate now passes end to end:

✓ framework files (present and valid)
✓ ownership (system files unmodified)
✓ python syntax + adapter self-test
✓ operations resolve (every declared op has a runner)
✓ Gate: all runnable steps passed

Result

Before After
Conformance 57/60, 3 failed 60/60, 0 failed
Toolkit class 1/4 4/4

pnpm -r build and pnpm -r test both clean.

Note

This is independent of #6 (the delete guard). #6 shows red for exactly these three failures and nothing else — once this merges, #6 goes green with no change to it.

I deliberately did not write specs for base and pocketbase-react. Deciding what those two blueprints are meant to become is a call for whoever owns them, not something to settle by writing plausible-sounding requirements for apps that do not exist yet.

CI has been red on main for five commits. Three blueprints failed
`scaffold → canon → validate` with 43 unfilled `<REPLACE: …>` placeholders in
`reference/requirements.md`, and the suite read that as three broken blueprints.

It was one broken assertion and one genuinely missing spec.

The assertion said every blueprint scaffolds into an app that passes `validate`.
That is true of react-node, which ships a finished to-do starter — a View, a
model and operations. It is not true, and is not meant to be true, of two others:

  blueprint-base            framework files only, no runtime code at all: no
                            entities, no operations, a pipeline of placeholder
                            echoes. The author picks the stack.
  blueprint-pocketbase-react  adapter hooks only. Its `tasks` collection is
                            created by hand in PocketBase and `install` prints
                            "download the pocketbase binary".

For those two, `validate` refusing a fresh scaffold is `validate` WORKING. An
unauthored spec must not pass, or the check means nothing on the apps that are
finished. The only way to make the old assertion true would have been to invent
a spec for an app that does not exist — three fictional documents to turn a
truthful red into a decorative green. So the check is inverted for them instead,
and each carries the reason it is incomplete, in a string that should read
wrongly the day someone finishes the blueprint and forgets this line.

blueprint-python-fastapi was the real gap. It ships a complete, runnable agent
app — model, operations, adapter, working pipeline — and simply had no spec
written. It has one now, describing what it actually is rather than what the
other starter is: an app whose interface is the agent, with no browser View.
Q1–Q9 of the Quality Standard are recorded n/a with that reason rather than
quietly skipped, so an author who later adds a View knows those entries are
theirs to fill in. Its gate now passes end to end, framework files included.

Conformance: 57/60 with three failures, to 60/60.
@CraftOS-dev
CraftOS-dev merged commit 71a3b02 into main Sep 15, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants