Skip to content

Add BDD specification skill and bridge toolkit features to tests - #54

Merged
dieterbaier merged 4 commits into
mainfrom
feature/bdd-specification-skill
Jul 10, 2026
Merged

Add BDD specification skill and bridge toolkit features to tests#54
dieterbaier merged 4 commits into
mainfrom
feature/bdd-specification-skill

Conversation

@dieterbaier

Copy link
Copy Markdown
Member

Summary

Introduces a new bdd-specification skill that enforces Living Documentation through Behaviour-Driven Development, wires it as the strict default across the feature/implementation skills, and dogfoods it by specifying and testing the toolkit's own executable features.

1. New skill: bdd-specification

  • Language-agnostic Gherkin .feature spec per behaviour, bridged into unit tests via a strict scenario-to-test naming convention and Given/When/Then anchors — for target languages with or without a native BDD framework.
  • Reference guide (skills/references/bdd-writing-guide.md), prompt template, README/prompt-index entries, regenerated agent adapters.

2. Strict-default wiring

  • architecture-impact invokes it at request/analysis time; implement-issue-workflow invokes it at the latest at implementation time.
  • Applies to any new or changed observable behaviour (features, enhancements, behaviour-changing bug fixes). Relaxed only with an explicit, recorded human waiver. Pure refactorings keep existing specs green.
  • Guidance on .feature placement: co-locate for the runner, surface into arc42 docs via include::.

3. BDD test specifications for the toolkit's features

  • Gherkin living documentation under features/ for the metamodel validator, the documentation generators, the agent adapter generator, and the validation CLI.
  • test/validate_metamodel_test.rb: existing 15 tests adapted to the bridge + 12 new validator-rule cases (27 tests).
  • test/validate_metamodel_cli_test.rb (new): CLI behaviour via subprocess (4 tests).
  • test/build-agent-adapters.test.mjs (new, node:test, no dependency): build, --check, staleness, adapter_expose, against an isolated workspace (4 tests).
  • CI: added Node setup + CLI/JS test steps to .github/workflows/validate.yml; README test docs.

Verification

Suite Result
ruby -Itest test/validate_metamodel_test.rb 27 runs, 0 failures
ruby -Itest test/validate_metamodel_cli_test.rb 4 runs, 0 failures
node --test test/build-agent-adapters.test.mjs 4 pass, 0 fail
ruby scripts/validate-metamodel.rb exit 0
node scripts/check-agent-adapters.js adapters current

Working tree stays clean after all test runs.

Out of scope

Container-based render scripts (build.sh, scripts/render-presentation.sh) remain untested — they need Docker/Podman; tracked as a follow-up in the README.

🤖 Generated with Claude Code

Dieter Baier and others added 2 commits July 10, 2026 10:36
Introduce a bdd-specification skill that enforces Living Documentation
through Behaviour-Driven Development: a language-agnostic Gherkin
feature spec bridged into unit tests via a strict scenario-to-test
naming convention and Given/When/Then anchors, for target languages
with or without a native BDD framework.

Wire it as the strict default for any new or changed observable
behaviour from architecture-impact (request/analysis time) and
implement-issue-workflow (implementation time). It is relaxed only
with an explicit, recorded human waiver; pure refactorings keep the
existing specs green.

Add a bdd-writing-guide reference, a prompt template, README and
prompt-index entries, and regenerate the agent adapters.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Dogfood the bdd-specification skill on the toolkit's own executable
features. Add Gherkin living documentation under features/ for the
metamodel validator, the documentation generators, the agent adapter
generator, and the validation CLI.

Bridge the specifications into tests with the strict scenario-to-test
naming convention and Given/When/Then anchors:
- adapt the existing validator/generator unit tests and add the missing
  validator-rule cases (duplicate ids, relation shape and keys, unknown
  relation type, front-matter errors, missing docs target, generated
  skip, report status);
- add a CLI integration test driving the script via a subprocess;
- add node:test coverage for the adapter generator (build, --check,
  staleness, adapter_expose) against an isolated workspace.

Wire the new CLI and JS test steps plus Node setup into the validate
workflow, and document the test layout in the README.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

Copy link
Copy Markdown
Member Author

Review feedback

The overall direction is strong: the scope is correctly limited to observable behaviour, the human-waiver rule prevents agents from silently bypassing the contract, existing test frameworks are preferred over introducing BDD dependencies by default, and the additional CLI/generator tests are a real improvement.

I would still adjust three conceptual points before merging.

1. Do not couple one Gherkin scenario to exactly one unit test

The current contract says that every scenario has exactly one corresponding test and that behaviour tests trace back to scenarios. This mixes two different abstraction levels:

  • Gherkin scenarios describe observable business or system behaviour.
  • Unit tests verify the technical decomposition of that behaviour.
  • One scenario may require several unit tests for boundaries, algorithms, error paths, persistence failures, events, and similar implementation concerns.
  • Many legitimate unit tests do not justify a separate business-readable scenario.

Otherwise, the feature files risk becoming technical test catalogues written in Gherkin syntax rather than useful Living Documentation.

Suggested rule:

Every scenario must have at least one identifiable automated verification. Prefer a one-to-one mapping to an acceptance, component, or use-case test where practical. Supporting unit tests do not require separate Gherkin scenarios.

A scenario should also be allowed to map to a test class, nested test group, or several supporting tests rather than exactly one test method.

The corresponding wording in skills/references/bdd-writing-guide.md should be relaxed as well, especially Keep one test per scenario and every behaviour test traces to a scenario.

2. Either validate the traceability or weaken the claim of strict coupling

The PR calls the .feature files Living Documentation and requires a strict scenario-to-test naming convention. At the moment, however, this link is only maintained through matching names, comments, and human review. No CI check verifies that each scenario still has an automated counterpart.

This means a scenario can be renamed, added, or removed while all tests and CI remain green. In that case, the documentation is not technically kept alive by the build.

Two viable approaches:

Pragmatic approach

Weaken the claim from strict/executable coupling to reviewer-verifiable traceability:

Maintain reviewer-verifiable traceability between scenarios and automated tests through stable scenario identifiers or naming conventions.

Strict approach

Add a lightweight dependency-free consistency check. Stable IDs would be more robust than scenario titles, for example:

@scenario-id(PAYMENT-003)
Scenario: Payment is rejected when the credit limit is exceeded

and in the test code:

@Scenario("PAYMENT-003")

Names are useful for readability, but poor identifiers: improving the wording of a scenario should not silently break traceability.

3. arc42 Chapter 6 should not be the default home for every scenario

Chapter 6 is a good location when a scenario explains runtime collaboration between architectural building blocks. But not every Gherkin scenario is an architecture-relevant runtime scenario.

A more differentiated rule would be:

Surface architecturally relevant scenarios in the most appropriate arc42 section. Use Chapter 6 when the scenario explains runtime collaboration, Chapter 10 for quality-related behaviour, and requirement-oriented sections or references for functional acceptance behaviour.

Otherwise, Chapter 6 may gradually become a collection of validation and acceptance details rather than a focused Runtime View.

Conclusion

The implementation and CI changes look solid, but I recommend adjusting the contract semantics before merging:

  1. Map scenarios primarily to acceptance/component/use-case verification, not exactly one unit test.
  2. Either introduce machine-checkable traceability or describe the bridge as a review convention rather than strict executable coupling.
  3. Choose the arc42 location based on architectural relevance instead of defaulting all scenarios to Chapter 6.

With those changes, the skill should provide strong BDD guidance without turning Living Documentation into a costly naming convention.

Address three conceptual review points on PR #54:

1. Decouple scenarios from unit tests. A scenario now maps to at least
   one automated verification (prefer acceptance/component/use-case),
   may map to a test class/group/several tests, and supporting unit
   tests need no separate scenario. Removes the exactly-one-test rule
   from the skill, the writing guide, and the wiring skills so feature
   files stay Living Documentation rather than Gherkin test catalogues.

2. Weaken the coupling claim to reviewer-verifiable traceability rather
   than strict executable coupling, since no CI check enforces the link.
   Recommend a stable scenario id (for example @scenario-id(...)) as a
   more robust identifier than the title. No new dependency or checker.

3. Choose the arc42 location by architectural relevance instead of
   defaulting every scenario to Chapter 6: Chapter 6 for runtime
   collaboration, Chapter 10 for quality behaviour, requirement-oriented
   sections for functional acceptance; non-architectural scenarios may
   stay in features/.

Docs-only change; validator, CLI, and adapter tests remain green.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@dieterbaier

Copy link
Copy Markdown
Member Author

Danke für das Review — alle drei Punkte sind in 33fd7b5 umgesetzt (reine Doku-/Vertragsänderung; Validator-, CLI- und Adapter-Tests bleiben grün).

1. Szenario ↔ Unit-Test entkoppelt. Neue Regel im Skill, im bdd-writing-guide.md und in den verdrahteten Skills: jedes Szenario hat mindestens eine identifizierbare automatisierte Verifikation; 1:1 auf einen Akzeptanz-/Komponenten-/Use-Case-Test bevorzugt, ein Szenario darf auf eine Testklasse/-gruppe oder mehrere Tests abbilden, und stützende Unit-Tests (Grenzfälle, Fehlerpfade, Persistenz, Events) brauchen kein eigenes Szenario. Die „exactly one test per scenario"-Formulierung ist überall entfernt. Deine vorgeschlagene Regel habe ich nahezu wörtlich übernommen.

3. arc42-Ort nach architektonischer Relevanz. Kapitel 6 ist nicht mehr Default. Neue Differenzierung: Ch6 für Runtime-Kollaboration zwischen Bausteinen, Ch10 für qualitätsbezogenes Verhalten, anforderungsorientierte Sektionen/Referenzen für funktionale Akzeptanz — und nicht-architekturrelevante Szenarien dürfen in features/ bleiben, ohne in arc42 zu wandern.

2. Traceability — Entscheidung nötig. Ich habe den pragmatischen Pfad gewählt: der Bridge ist jetzt explizit als reviewer-verifizierbare Konvention beschrieben, nicht als build-erzwungene/ausführbare Kopplung (Skill, README und Feature-Header entsprechend umformuliert). Zusätzlich empfiehlt das Skill jetzt stabile Szenario-IDs (z. B. @scenario-id(PAYMENT-003) + passender Marker am Test) als robusteren Identifier als den Titel, da Umbenennungen sonst die Traceability still brechen.

Bewusst nicht hinzugefügt: der maschinenprüfbare CI-Check aus deinem „strict approach" — das hätte stabile IDs in allen .feature-Dateien und Tests plus einen neuen (wenn auch dependency-freien) Checker bedeutet, was der „keine neue Automatisierung/Dependency by default"-Linie des PRs widerspricht.

👉 Wenn du für dieses Repo doch die strikte, maschinenprüfbare Variante möchtest, rüste ich sie gern nach: @scenario-id-Tags in die vier .feature-Dateien, Marker in die Tests, und einen kleinen dependency-freien Consistency-Checker als zusätzlichen CI-Schritt. Sag einfach Bescheid.

@dieterbaier

Copy link
Copy Markdown
Member Author

Kleine Restbeobachtung

Im Writing Guide steht in der Einleitung noch:

keep them strictly coupled to the tests

Direkt darunter wird diese Kopplung allerdings korrekt relativiert und als reviewer-verifizierbare Konvention erklärt.
Das ist kein Merge-Blocker. Ich würde strictly coupled optional noch durch clearly traceable ersetzen, damit schon der Einstieg exakt zur späteren Regel passt.

Fazit

Approved / mergebar.

Die Nacharbeit trifft das Feedback sehr gut und macht den Skill wesentlich praxistauglicher, ohne den Anspruch an Living Documentation weichzuspülen.

Replace "strictly coupled to the tests" with "clearly traceable to the
tests" in the writing guide intro so the opening matches the
reviewer-verifiable traceability rule described below it.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@dieterbaier
dieterbaier merged commit df25079 into main Jul 10, 2026
1 check passed
dieterbaier pushed a commit that referenced this pull request Jul 10, 2026
Address three conceptual review points on PR #54:

1. Decouple scenarios from unit tests. A scenario now maps to at least
   one automated verification (prefer acceptance/component/use-case),
   may map to a test class/group/several tests, and supporting unit
   tests need no separate scenario. Removes the exactly-one-test rule
   from the skill, the writing guide, and the wiring skills so feature
   files stay Living Documentation rather than Gherkin test catalogues.

2. Weaken the coupling claim to reviewer-verifiable traceability rather
   than strict executable coupling, since no CI check enforces the link.
   Recommend a stable scenario id (for example @scenario-id(...)) as a
   more robust identifier than the title. No new dependency or checker.

3. Choose the arc42 location by architectural relevance instead of
   defaulting every scenario to Chapter 6: Chapter 6 for runtime
   collaboration, Chapter 10 for quality behaviour, requirement-oriented
   sections for functional acceptance; non-architectural scenarios may
   stay in features/.

Docs-only change; validator, CLI, and adapter tests remain green.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@dieterbaier
dieterbaier deleted the feature/bdd-specification-skill branch July 10, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant