From fdf171afcf1418192ae070582e2cef59cb9aea71 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 05:56:46 +0300 Subject: [PATCH 01/14] docs(emitters): hold every variant an anyOf value admits An anyOf lets one value satisfy several variants, and the emitter design lowered every untagged union to a closed sum that holds exactly one. Where two variants admit the same value, holding the first misnames it and drops the members only the others declare. The plan now proves, once per union, whether two variants of an anyOf can admit one value (TypePlan.Inclusive), by JSON kind, by finite value sets and by a member one model requires that the other cannot hold. A union it proves disjoint lowers as a closed sum, as a oneOf does. One it cannot is inclusive: it holds an optional member per variant, set for each variant the value admits, decodes into every candidate variant, and encodes the merge, a disagreement being an encode error. A null a nullable variant admits is held by the reference, never by a variant. Go lowers an inclusive union to a struct with a pointer per variant and Kotlin to a class with a nullable property per variant. T-29 pins the round trip of a value two open models admit. --- docs/emitters/plan/types.md | 33 +++++++++++++++ docs/emitters/refine/lowering-table.md | 2 +- docs/emitters/refine/steps.md | 4 +- docs/emitters/refine/unions.md | 53 +++++++++++++++++++++++- docs/emitters/targets/go/lowering.md | 4 +- docs/emitters/targets/go/unions.md | 24 +++++++++++ docs/emitters/targets/kotlin/lowering.md | 2 +- docs/emitters/targets/kotlin/unions.md | 9 ++++ docs/emitters/testing/mandatory-tests.md | 9 ++++ 9 files changed, 132 insertions(+), 8 deletions(-) diff --git a/docs/emitters/plan/types.md b/docs/emitters/plan/types.md index 72afbe89e..1a272abdb 100644 --- a/docs/emitters/plan/types.md +++ b/docs/emitters/plan/types.md @@ -14,6 +14,8 @@ type TypePlan struct { Discrim *DiscrimFacts // a union's or hierarchy's discriminator, as variant IDs Open bool // a union or polymorphic base keeps a value no variant // recognizes rather than refusing it (policy.md, ../refine/unions.md) + Inclusive bool // an any_of union two of whose variants can admit one value, + // holding every variant a value admits (below); false otherwise Scalar *ScalarChain // a scalar's Base chain, walked once; nil = not a scalar Shapes map[ir.Lifecycle]ModelShape // the wire shapes the document exercises (below) Alias ir.TypeID // the type every reference to this one is emitted as, this @@ -69,6 +71,37 @@ to it. A conjunct that is no model, as `Tags` is in `allOf: [{$ref: Pet}, {$ref: `Tags` is a list, is in `Unflattened` as the entry names it: the declaration over `Flattened` does not hold what it admits, so each refiner lowers it or degrades with a code of its own. +## Variants that can share a value + +`Inclusive` is true on an `any_of` union when two of its variants can admit one value, and false on +every other shape. What admitting a value means is in +[unions](../refine/unions.md#a-value-several-variants-admit). The plan proves two variants disjoint, +admitting no value in common, by these rules, and a pair none of them separates can share one: + +- **Kinds.** Their JSON kinds have none in common. A variant's kinds are its type's: object for a + model or a map, array for a list or a tuple, its wire kind for a primitive, its values' for an + enum or a literal, its variants' for a union, its root's for a scalar, its base's for a model that + only stands over one, and every kind for `any`. A nullable reference adds `null`, except a + variant's own, whose `null` the union holds + ([unions](../refine/unions.md#a-null-a-variant-admits)), and an integer is a number too. +- **Values.** Each admits finitely many values, as a closed enum, a literal or a union of them does, + and no value is in both, compared as JSON values. +- **Members.** Both are models, read through `Flattened`, and one requires a member the other either + does not declare and is closed, or declares at a position disjoint from it by these rules. A pair + already being compared counts as sharing, so a cycle of models ends the walk. + +The rules are sound, not complete. A pair they leave sharing may still admit no common value, as a +`date` and a `uuid` string do, and is then inclusive: it holds every value correctly, though more +loosely than a closed sum would. `Constraints` take no part, since admitting a value leaves them to +validation, so two variants that differ only in a bound do share values. + +On the golden spec (spec @319dc756, morphic @b206dc99), 515 unions are untagged, not exclusive, and +have two or more variants. These rules prove 419 of them disjoint, and those lower as a closed sum. +Of the 96 left, 72 have two variants that both admit an object, such as the 16 identity providers +an Access listing returns. Each refines one base whose `type` enum lists all 16, and each `config` +is open with no required member, so nothing separates them: one provider's value is admitted by +every other whose `config` declares none of its members at another type. + ## ModelShape — visibility projection One logical model produces N wire shapes across the lifecycle. `ModelShape(model, lifecycle)` is a diff --git a/docs/emitters/refine/lowering-table.md b/docs/emitters/refine/lowering-table.md index 651a3173b..a00371ac3 100644 --- a/docs/emitters/refine/lowering-table.md +++ b/docs/emitters/refine/lowering-table.md @@ -23,7 +23,7 @@ IR bug (INV9), not a compiler change. | anonymous decorator: a `Model` with only a `Base`, or a `Scalar` with neither `Constraints` nor `Encoding`, beside annotations ([decorators](decorators.md)) | — | | `Model` whose `Base` is no model, not a decorator | — | | `AdditionalProps` / `Additional` | pattern channel | -| `Union` (four wire shapes, [unions](unions.md)) | `WireName`/`WireID` per variant; an unknown variant's value | +| `Union` (four wire shapes, [unions](unions.md)) | `WireName`/`WireID` per variant; an unknown variant's value; every variant an inclusive union's value admits | | `Enum` open/closed/flags | member `WireName` | | `List`/`MapT`/`Tuple` | packed/expanded | | `Literal` | — | diff --git a/docs/emitters/refine/steps.md b/docs/emitters/refine/steps.md index de5824618..7850552d9 100644 --- a/docs/emitters/refine/steps.md +++ b/docs/emitters/refine/steps.md @@ -8,9 +8,9 @@ | `LowerLiterals` | `Literal{Value}` | typed constant / single-value type (incl. `symbol`) | [containers](../../ir/containers.md) | | `LowerModels` | `TypePlan.Flattened`, `Required`, `Undeclared`, `Unflattened`, `Base`/`Implements`/`Mixins`, `AdditionalProps`, `Additional`, `Positional` | a declaration over the flattened properties, an `Undeclared` member a required field typed as the catch-all types it, the catch-all holding the undeclared members, unknown members rejected on `closed`, an `Unflattened` conjunct lowered or degraded with a code | [models](../../ir/models.md) | | `ProjectVisibilityShapes` | `plan.ModelShape` per lifecycle | one declaration per shape, such as `User` / `UserCreate` / `UserPatch` | [properties](../../ir/properties.md#visibility) | -| `LowerOptionality` | `TypePlan.Required` (a member's wire presence, [TypePlan](../plan/types.md)) ⊥ `TypeRef.Nullable` ⊥ `Property.Presence`; `Facts.Component` | the four+ states in the target's forms; a reference on a cycle the target holds by value broken as its type system requires ([recursive references](models.md#recursive-references)) | [properties](../../ir/properties.md); INV8 | +| `LowerOptionality` | `TypePlan.Required` (a member's wire presence, [TypePlan](../plan/types.md)) ⊥ `TypeRef.Nullable`, a reference to a union with a nullable variant counting as nullable ([unions](unions.md#a-null-a-variant-admits)) ⊥ `Property.Presence`; `Facts.Component` | the four+ states in the target's forms; a reference on a cycle the target holds by value broken as its type system requires ([recursive references](models.md#recursive-references)) | [properties](../../ir/properties.md); INV8 | | `LowerConstraints` | `Constraints` on properties/scalars/lists/maps | field-level validation metadata + a refine-built validation method structure | [constraints](../../ir/constraints.md) | -| `LowerUnions` | `Union{Variants, Exclusive, WireTagged, Discriminator}`, a polymorphic base's `Discriminator` or `Abstract` with `Facts.Subtypes`, `TypePlan.Union`, `TypePlan.Open` | the target's closed sum, one member per variant or subtype + a fallback on an open one | [unions](../../ir/unions.md); INV2 | +| `LowerUnions` | `Union{Variants, Exclusive, WireTagged, Discriminator}`, a polymorphic base's `Discriminator` or `Abstract` with `Facts.Subtypes`, `TypePlan.Union`, `TypePlan.Open`, `TypePlan.Inclusive` | the target's closed sum, one member per variant or subtype + a fallback on an open one; an inclusive union one optional member per variant, each set when its variant admits the value ([unions](unions.md#a-value-several-variants-admit)) | [unions](../../ir/unions.md); INV2 | | `LowerDiscriminator` | `Discriminator{Property/PropertyName/Index, Mapping→TypeID, Default, Envelope}` | a tag-dispatched decoder + typed factories | [models](../../ir/models.md)–[unions](../../ir/unions.md) | | `LowerOpenEnums` | `Enum.Closed`, `ValueType`, `Members`, `Flags`, `FallbackMember` | the target's closed, open and flags forms; an open one keeps an unknown value | [enums](../../ir/enums.md) | | `ExtractInterfaces` | `Implements`, `OperationGroup` | interfaces / client interfaces for testability | [models](../../ir/models.md) | diff --git a/docs/emitters/refine/unions.md b/docs/emitters/refine/unions.md index ff468cd09..572c88b71 100644 --- a/docs/emitters/refine/unions.md +++ b/docs/emitters/refine/unions.md @@ -33,8 +33,8 @@ survive for (de)serialization. **Unions never degrade to optional-field merges** *body* union into method arguments is an opt-in `ShapeHint` ([shaping hints](../contract/shaping-hints.md)), never a silent loss (the oagen counterexample, prior-art §1). A future TS/Python target lowers the *same* facts to a native `A | B | C`; the decision differs, the -fact base is identical. What a union holds when a value satisfies several variants of an `anyOf` is -#959's. +fact base is identical. What a union holds when several variants of an `anyOf` admit one value is +[below](#a-value-several-variants-admit). Within the untagged case, choosing *how* a variant is recognized on decode is refine's job, and ogen's `schema_gen_sum.go` is the battle-tested enumeration to lift: try, in order, explicit @@ -47,6 +47,55 @@ below, which keeps its bytes as they arrived — a degrade, never a whole-docume ([T-7](../testing/mandatory-tests.md)). That graceful-degradation margin is exactly what the ABI seam buys and the fused-pipeline generator cannot have. +## A null a variant admits + +A variant whose reference is `Nullable` admits `null`, and `null` holds nothing a variant could keep +apart, so the union holds it itself. Every reference to a union with a nullable variant lowers as a +nullable reference does ([optionality](models.md#optionality--nullability--presence-inv8)), and a +`null` never reaches a variant. The golden spec's single-result envelope declares `result` as +`anyOf: [{type: object, nullable: true}, {type: string, nullable: true}]`: an object, a string or +`null`, the last held by the reference. + +## A value several variants admit + +An `anyOf` lets one value satisfy several variants, and what the union holds then turns on +`TypePlan.Inclusive`, which the plan proves once +([TypePlan](../plan/types.md#variants-that-can-share-a-value)): + +- **Not inclusive.** No value is admitted by two variants, so holding the one that admits it loses + nothing, and the union lowers as a `oneOf` does: the target's closed sum, recognized as above. +- **Inclusive.** The union holds every variant that admits the value, each whole and under its own + name: an optional member per variant, set when that variant admits the value. Holding only the + first would misname the value and drop what the others declare. An Access identity provider would + decode as whichever provider is listed first, whatever its `type`, and its own `config` members + would be lost. + +A tag picks one variant, so a discriminated or key-tagged union is never inclusive, whatever +`Exclusive` says. + +A variant admits a value when the value decodes into it, with each codec, closed model, closed enum +and literal refusing what it refuses, and every model the value reaches carries each member its +`TypePlan.Required` lists. `Constraints` take no part, since they are validation's on every decoded +value. So whether a variant admits a value never depends on a bound. + +Decoding an inclusive union tries each variant whose kinds include the value's JSON kind, in +declaration order, and sets every one that admits the value. If none admits it, the order +[below](#a-value-no-variant-recognizes) applies: the fallback on an open union, a coded decode error +on a closed one. Trying every variant is what `anyOf` asks: no step of the ladder above can tell +whether a second variant admits a value too. + +Encoding writes the merge of what each set variant encodes. Objects merge member by member, in the +order the first set variant writes them, then each later variant's new members; arrays merge element +by element and must have one length; scalars must be equal. A member or element two variants write +must be the same JSON value, numbers compared by value, and a disagreement is an encode error, never +a silent pick. A union with no variant set refuses to encode and fails validation, and the fallback +is never set beside a variant. + +This is not the optional-field merge [the rule above](#variant-identity-and-recognition) forbids. +That merge pools the variants' properties into one bag, losing which variant a member belongs to and +admitting values no variant admits. Here each variant keeps its identity and its whole shape, and +validation still requires one. + ## A value no variant recognizes All four wire shapes decode in one order: the variant the tag or the key names, or the one diff --git a/docs/emitters/targets/go/lowering.md b/docs/emitters/targets/go/lowering.md index ffa288524..d31982970 100644 --- a/docs/emitters/targets/go/lowering.md +++ b/docs/emitters/targets/go/lowering.md @@ -17,7 +17,7 @@ Go's normative lowering of each construct of the [lowering table](../../refine/l | anonymous decorator: a `Model` with only a `Base`, or a `Scalar` with neither `Constraints` nor `Encoding`, beside annotations ([decorators](../../refine/decorators.md)) | no declaration: each reference lowered as one to the first type along its `Base` that is not a decorator, its docs, deprecation and examples folded onto the carrier | | `Model` whose `Base` is no model, not a decorator | with no property, catch-all or discriminator of its own, lowered as the `Scalar` row lowers an alias over that base, its constraints, docs and examples binding the position ([scalars](../../../ir/scalars.md)); a union base beside properties distributed over the variants as [models](../../../ir/models.md#composition-beside-a-union) distributes a union co-declared with a composition; anything else a diagnosed degrade | | `AdditionalProps` / `Additional` | catch-all `map` as the embedded fallback; `closed`→`RejectUnknownMembers(true)`, pinned per model ([`User`](models.md#user-in-go)) | -| `Union` (four wire shapes, [unions](../../refine/unions.md#the-wire-shapes)) | a struct holding a sealed interface + marker + a wrapper per variant + a tag-dispatch (de)serializer on the struct; `Open` → an `Unknown` fallback | +| `Union` (four wire shapes, [unions](../../refine/unions.md#the-wire-shapes)) | a struct holding a sealed interface + marker + a wrapper per variant + a tag-dispatch (de)serializer on the struct; `Open` → an `Unknown` fallback; an inclusive `anyOf` → a struct with a pointer per variant ([unions](unions.md#an-inclusive-union)) | | `Enum` open/closed/flags | consts named by enum then member (`StatusActive`, [enums](types.md#enums)): closed→typed consts; open→string+consts+`Unknown`; flags→bitset; `FallbackMember` on decode | | `List`/`MapT`/`Tuple` | `[]T`/`map[K]V`/positional struct (optional past `MinItems`, `Rest`→trailing slice); `List.Encoding`→serializer mode | | `Literal` | typed constant (incl. `symbol`) | @@ -51,7 +51,7 @@ lowers to in Go. | `ProjectVisibilityShapes` | `User` / `UserCreate` / `UserPatch` structs | | `LowerOptionality` | the four+ states → `T` / `*T` / `Opt[T]`/`Nil[T]`/`OptNil[T]` / `omitzero` + presence bit; a reference on a cycle the target holds by value held through a pointer ([recursive references](models.md#recursive-references)) | | `LowerConstraints` | field-level `validate` tags + a refine-built `Validate()` method structure | -| `LowerUnions` | a struct holding a sealed interface + marker method + a wrapper per variant or subtype + a fallback on an open one | +| `LowerUnions` | a struct holding a sealed interface + marker method + a wrapper per variant or subtype + a fallback on an open one; on an inclusive union, a struct with a pointer per variant | | `LowerDiscriminator` | tag-dispatched `UnmarshalJSONFrom` on the union's struct + typed factories | | `LowerOpenEnums` | closed → typed consts; open → `string`+consts+`Unknown(raw)`; flags → bitset | | `ExtractInterfaces` | Go interfaces / client interfaces for testability | diff --git a/docs/emitters/targets/go/unions.md b/docs/emitters/targets/go/unions.md index 363a31486..226fc2ad5 100644 --- a/docs/emitters/targets/go/unions.md +++ b/docs/emitters/targets/go/unions.md @@ -43,6 +43,30 @@ The fallback ([the shared rule](../../refine/unions.md#a-value-no-variant-recogn variant, `PaymentUnknown`, holding the value as `jsontext.Value`. It re-encodes as the same JSON value, though json/v2 normalizes whitespace and string escapes as it writes. +## An inclusive union + +An inclusive union ([the shared rule](../../refine/unions.md#a-value-several-variants-admit)) holds +every variant a value admits, so it is a struct with a pointer per variant and no interface: + +```go +type Target struct { // anyOf: [Zone, Account], which one value can fit + Zone *Zone // set when Zone admits the value + Account *Account // set when Account admits it, beside Zone or alone + Unknown jsontext.Value // an open union's fallback, never set beside a variant +} +``` + +- **A pointer per variant**, whatever the variant's Go type, so nil means "does not admit it" for a + slice or a string too. Each field is named as the variant's wrapper in a closed sum is, so two + variants that lower to one Go type stay apart. +- **The struct declares the codec pair**, as a closed sum's struct does. `UnmarshalJSONFrom` buffers + the value (`dec.ReadValue()`), decodes it into each variant under `dec.Options()`, and keeps each + that admits it: the decode succeeds, and the unexported presence check generated beside the + variant's `Validate`, which reads no `Constraints`, finds every required member. `MarshalJSONTo` + writes the merge. +- **`Validate` reports a struct with no field set**, and validates each field that is. With no sum + to close, it has no marker, no wrapper and no switch-completeness test. + ## Discriminators `LowerDiscriminator` emits a tag-dispatched `UnmarshalJSONFrom` that peeks the discriminator diff --git a/docs/emitters/targets/kotlin/lowering.md b/docs/emitters/targets/kotlin/lowering.md index 83b8be2f7..be46aac3d 100644 --- a/docs/emitters/targets/kotlin/lowering.md +++ b/docs/emitters/targets/kotlin/lowering.md @@ -19,7 +19,7 @@ where Multiplatform differs from the JVM ([Multiplatform common code](multiplatf | anonymous decorator: a `Model` with only a `Base`, or a `Scalar` with neither `Constraints` nor `Encoding`, beside annotations ([decorators](../../refine/decorators.md)) | no declaration: each reference lowered as one to the first type along its `Base` that is not a decorator, its docs, deprecation and examples folded onto the carrier | | `Model` whose `Base` is no model, not a decorator | with no property, catch-all or discriminator of its own, lowered as the `Scalar` row lowers an alias over that base, its constraints, docs and examples binding the position ([scalars](../../../ir/scalars.md)); a union base beside properties distributed over the variants as [models](../../../ir/models.md#composition-beside-a-union) distributes a union co-declared with a composition; anything else a diagnosed degrade | | `AdditionalProps` / `Additional` | catch-all `Map` filled by a serializer that reads the whole object ([unknown members](models.md#unknown-members)); strict `Json`, `@JsonIgnoreUnknownKeys` on each other open model | -| `Union` (four wire shapes, [unions](../../refine/unions.md#the-wire-shapes)) | sealed interface + one wrapper class per variant + a `KSerializer` per wire shape; `Open` → an `Unknown` fallback; KMP: not exhaustive in Swift | +| `Union` (four wire shapes, [unions](../../refine/unions.md#the-wire-shapes)) | sealed interface + one wrapper class per variant + a `KSerializer` per wire shape; `Open` → an `Unknown` fallback; KMP: not exhaustive in Swift; an inclusive `anyOf` → a class with a nullable property per variant ([unions](unions.md#an-inclusive-union)) | | `Enum` open/closed/flags | closed → `enum class` with `@SerialName` members, `FallbackMember` on decode; open → value class over the value type with a constant per member, which keeps an unknown value; flags → value class over an integer | | `List`/`MapT`/`Tuple` | `List`/`Map`/a class with a positional serializer (nullable past `MinItems`, `Rest` → trailing `List`) | | `Literal` | `const val`, or an `object` for a non-primitive value | diff --git a/docs/emitters/targets/kotlin/unions.md b/docs/emitters/targets/kotlin/unions.md index 25c7a4277..c03216fa4 100644 --- a/docs/emitters/targets/kotlin/unions.md +++ b/docs/emitters/targets/kotlin/unions.md @@ -12,3 +12,12 @@ Kotlin's variant classes are the same wrapper as [Go's](../go/unions.md), for th The fallback ([the shared rule](../../refine/unions.md#a-value-no-variant-recognizes)) is `Payment.Unknown`, holding a `JsonElement`. + +## An inclusive union + +An inclusive union ([the shared rule](../../refine/unions.md#a-value-several-variants-admit)) is a +final class with a nullable property per variant, named as the variant's wrapper class is, and +`unknown: JsonElement?` on an open one. Its `KSerializer` reads the value as a `JsonElement`, +decodes it into each variant through `Json.decodeFromJsonElement`, and keeps each that admits it: +the decode succeeds and no required member was absent. It encodes the merge. `validate()` reports a +value with no property set, and no `init` check does, since a decoded value always has one. diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index 817580617..3b035c68b 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -142,3 +142,12 @@ throughout these documents. open. An option beside it is taken in priority order, an option ANDing it with a declared scheme needs both, and the service's credentials are never sent in its place ([auth](../operations/auth.md#unresolved-schemes)). +29. **T-29 · Inclusive unions** — on every target, an `anyOf` over two open models that one value + fits decodes that value into both and re-encodes it as an equal JSON value; a member the two + write at different values is an encode error, and a union with neither set refuses to encode. + An `anyOf` the plan proves disjoint, by JSON kind, by enum values and by a member one model + requires and the other cannot hold, lowers as a closed sum and decodes each value into the one + variant admitting it, and a `null` two nullable variants admit is held by the reference, as a + nullable one's is. A value no variant admits takes the fallback on an open union and is a coded + error on a closed one ([unions](../refine/unions.md#a-value-several-variants-admit), + [TypePlan](../plan/types.md#variants-that-can-share-a-value)). From 6ad8bb65f5dc1c61bfcb9ca21bd805c474f8b743 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:00:19 +0300 Subject: [PATCH 02/14] docs(emitters): sweep the inclusive union through every rule it reaches A positional model is an array on the wire, so the kinds rule now says so and the member rule reads only models laid out by name. A stream that ends on a terminal variant now ends on an element holding one, among the several an inclusive union holds. A null literal variant now lowers as a nullable variant does, so either spelling of a null member reaches the reference. Inclusive joins the plan fields the plan-or-refine test keeps, and the docs emitter says when a value may be several variants at once. --- docs/emitters/docs-and-mock.md | 3 ++- docs/emitters/operations/streaming.md | 6 ++++-- docs/emitters/plan/plan-or-refine.md | 13 ++++++++----- docs/emitters/plan/types.md | 17 +++++++++-------- docs/emitters/refine/steps.md | 2 +- docs/emitters/refine/unions.md | 14 ++++++++------ docs/emitters/targets/go/operations.md | 5 +++-- docs/emitters/testing/mandatory-tests.md | 7 ++++--- 8 files changed, 39 insertions(+), 28 deletions(-) diff --git a/docs/emitters/docs-and-mock.md b/docs/emitters/docs-and-mock.md index 96ac56ce7..259ad62a4 100644 --- a/docs/emitters/docs-and-mock.md +++ b/docs/emitters/docs-and-mock.md @@ -12,7 +12,8 @@ plan once: docs and mocks agree with the SDK by construction, not by re-derivati *same* IR-ID lookup the Go emitter uses for doc comments, so a link follows what its target renders as. Examples use the schema arm (`Value/Headers`) vs the operation arm (`Input/Output/Error`) correctly. A union `TypePlan.Open` marks is documented as one a response - may extend, and every `PlannedError` and `PlannedResponse` as the body and headers it declares. No + may extend, one `TypePlan.Inclusive` marks as one whose value may be several variants at once, and + every `PlannedError` and `PlannedResponse` as the body and headers it declares. No runtime `Policy` is consulted — proving the IR/policy split pays off. - **Mock-server emitter (`emitters/mock`).** Reuses `PrimaryResponse`, the content `Return` elects and the IR `Examples` to synthesize canned responses, `Others` to answer as each declared response diff --git a/docs/emitters/operations/streaming.md b/docs/emitters/operations/streaming.md index 58f29b4ce..ff7de1201 100644 --- a/docs/emitters/operations/streaming.md +++ b/docs/emitters/operations/streaming.md @@ -26,8 +26,10 @@ decode](../plan/responses.md#a-body-that-does-not-decode)). What a stream yields `Recv`: - an `Element` lowers as a field of its type does ([lowering table](../refine/lowering-table.md)), - whatever its kind. A stream ends after a variant whose `Variant.Event.Terminal` is true, so an - event union needs nothing of its own. Under `FrameMembers` the element decodes from the frame's + whatever its kind. A stream ends after an element holding a variant whose + `Variant.Event.Terminal` is true, any of the several an [inclusive + union](../refine/unions.md#a-value-several-variants-admit) holds included, so an event union + needs nothing of its own. Under `FrameMembers` the element decodes from the frame's `data`, `event`, `id` and `retry`, and otherwise from its `data`. - no `Element` yields each frame's text, unread. - a `bytes` side has no frames: a side the client reads lowers as a file does ([return diff --git a/docs/emitters/plan/plan-or-refine.md b/docs/emitters/plan/plan-or-refine.md index 028b8812b..a34e12438 100644 --- a/docs/emitters/plan/plan-or-refine.md +++ b/docs/emitters/plan/plan-or-refine.md @@ -35,11 +35,14 @@ The procedure decided every existing field: included. 2. **Derivations several artifacts expose** stay: `Return`, `Others`, `Errors`, `Auth`, `Bindings`, `Stream` and `LRO`; and `Flattened`, `Required`, `Undeclared`, `Unflattened`, `Union`, `Discrim`, - `Scalar`, `Shapes`, `Alias` and `Reach`. `Required`, `Undeclared` and `Unflattened` are read - along the flattening walk ([TypePlan](types.md)), `Alias` classifies a decorator over several of - its fields and resolves its chain ([decorators](../refine/decorators.md)), and `Reach` walks - every operation ([output units](../refine/output-units.md)); SDKs and docs that disagreed on any - of them would mark a member optional, name a field's type or place a type differently. + `Inclusive`, `Scalar`, `Shapes`, `Alias` and `Reach`. `Required`, `Undeclared` and `Unflattened` + are read along the flattening walk ([TypePlan](types.md)), `Inclusive` compares every pair of a + union's variants ([TypePlan](types.md#variants-that-can-share-a-value)), `Alias` classifies a + decorator over several of its fields and resolves its chain + ([decorators](../refine/decorators.md)), and `Reach` walks every operation ([output + units](../refine/output-units.md)); SDKs and docs that disagreed on any of them would mark a + member optional, name a field's type, hold one variant where another holds several, or place a + type differently. 3. **Copies of what the IR states** are gone: whether there is a body, one-way-ness and idempotency ([OpPlan](operations.md)), a model's `Base`, `Implements` and `Mixins`, an enum's openness, value type, flags and fallback member, a union's variant list, and the operation-group tree diff --git a/docs/emitters/plan/types.md b/docs/emitters/plan/types.md index 1a272abdb..0e5ec427f 100644 --- a/docs/emitters/plan/types.md +++ b/docs/emitters/plan/types.md @@ -79,16 +79,17 @@ every other shape. What admitting a value means is in admitting no value in common, by these rules, and a pair none of them separates can share one: - **Kinds.** Their JSON kinds have none in common. A variant's kinds are its type's: object for a - model or a map, array for a list or a tuple, its wire kind for a primitive, its values' for an - enum or a literal, its variants' for a union, its root's for a scalar, its base's for a model that - only stands over one, and every kind for `any`. A nullable reference adds `null`, except a - variant's own, whose `null` the union holds - ([unions](../refine/unions.md#a-null-a-variant-admits)), and an integer is a number too. + model or a map, array for a list, a tuple or a `Positional` model, its wire kind for a primitive, + its values' for an enum or a literal, its variants' for a union, its root's for a scalar, its + base's for a model that only stands over one, and every kind for `any`. A nullable reference adds + `null`, and an integer is a number too. A variant's own `null` is the union's + ([unions](../refine/unions.md#a-null-a-variant-admits)), so the rules leave it out, and a variant + admitting only `null` takes no part. - **Values.** Each admits finitely many values, as a closed enum, a literal or a union of them does, and no value is in both, compared as JSON values. -- **Members.** Both are models, read through `Flattened`, and one requires a member the other either - does not declare and is closed, or declares at a position disjoint from it by these rules. A pair - already being compared counts as sharing, so a cycle of models ends the walk. +- **Members.** Both are models laid out by name, read through `Flattened`, and one requires a member + the other either does not declare and is closed, or declares at a position disjoint from it by + these rules. A pair already being compared counts as sharing, so a cycle of models ends the walk. The rules are sound, not complete. A pair they leave sharing may still admit no common value, as a `date` and a `uuid` string do, and is then inclusive: it holds every value correctly, though more diff --git a/docs/emitters/refine/steps.md b/docs/emitters/refine/steps.md index 7850552d9..a6c79a7f3 100644 --- a/docs/emitters/refine/steps.md +++ b/docs/emitters/refine/steps.md @@ -8,7 +8,7 @@ | `LowerLiterals` | `Literal{Value}` | typed constant / single-value type (incl. `symbol`) | [containers](../../ir/containers.md) | | `LowerModels` | `TypePlan.Flattened`, `Required`, `Undeclared`, `Unflattened`, `Base`/`Implements`/`Mixins`, `AdditionalProps`, `Additional`, `Positional` | a declaration over the flattened properties, an `Undeclared` member a required field typed as the catch-all types it, the catch-all holding the undeclared members, unknown members rejected on `closed`, an `Unflattened` conjunct lowered or degraded with a code | [models](../../ir/models.md) | | `ProjectVisibilityShapes` | `plan.ModelShape` per lifecycle | one declaration per shape, such as `User` / `UserCreate` / `UserPatch` | [properties](../../ir/properties.md#visibility) | -| `LowerOptionality` | `TypePlan.Required` (a member's wire presence, [TypePlan](../plan/types.md)) ⊥ `TypeRef.Nullable`, a reference to a union with a nullable variant counting as nullable ([unions](unions.md#a-null-a-variant-admits)) ⊥ `Property.Presence`; `Facts.Component` | the four+ states in the target's forms; a reference on a cycle the target holds by value broken as its type system requires ([recursive references](models.md#recursive-references)) | [properties](../../ir/properties.md); INV8 | +| `LowerOptionality` | `TypePlan.Required` (a member's wire presence, [TypePlan](../plan/types.md)) ⊥ `TypeRef.Nullable`, a reference to a union with a variant admitting `null` counting as nullable ([unions](unions.md#a-null-a-variant-admits)) ⊥ `Property.Presence`; `Facts.Component` | the four+ states in the target's forms; a reference on a cycle the target holds by value broken as its type system requires ([recursive references](models.md#recursive-references)) | [properties](../../ir/properties.md); INV8 | | `LowerConstraints` | `Constraints` on properties/scalars/lists/maps | field-level validation metadata + a refine-built validation method structure | [constraints](../../ir/constraints.md) | | `LowerUnions` | `Union{Variants, Exclusive, WireTagged, Discriminator}`, a polymorphic base's `Discriminator` or `Abstract` with `Facts.Subtypes`, `TypePlan.Union`, `TypePlan.Open`, `TypePlan.Inclusive` | the target's closed sum, one member per variant or subtype + a fallback on an open one; an inclusive union one optional member per variant, each set when its variant admits the value ([unions](unions.md#a-value-several-variants-admit)) | [unions](../../ir/unions.md); INV2 | | `LowerDiscriminator` | `Discriminator{Property/PropertyName/Index, Mapping→TypeID, Default, Envelope}` | a tag-dispatched decoder + typed factories | [models](../../ir/models.md)–[unions](../../ir/unions.md) | diff --git a/docs/emitters/refine/unions.md b/docs/emitters/refine/unions.md index 572c88b71..48dc15ee7 100644 --- a/docs/emitters/refine/unions.md +++ b/docs/emitters/refine/unions.md @@ -49,12 +49,14 @@ seam buys and the fused-pipeline generator cannot have. ## A null a variant admits -A variant whose reference is `Nullable` admits `null`, and `null` holds nothing a variant could keep -apart, so the union holds it itself. Every reference to a union with a nullable variant lowers as a -nullable reference does ([optionality](models.md#optionality--nullability--presence-inv8)), and a -`null` never reaches a variant. The golden spec's single-result envelope declares `result` as -`anyOf: [{type: object, nullable: true}, {type: string, nullable: true}]`: an object, a string or -`null`, the last held by the reference. +A variant admits `null` when its reference is `Nullable` or its type is the `null` literal. `null` +holds nothing a variant could keep apart, so the union holds it itself: every reference to a union +with such a variant lowers as a nullable reference does +([optionality](models.md#optionality--nullability--presence-inv8)), a `null` never reaches a +variant, and a variant admitting nothing else lowers to no member. Both spellings the compiler can +give a null member (#618) then lower alike. The golden spec's single-result envelope declares +`result` as `anyOf: [{type: object, nullable: true}, {type: string, nullable: true}]`: an object, a +string or `null`, the last held by the reference. ## A value several variants admit diff --git a/docs/emitters/targets/go/operations.md b/docs/emitters/targets/go/operations.md index a2e0d6916..8db473b3a 100644 --- a/docs/emitters/targets/go/operations.md +++ b/docs/emitters/targets/go/operations.md @@ -29,8 +29,9 @@ operation has no input ([request shaping](../../refine/requests.md)). `Stream[T] a caller that stops early must do. What `T` is follows from `Recv`: - an `Element` is what its type lowers to. A union's is its struct - ([`Payment`](unions.md#payment-in-go)), and `Recv` returns `io.EOF` after a variant whose - `Variant.Event.Terminal` is true, so an event union needs nothing of its own beyond that. + ([`Payment`](unions.md#payment-in-go)), and `Recv` returns `io.EOF` after an element holding a + variant whose `Variant.Event.Terminal` is true, so an event union needs nothing of its own beyond + that. - no `Element` is `Stream[string]`, each frame's text, unread. - a `bytes` side has no frames and no `Stream`: `Recv` lowers as a file does, to `*RawBody` ([return shapes](returns.md)), and `Send` to an `io.Reader` field of the request type. diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index 3b035c68b..82f8ae826 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -147,7 +147,8 @@ throughout these documents. write at different values is an encode error, and a union with neither set refuses to encode. An `anyOf` the plan proves disjoint, by JSON kind, by enum values and by a member one model requires and the other cannot hold, lowers as a closed sum and decodes each value into the one - variant admitting it, and a `null` two nullable variants admit is held by the reference, as a - nullable one's is. A value no variant admits takes the fallback on an open union and is a coded - error on a closed one ([unions](../refine/unions.md#a-value-several-variants-admit), + variant admitting it. A `null` that two nullable variants admit, or a `null` literal variant + does, is held by the reference, as a nullable one's is. A value no variant admits takes the + fallback on an open union and is a coded error on a closed one + ([unions](../refine/unions.md#a-value-several-variants-admit), [TypePlan](../plan/types.md#variants-that-can-share-a-value)). From 56ef9543f6058c8c6edb8f99df5d33e46091ae72 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:11:20 +0300 Subject: [PATCH 03/14] docs(emitters): make plan.Policy data and define its envelopes plan.Policy held a function, PrimaryResponse, and Plan.Version hashed the policy. A function has no stable bytes: json/v2 refuses it, its printed form is an address that changes every run, and skipping it gives two policies that pick different responses one version. The policy is now data from a closed vocabulary: PrimaryResponse is an ordered list of rules and DefaultFails says whether an unpicked Default is the catch-all failure. Version hashes the document's digest without sources, a new plan.Revision bumped with every change to Build's output, and the policy's deterministic JSON. EnvelopeDetect named a type nothing defined. plan.Policy.Envelopes now lists envelopes a configuration names, none by default. Each matches a body by the bases it reaches or by the members it declares, and says where the value sits and which flag value means the call failed. A matched success returns that member, keeps the envelope reachable through WithResponse, and turns a 2xx whose flag says it failed into a failure with no fault or kind, decoded as the operation's matching error body. Failures and pages are read whole. T-30 pins the plan's identity and T-31 the envelope reads. --- docs/emitters/docs-and-mock.md | 8 +- docs/emitters/operations/errors.md | 6 +- docs/emitters/operations/returns.md | 8 ++ docs/emitters/plan/overview.md | 25 +++++- docs/emitters/plan/policy.md | 101 ++++++++++++++++++----- docs/emitters/plan/responses.md | 49 +++++++++-- docs/emitters/refine/steps.md | 1 + docs/emitters/targets/go/returns.md | 3 + docs/emitters/targets/kotlin/returns.md | 2 + docs/emitters/testing/mandatory-tests.md | 13 +++ 10 files changed, 186 insertions(+), 30 deletions(-) diff --git a/docs/emitters/docs-and-mock.md b/docs/emitters/docs-and-mock.md index 96ac56ce7..7faa17b47 100644 --- a/docs/emitters/docs-and-mock.md +++ b/docs/emitters/docs-and-mock.md @@ -12,10 +12,12 @@ plan once: docs and mocks agree with the SDK by construction, not by re-derivati *same* IR-ID lookup the Go emitter uses for doc comments, so a link follows what its target renders as. Examples use the schema arm (`Value/Headers`) vs the operation arm (`Input/Output/Error`) correctly. A union `TypePlan.Open` marks is documented as one a response - may extend, and every `PlannedError` and `PlannedResponse` as the body and headers it declares. No - runtime `Policy` is consulted — proving the IR/policy split pays off. + may extend, and every `PlannedError` and `PlannedResponse` as the body and headers it declares. An + enveloped response is documented as the `Type` a method returns and the `Envelope.Body` the wire + carries. No runtime `Policy` is consulted — proving the IR/policy split pays off. - **Mock-server emitter (`emitters/mock`).** Reuses `PrimaryResponse`, the content `Return` elects - and the IR `Examples` to synthesize canned responses, `Others` to answer as each declared response + and the IR `Examples` to synthesize canned responses, sending the whole `Envelope.Body` where a + response is enveloped, `Others` to answer as each declared response does, the error taxonomy to simulate declared failures, `TypePlan.Open` to say whether a variant the document does not list may be sent, the `BindingView` to route requests, and the visibility projections to honor the same wire shapes an SDK sends/receives. Refine lowers into a diff --git a/docs/emitters/operations/errors.md b/docs/emitters/operations/errors.md index 9f0305299..30b49d687 100644 --- a/docs/emitters/operations/errors.md +++ b/docs/emitters/operations/errors.md @@ -70,7 +70,11 @@ answers](../plan/responses.md#which-entry-answers)): - a `value` body decodes into `Body`, whatever `Type`'s kind, and a `void` or `raw` one leaves it nil, with `RawBody` holding whatever arrived; - a body that does not decode never hides the failure: `Body` stays nil and `BodyErr` holds the - decode error, since the status is what the caller must learn. + decode error, since the status is what the caller must learn; +- a failure an envelope reports in-band, a 2xx whose flag holds the failing value + ([envelopes](../plan/responses.md#envelopes)), takes no entry: it has no `Fault` and no kind, + since its status states neither, its `Body` is what `EnvelopeRead.Failure` decodes, and it is + retried only as its status would be, which no default retries. A body is buffered up to `Policy.Errors`' bound and no further, so an endless one cannot exhaust the caller; `RawBody` holds what was read, and a body cut there is one that does not decode. A union diff --git a/docs/emitters/operations/returns.md b/docs/emitters/operations/returns.md index 35634a209..a340cdec3 100644 --- a/docs/emitters/operations/returns.md +++ b/docs/emitters/operations/returns.md @@ -63,6 +63,14 @@ spelled as the target spells it ([Go](../targets/go/returns.md), A zero value would not do here: Kotlin's `Rule` has none, a pager has no value to zero, and a nil `*Rule` beside a nil error reads as a created rule until the caller's next line dereferences it. +- **An envelope** (`GET /zone` → `{success, errors, messages, result: Zone}`, under an envelope a + configuration names). **plan.** `Return.Type` is `Zone`, and `Return.Envelope` says it sits at + `result` and that `success: false` is a failure ([envelopes](../plan/responses.md#envelopes)). + **refine.** The method returns `Zone`, and a body whose flag says it failed is a failure, never a + `Zone` ([failures](errors.md)). The envelope's other members are not lost: an enveloped operation, + a paginated one aside, has a `WithResponse` method whatever its headers, and its `Response` holds + the envelope whole in `Envelope`, as `EnvelopeRead.Body` lowers, beside `Body`. On a response with + no envelope, `Envelope` is the target's empty value. - **A status the document does not list, and a body that does not decode** (`GET /zone` → `201`, or a `200` holding a proxy's HTML). **plan.** No field: [the plan](../plan/responses.md#which-entry-answers) says which entry answers. **refine.** The response diff --git a/docs/emitters/plan/overview.md b/docs/emitters/plan/overview.md index e9ff55e13..46da6a7d0 100644 --- a/docs/emitters/plan/overview.md +++ b/docs/emitters/plan/overview.md @@ -15,8 +15,11 @@ func Build(doc *ir.Document, facts *irfacts.Index, pp Policy) (*Plan, []ir.Diagn type Plan struct { Ops map[ir.OpID]OpPlan // one entry per operation Types map[ir.TypeID]TypePlan // one entry per registry type - Version string // hash(doc content-hash + PlanPolicy); feeds the manifest + Version string // the plan's identity (below), which the plan cache + // and the generation manifest key on } + +const Revision = 1 // bumped with every change to what Build returns ``` Every key is an IR stable ID — never `"METHOD /path"`, never a rendered name (INV3; the oagen @@ -37,3 +40,23 @@ derivation reads a type's contents, such as the property a page's items are foun them through `TypePlan.Alias` ([decorators](../refine/decorators.md)), and every consumer lowers a reference it reads through `Alias`, the plan's as the IR's, so no field holds a reference already lowered beside others that are not. + +## The plan's version + +`Version` is a SHA-256 hash of three things: the document's digest, `Revision`, and the policy's +encoding as `encoding/json/v2` writes it under `json.Deterministic(true)`. The digest is SHA-256 +over the document's canonical JSON (`MarshalJSONTo`) with `sources` left out, which `Build` never +reads, so a spec reformatted into the same IR keeps its plan's version. Two builds therefore share +a `Version` exactly when they share a document, a policy and the code that built them, and the plan +cache and the generation manifest ([write / integrate](../write-integrate.md)) can trust it. + +- **The policy is data** ([plan policy](policy.md#the-policy-is-data)), so its encoding is what it + says. A policy holding a function would encode to nothing stable: json/v2 refuses the field, its + printed form is an address that changes every run, and leaving it out gives two policies that pick + different responses one version. +- **`Revision` stands for the code.** A change to `Build` that changes its output for any document + bumps it, and the plan's golden corpus holds the bump: a golden that changes at an unmoved + `Revision` fails, as a shape change does at an unmoved `IRVersion` + ([IR version](../../ir/ir-version.md)). Without it, a fix to the plan would leave every cached + plan current. + diff --git a/docs/emitters/plan/policy.md b/docs/emitters/plan/policy.md index 8cbbdfd0a..5d076719e 100644 --- a/docs/emitters/plan/policy.md +++ b/docs/emitters/plan/policy.md @@ -5,35 +5,53 @@ Everything the plan *infers* rather than reads is a knob here; every inferred ou ```go type Policy struct { - ContentRanking []string // default: json > form > multipart > binary - PrimaryResponse func(*ir.Operation) *ir.Response // default: lowest 2xx, else the - // unconditional one, else the Default - EnvelopeDetect EnvelopePolicy // off unless explicitly enabled - Framings map[string]Framing // media type or +suffix → how it frames a stream (streams.md) - FrameMembers bool // default on: an SSE schema naming only frame - // fields describes the frame (streams.md) - OpenUnions bool // default on: a union a client reads keeps a - // variant it does not know (below) + ContentRanking []string // default: json > form > multipart > binary + PrimaryResponse []PrimaryRule // default: lowest_2xx, unconditional, default; the first + // rule an operation meets picks its primary (below) + DefaultFails bool // default on: a Default no rule takes is the catch-all + // of Errors (below) + Envelopes []Envelope // default none: what unwraps a success body (below) + Framings map[string]Framing // media type or +suffix → how it frames a stream + // (streams.md) + FrameMembers bool // default on: an SSE schema naming only frame + // fields describes the frame (streams.md) + OpenUnions bool // default on: a union a client reads keeps a + // variant it does not know (below) } + +type PrimaryRule string // "lowest_2xx" | "unconditional" | "default" ``` A test asserts that disabling a heuristic removes the corresponding `Inferred` plan field — the guard against a heuristic silently defaulting on and quietly making the plan non-neutral. +## The policy is data + +Every field is a value from a closed vocabulary the plan package defines, and none is a function. +So a policy encodes, compares and hashes as what it says, and `Plan.Version` hashes it +([the plan](overview.md#the-plans-version)). A function has no bytes to hash: its printed form is +an address that changes every run, and skipping it gives two policies that pick different responses +one version. A heuristic the vocabulary lacks is added to it, in this package and at a new +`plan.Revision`, so a change of behaviour always changes the version. A strategy registered by name +and version from outside was weighed and refused: changing its code without its version would be a +silent cache hit, which is the failure this rule removes. So was a caller-supplied ID hashed in the +policy's place, which fails the same way when the caller forgets it. + ## The default response is classed here The IR states an operation's `Default` — OpenAPI's `default` — as neither a success nor a failure, since it answers every status the others do not name ([responses](../../ir/responses.md)). -`PrimaryResponse` reads the whole operation for that reason, and its default takes the lowest 2xx -`Response`, an exact status before a range that starts there, else one with no condition, which is -an RPC's one response, or the `Default` where the operation declares neither, since a `default` -answering every status is then the operation's only description of a success. A `Default` that is -not the primary response is the catch-all of `Errors`, with its condition. Either use is marked -`Inferred`, and a policy may take it as neither or both. Disabling the heuristic leaves an operation -with no success response `void` ([body shapes](responses.md#body-shapes)) and its `Default` in no -plan field. `Streaming` summarizes only the streams an operation's success responses declare, so no -`StreamDetail` covers a `Default`, and a primary one streams only where a content of it declares an -`Item` ([streams](streams.md)). +`PrimaryResponse` reads the whole operation for that reason, rule by rule, and the first rule the +operation meets picks: `lowest_2xx` the lowest 2xx `Response`, an exact status before a range that +starts there; `unconditional` one with no condition, which is an RPC's one response; and `default` +the `Default`, which is the only description of a success an operation declaring neither has. A +`Default` that is not the primary response is the catch-all of `Errors`, with its condition, under +`DefaultFails`. Either use is marked `Inferred`, and a policy may take it as neither or both. +Leaving out the `default` rule and `DefaultFails` leaves an operation with no success response +`void` ([body shapes](responses.md#body-shapes)) and its `Default` in no plan field. `Streaming` +summarizes only the streams an operation's success responses declare, so no `StreamDetail` covers a +`Default`, and a primary one streams only where a content of it declares an `Item` +([streams](streams.md)). ## A union a client reads is open @@ -48,6 +66,51 @@ has no field saying a union is open or closed (#664); once it does, a declared a plan records the choice so that docs and a mock agree with the SDK ([docs and mocks](../docs-and-mock.md)). +## Envelopes + +An envelope wraps an API's success bodies in members of its own, such as a failure flag, messages +and page information, beside the one member a caller wants. An SDK that returns the envelope makes +every call site dig. One that unwraps it has to know where the value and the failure signal live, +so a configuration names its envelopes, and the default names none. + +```go +type Envelope struct { + Name string // recorded in each Inferred mark this envelope sets + Bases []ir.TypeID // matches a body whose model reaches one of these through Base or Mixins + Members []string // matches a body whose model declares every one of these wire names + Value []string // the wire-name path to what the reader gets in the body's place + Flag []string // the path to the member saying whether the call succeeded; nil = none + Fails *ir.Value // the Flag value meaning the call failed; set exactly when Flag is + Except []ir.OpID // operations whose bodies this envelope never matches +} +``` + +A body matches an `Envelope` when its type, read through `TypePlan.Alias`, is a model that reaches +one of `Bases` through `Base` or `Mixins`, or declares every name in `Members`, read through +`Flattened`. Either way it must also declare the members `Value` and `Flag` name. A body takes the +first envelope it matches, and none of them on an operation its `Except` lists. +[Responses](responses.md#envelopes) says how a matched body is read. Each field the match sets is +marked `Inferred` with the envelope's `Name`. An `Except` or `Bases` entry naming nothing in the +document is reported, as is a `Flag` without `Fails`. + +A body is matched by its members as well as by its composition, because composition alone misses +real envelopes. On the golden spec (spec @319dc756, morphic @b206dc99), the primary bodies of 2,665 +of 3,763 operations declare `success`, `errors`, `messages` and `result`. 2,238 of them reach a +named component declaring the `success` flag through `Base` or `Mixins`, 113 distinct components in +all, since products declare their own; the other 427 declare it inline, and no `Bases` list reaches +them. One `Members` entry matches all 2,665, and since it names every member, a body that merely has +a `result` is never unwrapped by accident: + +```go +plan.Envelope{ + Name: "result", + Members: []string{"success", "errors", "messages", "result"}, + Value: []string{"result"}, + Flag: []string{"success"}, + Fails: &ir.Value{Kind: ir.ValueBool, Bool: false}, +} +``` + ## Framings Framings state what a media type means, not a guess about an API, so the table marks nothing diff --git a/docs/emitters/plan/responses.md b/docs/emitters/plan/responses.md index bd9d3a2fa..5ec046dcf 100644 --- a/docs/emitters/plan/responses.md +++ b/docs/emitters/plan/responses.md @@ -7,10 +7,23 @@ declares, read by one `ResponsePlan`. // ResponsePlan is how one response reaches its reader. A success and a failure are read by the // same rules, so the election, the shapes, the codec and the headers have one home. type ResponsePlan struct { - Shape BodyShape // value | void | raw | stream - Content *ir.Content // the content elected from the payload; nil = nothing to read - Type *ir.TypeRef // what a value body decodes into, of any kind; nil unless Shape is value - Headers []ir.Header // the headers the response declares, each whole (below) + Shape BodyShape // value | void | raw | stream + Content *ir.Content // the content elected from the payload; nil = nothing to read + Type *ir.TypeRef // what a value body gives its reader, of any kind; nil unless + // Shape is value + Envelope *EnvelopeRead // how an enveloped body is read; nil = the body decodes into + // Type (below) + Headers []ir.Header // the headers the response declares, each whole (below) +} + +// EnvelopeRead is a body one of plan.Policy.Envelopes matched (policy.md), as it is read here. +type EnvelopeRead struct { + Name string // the Envelope that matched + Body ir.TypeRef // what the body decodes into: the elected content's type + Value *ir.PropPath // where Type sits inside Body; nil on a pager's page, read whole + Flag *ir.PropPath // the member saying whether the call succeeded; nil = none + Fails *ir.Value // the Flag value meaning it failed + Failure *ir.TypeRef // what a failing body decodes into; nil = the target's untyped JSON } // BodyShape is how a body reaches its reader, a small closed enum: @@ -68,12 +81,36 @@ in a second channel. The first rule that applies decides: plan](overview.md), [decorators](../refine/decorators.md)), so an anonymous `allOf` decorator around a model returns the model. Its `Nullable` is the content's, and a nullable return takes the target's nullable form ([optionality](../refine/models.md#optionality--nullability--presence-inv8)). -Unwrapping an envelope, which `plan.Policy` may enable ([plan policy](policy.md)), would return a -value found inside the content instead; what it records and where is #695's. Which codec reads a +An envelope, which `plan.Policy` may name ([plan policy](policy.md#envelopes)), gives the reader a +value found inside the content instead, as [envelopes](#envelopes) says. Which codec reads a media type is the target's, and a media type it has none for is a coded degrade of that target's that reads the body as `raw`. Every body of every document takes one of the four, since the rules end in a default; [Return shapes](../operations/returns.md) traces each through the first targets. +## Envelopes + +A `value` body that one of `plan.Policy.Envelopes` matches ([plan policy](policy.md#envelopes)) is +read through it, and `Envelope` records how, marked `Inferred` with the envelope's name: + +- **The value.** The body decodes into `Envelope.Body`, and the reader gets the member at `Value`: + `Type` is that member's type, so `Return.Type` is the `Zone` inside `{success, errors, messages, + result: Zone}`. The rest of the envelope stays reachable through the response + ([return shapes](../operations/returns.md)). A paginated operation's page is read whole, as any + page is: `Value` is nil, `Type` stays the page, and its items are where `Pagination.Items` says. +- **A failure in-band.** Where `Flag` is set, the decoder reads that member before decoding the + body, as a discriminator's tag is read ([unions](../refine/unions.md#discriminators)). A body + whose flag holds `Fails` is a failure despite its 2xx status, and decodes into `Failure`: the + `Type` of the first entry of `Errors` whose body the same envelope matches, or the target's + untyped JSON where none does. It is never decoded as `Body`, since an API whose success envelope + declares `success: true` alone sends a failure no success type admits + ([failures](../operations/errors.md#every-failure-has-one-place)). On the golden spec, 2,321 of + the 2,665 enveloped operations declare an error case the envelope matches, 2,225 of them only + one such type. +- **Only a success.** A failure is read whole, so an entry of `Errors` has no `Envelope`, and a + body that does not match is read as it would be with no envelope configured. + +An entry of `Others` is read by the same rules, so `AsReturn` compares the types its reader gets. + ## Headers `Headers` are what the response declares, each whole, so its name, its type and the `Style` and diff --git a/docs/emitters/refine/steps.md b/docs/emitters/refine/steps.md index de5824618..c54ac8cc6 100644 --- a/docs/emitters/refine/steps.md +++ b/docs/emitters/refine/steps.md @@ -21,6 +21,7 @@ | `LowerPagination` | `plan.PagePlan` (item unwrap, cursor paths) | an auto-pager; pacing left to policy | [interaction-patterns](../../ir/interaction-patterns.md) | | `LowerStreaming` | `plan.StreamPlan` (direction; each side's content, framing and element) | a method per shape the operation offers; a stream over an element of any kind; a `bytes` side lowered as a file; terminal ends | [interaction-patterns](../../ir/interaction-patterns.md) | | `LowerLRO` | `plan.LROPlan` (final-state-via, poll/final ops, result path) | poller type + one poll and a poll until done, which returns the final type | [interaction-patterns](../../ir/interaction-patterns.md) | +| `UnwrapEnvelopes` | `ResponsePlan.Envelope` | in the response decoder, the flag read before the body, a failing body the runtime's failure value decoded as `Failure`, and the member at `Value` returned in the body's place; the envelope whole in a `WithResponse` method's `Response` ([envelopes](../plan/responses.md#envelopes), [return shapes](../operations/returns.md)) | [responses](../../ir/responses.md) | | `LowerErrors` | `plan.PlannedError` | no error type: each entry's conditions, body decoder and declared facts in the operation's response decoder, which fills the runtime's failure value ([failures](../operations/errors.md)) | [responses](../../ir/responses.md) | | `LowerOtherResponses` | `OpPlan.Others` | each entry's conditions in the operation's response decoder: `AsReturn` and a 2xx no entry answers read as `Return`, no body a nullable return or an ended pager or stream, any other a body decoder filling the runtime's `OtherResponse`, as a body that does not decode does ([return shapes](../operations/returns.md)) | [responses](../../ir/responses.md) | | `PlaceDeclarations` | `Provenance.Source`, `TypePlan.Reach`, the target's placement options | the output unit each declaration lands in: one package unless a partition or a group split moves it ([output units](output-units.md)) | [provenance](../../ir/provenance.md) | diff --git a/docs/emitters/targets/go/returns.md b/docs/emitters/targets/go/returns.md index eae2b1909..50265c0b8 100644 --- a/docs/emitters/targets/go/returns.md +++ b/docs/emitters/targets/go/returns.md @@ -12,6 +12,9 @@ How each [return shape](../../operations/returns.md) lowers in Go. `func (c *Client) GetZoneWithResponse(ctx) (*Response[*Zone, GetZoneHeaders], error)`. The `Response`'s `Body` is `struct{}` for `void`, and its `Headers` is `struct{}` where they declare none. +- **An envelope.** `GetZone` returns `*Zone`, and `GetZoneWithResponse`'s `Response` holds the + envelope in `Envelope any`, a `*GetZoneResponse` there and nil on an operation with no envelope, + as `OtherResponse.Body` holds a body of any kind. - **Another declared response.** With no body, `DeleteMethod` returns a nil `*Deleted` and a nil error, and a stream's `Recv` returns `io.EOF`. diff --git a/docs/emitters/targets/kotlin/returns.md b/docs/emitters/targets/kotlin/returns.md index b4b713c80..1bf6e129c 100644 --- a/docs/emitters/targets/kotlin/returns.md +++ b/docs/emitters/targets/kotlin/returns.md @@ -8,6 +8,8 @@ How each [return shape](../../operations/returns.md) lowers in Kotlin. - **No body.** `Unit`. - **Declared headers.** `suspend fun getZoneWithResponse(): Response`. The `Response`'s `Body` is `Unit` for `void`, and its `Headers` is `Unit` where they declare none. +- **An envelope.** `suspend fun getZone(): Zone`, and `getZoneWithResponse()`'s `Response` holds + the envelope in `val envelope: Any?`, null on an operation with no envelope. - **Another declared response.** With no body, `suspend fun deleteMethod(): Deleted?`. ## `OtherResponse` diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index 817580617..f7795d939 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -142,3 +142,16 @@ throughout these documents. open. An option beside it is taken in priority order, an option ANDing it with a declared scheme needs both, and the service's credentials are never sent in its place ([auth](../operations/auth.md#unresolved-schemes)). +30. **T-30 · Plan identity** — one document under one policy builds plans of one `Version` across + processes and runs, and the policy round-trips through JSON unchanged; changing any policy + field, the `Revision`, or the document outside `sources` changes the `Version`, and changing + only `sources` does not. A plan golden that changes at an unmoved `Revision` fails + ([the plan's version](../plan/overview.md#the-plans-version)). +31. **T-31 · Envelopes** — under an envelope matched by `Members` and one matched by `Bases`, on + every target, a method returns the member at `Value`, its `WithResponse` method holds the + envelope whole, and a 2xx whose flag holds `Fails` is a failure with no fault or kind whose body + decodes as `Failure`, even where the success type admits only `true`. A paginated operation's + page is read whole, its flag still read. A body naming only some of the members, an operation + `Except` lists and a failure body are read as they would be with no envelope, and the default + policy unwraps nothing ([envelopes](../plan/responses.md#envelopes), + [plan policy](../plan/policy.md#envelopes)). From 6a3acaa9e078be019a016b077bad1e4be6a15513 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:27:08 +0300 Subject: [PATCH 04/14] docs(emitters): layer runtime policy and read embedded policy Runtime policy was one tree per project, so an operation needing its own timeout or retry had nowhere to say so, and a document's own SDK configuration, kept verbatim as vendor extensions, would have been parsed by each emitter its own way. Policy and hints are now layers, each holding a document-wide override and overrides by operation: what a document embeds, the project's, the emitter's. A setting takes the highest layer that sets it, an operation's value over its layer's document-wide one, and a declared IR fact beats every layer. A new emitters/embed reader turns vendor extensions a configuration maps into the lowest layer of each, decoding each value strictly and reporting what it cannot read. The vocabulary gains connection-error retries, Retry-After, a throttling backoff and redirects. Whether an operation is safe or idempotent is a plan fact, OpPlan.Effects: declared, else from its HTTP method. An enum a client reads keeps a value it does not list, as a union does, and a oneOf value two variants admit decodes as plan.Policy.ExclusiveMatch says. A service hint makes a parameter a client setting; nothing is hoisted by inference, since a shared parameter stops being shared as an API grows, and the plan only reports candidates. --- docs/architecture.md | 10 +- docs/emitters/README.md | 3 +- docs/emitters/contract/embedded-policy.md | 80 ++++++++++++++++ docs/emitters/contract/emitter.md | 7 +- docs/emitters/contract/runtime-policy.md | 102 ++++++++++++++++++--- docs/emitters/contract/shaping-hints.md | 20 ++++ docs/emitters/docs-and-mock.md | 27 +++--- docs/emitters/operations/client.md | 29 ++++++ docs/emitters/operations/errors.md | 6 +- docs/emitters/packages.md | 3 +- docs/emitters/plan/operations.md | 18 +++- docs/emitters/plan/plan-or-refine.md | 21 +++-- docs/emitters/plan/policy.md | 38 ++++++++ docs/emitters/plan/types.md | 5 +- docs/emitters/refine/enums.md | 6 ++ docs/emitters/refine/requests.md | 6 +- docs/emitters/refine/steps.md | 4 +- docs/emitters/refine/unions.md | 7 ++ docs/emitters/targets/go/lowering.md | 2 +- docs/emitters/targets/go/operations.md | 4 +- docs/emitters/targets/go/types.md | 5 +- docs/emitters/targets/kotlin/lowering.md | 2 +- docs/emitters/targets/kotlin/operations.md | 3 +- docs/emitters/testing/mandatory-tests.md | 32 ++++++- 24 files changed, 378 insertions(+), 62 deletions(-) create mode 100644 docs/emitters/contract/embedded-policy.md diff --git a/docs/architecture.md b/docs/architecture.md index e01f8e258..0d48a32ca 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -442,10 +442,12 @@ logical exception-kind and fault, doc-URL template), **telemetry** (request- ID and client-telemetry headers), **logging** (a closed lifecycle-event list), **user-agent** construction (identifier template, app-info enrichment), **idempotency injection** (header name, auto-generate rules), **pagination pacing** (auto-page delay), and **request guards** (option -keys that must not appear as params — misuse detection). Delivery model: canonical defaults + -deep-partial per-emitters/per-project overrides. Precedence rule: *declared IR facts win over -policy defaults* — `ErrorCase.Retryable/Fault` and `Operation.Idempotency` come from the spec; -policy fills in where the spec is silent, never the reverse. +keys that must not appear as params — misuse detection), widened by connection-error retries, +`Retry-After` and **redirects**. Delivery model: canonical defaults + deep-partial layers, the +policy a document embeds below the project's and the project's below each emitter's, each with +per-operation overrides (`docs/emitters/contract/runtime-policy.md`). Precedence rule: *declared +IR facts win over policy defaults* — `ErrorCase.Retryable/Fault` and `Operation.Idempotency` come +from the spec; policy fills in where the spec is silent, never the reverse. ### 2.5 Changing the IR diff --git a/docs/emitters/README.md b/docs/emitters/README.md index af00e3d61..257c84afa 100644 --- a/docs/emitters/README.md +++ b/docs/emitters/README.md @@ -42,7 +42,8 @@ its own, and answers every construct of the [lowering table](refine/lowering-tab - [Target options](contract/options.md): strict, versioned, each default belonging to a version. - [The type table](contract/type-table.md): which target type each IR leaf type becomes. - [Runtime policy](contract/runtime-policy.md) and [shaping hints](contract/shaping-hints.md): the - two inputs that are not the IR. + two inputs that are not the IR, and [the policy a document embeds](contract/embedded-policy.md), + read into each as its lowest layer. **The plan: language-neutral decisions, computed once** diff --git a/docs/emitters/contract/embedded-policy.md b/docs/emitters/contract/embedded-policy.md new file mode 100644 index 000000000..64eb21752 --- /dev/null +++ b/docs/emitters/contract/embedded-policy.md @@ -0,0 +1,80 @@ +# Policy and hints a document embeds + +A document often carries its SDK's configuration as vendor extensions: retries and timeouts at the +document and on an operation, the statuses that count as failures, method and parameter names, the +sub-client an operation belongs to, an operation exposed under a second name, a parameter every +call sends. The compiler keeps each verbatim, an `Unmodeled` entry of reason `vendor_extension` +([unmodeled](../../ir/unmodeled.md)), and left there every emitter would parse them its own way. So +one reader, `emitters/embed`, turns them into the two inputs they configure, [runtime +policy](runtime-policy.md) and [shaping hints](shaping-hints.md), each a layer below the project's. + +```go +package embed + +type Mapping struct { + Keys map[string]Target // an extension's key as Unmodeled holds it, such as + // "openapi:x-sdk-retries", → what it configures + Disabled bool // reads nothing +} + +type Target struct { + Tree string // "policy" | "hint" + Path []string // the member path inside that tree's JSON form, + // such as ["retry", "maxAttempts"] or ["rename"] +} + +type Layer struct { + Policy policy.Layer // the lowest layer of runtime policy + Hints map[emitters.EntityID]emitters.ShapeHint // the lowest layer of hints +} + +func Read(doc *ir.Document, m Mapping) (Layer, []ir.Diagnostic) +``` + +The engine calls `Read` once per document, before it resolves the layers, so an emitter sees only +the resolved policy and hints and never an extension (INV1). + +## What a key configures + +A key configures the node holding it. A `policy` key on the document configures every operation, +and one on an operation configures that operation, `Layer.Policy.Ops` keyed by its `OpID` +([layers](runtime-policy.md#layers-and-operations)). A `hint` key configures the node's own ID, so +a key on a parameter renames that parameter, on an operation that method, and on a service that +client. A key a mapping names on any other node, such as a `policy` key on a property, configures +nothing and is reported. + +The value is decoded into the field `Path` names, as the tree's own JSON form reads it, refusing an +unknown member. A value of the wrong shape is reported as a warning at the extension and never +coerced, since the document means something by the key that the field does not hold. The entry +stays in `Unmodeled` either way, as a promotion leaves it +([promotion](../../ir/unmodeled.md#promotion)), so a consumer that disagrees with the mapping reads +the entry instead. + +## The mapping is a configuration + +The reader runs by default, and its default mapping is empty. The compiler's promotion policy ships +keys whose reading is a convention independent of any generator, such as `x-deprecated-reason` +([`extension-promotion`](../../compilers/openapi.md#extension-promotion)). No key for a retry, a +timeout or an SDK method name has one: each is a generator's own spelling, and its value's shape is +that generator's. So a configuration names the keys its documents use and what each one sets, and a +key it does not name stays where it was. + +A mapping reads a key as `Unmodeled` holds it, the format's prefix included, and only entries of +reason `vendor_extension`, whose key is the source's own spelling. That is the stability #429 asks +the IR to state for such keys. A key under any other reason, such as `no_ir_home`, names a gap the +IR may close, and a mapping naming one is refused. + +## Precedence + +The embedded layer sits below the project's and the emitter's, for policy +([layers](runtime-policy.md#layers-and-operations)) and for hints alike: what a document's author +embeds is a default the SDK's author can override, and the override is reported as an info +diagnostic, so the document's author can see it was overruled. A declared IR fact still beats every +layer ([precedence](runtime-policy.md#precedence--declared-ir-facts-win-hard-rule)). + +## Measured on the golden spec + +The golden spec (spec @319dc756, morphic @b206dc99) carries hints nearly everywhere: a method name +and a group name on 3,761 of its 3,763 operations, a name on 118 parameters and on 30 properties, +each under a generator's own key. It carries no retry or timeout key. A mapping reading those keys +gives each method the name its document chose, which an emitter would otherwise derive. diff --git a/docs/emitters/contract/emitter.md b/docs/emitters/contract/emitter.md index 8dccc2035..e0db201b5 100644 --- a/docs/emitters/contract/emitter.md +++ b/docs/emitters/contract/emitter.md @@ -27,8 +27,11 @@ type EmitterInput struct { Doc *ir.Document // the ABI — the ONLY structural input (INV1) Facts *irfacts.Index // what Doc implies: subtypes, usage, recursion (docs/ir/derived-facts.md), indexed ONCE by the engine, shared with the plan (below) Plan *plan.Plan // language-neutral decisions, computed ONCE by the engine (../plan/overview.md) - Policy policy.Policy // runtime/SDK behavior — a SEPARATE tree, never in the IR (INV10; runtime-policy.md) - Hints map[EntityID]ShapeHint// SDK-shaping hints keyed by IR stable ID (shaping-hints.md) + Policy policy.Resolved // runtime/SDK behavior per operation, its layers resolved — + // a SEPARATE tree, never in the IR (INV10; runtime-policy.md, + // embedded-policy.md) + Hints map[EntityID]ShapeHint// SDK-shaping hints keyed by IR stable ID, their layers resolved + // (shaping-hints.md) Scope *Scope // nil = whole document; else "emit only these" — full Doc still // consumed so global decisions stay byte-identical Prior *PriorState // prior manifest + prior surface for write/integrate; nil first run diff --git a/docs/emitters/contract/runtime-policy.md b/docs/emitters/contract/runtime-policy.md index 75e3a9b6e..d4b615fa6 100644 --- a/docs/emitters/contract/runtime-policy.md +++ b/docs/emitters/contract/runtime-policy.md @@ -7,13 +7,15 @@ SDKs, docs, and mock servers without dragging SDK opinions along. ## The canonical vocabulary -From oagen's production `SdkBehavior`, the best real-world enumeration (architecture.md §2.4): +From oagen's production `SdkBehavior`, the best real-world enumeration (architecture.md §2.4), +widened by the behaviors every SDK has to choose and a caller has to be able to set: ```go package policy type Policy struct { - Retry Retry // retryable status codes, max attempts, Backoff{initial, multiplier, max, jitter} + Retry Retry // when and how a request is sent again (below) + Redirects Redirects // how a redirect is followed (below) Timeout Timeout // default seconds + env override Errors ErrorTax // status→logical exception-kind and fault, doc-URL template, the bound // on a buffered body (../operations/errors.md, ../operations/returns.md) @@ -25,24 +27,102 @@ type Policy struct { Guards RequestGuard // option keys that must NOT appear as params (misuse detection) } +type Retry struct { + Statuses []int // statuses retried where the IR states nothing; default 408, 429, + // 500, 502, 503, 504 + MaxAttempts int // default 3, the first attempt included + Backoff Backoff // {Initial, Multiplier, Max, Jitter} + Throttled Backoff // the backoff after a failure marked throttling + // (../operations/errors.md) + Connection string // a request that got no response: "never" | "idempotent" (default) | + // "always" + RetryAfter bool // default on: wait what a Retry-After header asks, up to Backoff.Max +} + +type Redirects struct { + Max int // default 10; 0 follows none, returning the 3xx as a response +} + func Defaults() Policy func Merge(base Policy, over DeepPartial) Policy // scalars override; ARRAYS REPLACE (the oagen rule) ``` +- **A request that got no response** may have reached the server and acted, so by default it is + sent again only where the operation is idempotent, as `OpPlan.Effects` says + ([OpPlan](../plan/operations.md)). A status answers that question, so `Statuses` apply whatever + the operation does. An idempotency key the runtime generates (`Idempotency`) makes an operation + idempotent only where the IR declares its token, since only then is the server known to honor + one. +- **`Retry-After`** is the server saying when to come back. Honoring it is the default, and it never + waits past `Backoff.Max`, so a server cannot park a caller indefinitely; a value past it ends the + retries and returns the response. +- **A redirect** is followed as RFC 9110 §15.4 says: 307 and 308 send the method and body again, + 301, 302 and 303 send a `GET`. A redirect to another origin never carries the client's + credentials, which no setting changes, since a server cannot vouch for one it is not. + +What a response may hold that the document does not list is not runtime policy, because it changes a +type or a decoder rather than the boilerplate around them, and docs and mocks have to agree with it. +A union a client reads keeps an unknown variant and an enum an unknown value under `plan.Policy`, +and a `oneOf` two variants admit decodes as `plan.Policy` says ([plan policy](../plan/policy.md)). +Unknown members follow each model's `Additional`, a declared fact, so no setting widens a model the +spec closes or closes one it leaves open. + ## Delivery -Canonical `Defaults()` + deep-partial per-emitters/per-project overrides. Arrays replace entirely -(not concat) — an explicit, documented rule. Policy is delivered in `EmitterInput.Policy`; refiners -read it via `Ctx.Policy`, and it shapes only the generated *runtime binding* (the retry loop, -transport config, user-agent string, telemetry headers, pagination pacing) behind the +Canonical `Defaults()` + deep-partial overrides. Arrays replace entirely (not concat) — an +explicit, documented rule. Policy is delivered resolved, per operation, in `EmitterInput.Policy`; +refiners read it via `Ctx.Policy`, and it shapes only the generated *runtime binding* (the retry +loop, transport config, user-agent string, telemetry headers, pagination pacing) behind the `Transport`/`Serializer` abstraction. It never enters the typed structural model ([runtime abstraction](../emit/templates.md#serialization-behind-a-runtime-abstraction)). +## Layers and operations + +An API's operations rarely want one setting each: an export wants a longer timeout, a `POST` no +retry, a rate-limited endpoint its own backoff. So every source of policy is a layer holding a +document-wide override and overrides keyed by operation, merged by the same `Merge` rules: + +```go +type Layer struct { + Document DeepPartial // what the layer sets for every operation + Ops map[ir.OpID]DeepPartial // what it sets for one operation, over its Document +} + +type Resolved struct { + Document Policy // what an operation no layer names is given + Ops map[ir.OpID]Policy // each operation some layer names, resolved whole +} + +func Resolve(layers ...Layer) (Resolved, []ir.Diagnostic) // lowest first: Defaults() is implied +``` + +The layers, lowest first, are the policy a document embeds ([embedded +policy](embedded-policy.md)), the project's, and the emitter's. A setting takes the value of the +highest layer that sets it, for the operation or for the whole document, and within that layer the +operation's value over the document's. So a project that turns retries off turns them off +everywhere, whatever a document embeds for one operation, and a document's own per-operation +setting still beats its own document-wide one. Ordering by scope first would let a document's +per-operation settings outvote a project-wide choice, which then could not be made without +naming every operation. + +A key naming no operation of the document is reported, never ignored, since an operation whose ID +changed would otherwise lose its settings in silence. Where a higher layer replaces what the +document embeds, an info diagnostic names the setting, both values and the operation, so the +document's author can see it was overruled. + ## Precedence — declared IR facts win (hard rule) `ErrorCase.Retryable/Throttling/Fault` and `Operation.Idempotency` come from the spec; policy fills -in only where the spec is silent — never the reverse. The refiner enforces this by reading the IR -fact first and consulting policy only on `nil`, or for a `Fault`, on `""`. A `policy-precedence` -test asserts a spec `Retryable:false` beats a policy that lists that status as retryable -([T-9](../testing/mandatory-tests.md)). When policy is overridden by a spec fact, an -`Inferred`-marked diagnostic is emitted so nothing is silent. +in only where the spec is silent — never the reverse, and at every layer. The refiner enforces +this by reading the IR fact first and consulting policy only on `nil`, or for a `Fault`, on `""`. A +`policy-precedence` test asserts a spec `Retryable:false` beats a policy that lists that status as +retryable, at a document-wide setting and at an operation's ([T-9](../testing/mandatory-tests.md)). +When policy is overridden by a spec fact, an `Inferred`-marked diagnostic is emitted so nothing is +silent. + +## A setting that cannot apply + +A setting the plan or a target cannot honor is reported, never dropped and never quietly replaced +by a stricter behavior: a `Redirects.Max` a target's transport cannot bound, or a per-operation +`Timeout` on a runtime that keeps one per client. The report is a coded diagnostic of the target's, +naming the setting and the operation, and the engine decides whether it is fatal (INV5). diff --git a/docs/emitters/contract/shaping-hints.md b/docs/emitters/contract/shaping-hints.md index 0592e29c1..5f1db0f34 100644 --- a/docs/emitters/contract/shaping-hints.md +++ b/docs/emitters/contract/shaping-hints.md @@ -9,11 +9,20 @@ drops a hint (INV3): type ShapeHint struct { Rename string // presentation rename for the entity's symbol; "" = none Remount *ir.GroupID // move an operation under a different sub-client; nil = keep + AlsoMount []ir.GroupID // expose an operation under these sub-clients too, each + // method a second name for the one its own group places SplitUnionBody bool // flatten a request-body union into typed wrapper methods — OPT-IN, // the one sanctioned union→arguments collapse (../refine/unions.md); default off URLBuilder bool // expose a URL-builder variant of the operation + ClientParams []ParamKey // on a service: the parameters its client holds as settings + // (../operations/client.md#client-settings) Extra map[string]ir.RawValue // free-form, forward-compatible per-target ergonomics } + +type ParamKey struct { // a parameter as every operation names it, never by ID alone + In string // "path" | "query" | "header" | "cookie" + Name string // its wire name +} ``` `Extra` is a bare raw-JSON map rather than `ir.RawConfig`: `RawConfig` names *declared protocol @@ -23,3 +32,14 @@ it is emitter input authored alongside the SDK, and the IR never sees it. Method-name *derivation* is itself an injectable policy marked `Inferred`, disable-able and overridable. Per-target naming/compat overlays are the same shape: an emitter input keyed by IR ID, so one IR document drives different compat baselines per language. + +Hints come in layers, as runtime policy does ([layers](runtime-policy.md#layers-and-operations)): +the hints a document embeds ([embedded policy](embedded-policy.md)), then the project's, then the +emitter's, each field taking the highest layer's value. An ID no entity of the document has is +reported, never ignored. + +`AlsoMount` answers an operation a document exposes under two names, which one `Remount` cannot. +Each sub-client it names gets a method calling the operation, keyed in the manifest under the +operation's ID with the group beside it, as a declaration derived from an entity is ([write / +integrate](../write-integrate.md)), so the surface differ sees one operation reached two ways. + diff --git a/docs/emitters/docs-and-mock.md b/docs/emitters/docs-and-mock.md index 876487d5a..bfee42546 100644 --- a/docs/emitters/docs-and-mock.md +++ b/docs/emitters/docs-and-mock.md @@ -11,21 +11,22 @@ plan once: docs and mocks agree with the SDK by construction, not by re-derivati [`Docs.Refs`](../ir/annotations.md#references-in-prose) marks resolve to intra-doc links via the *same* IR-ID lookup the Go emitter uses for doc comments, so a link follows what its target renders as. Examples use the schema arm (`Value/Headers`) vs the operation arm - (`Input/Output/Error`) correctly. A union `TypePlan.Open` marks is documented as one a response - may extend, one `TypePlan.Inclusive` marks as one whose value may be several variants at once, and - every `PlannedError` and `PlannedResponse` as the body and headers it declares. An enveloped - response is documented as the `Type` a method returns and the `Envelope.Body` the wire carries. No - runtime `Policy` is consulted — proving the IR/policy split pays off. + (`Input/Output/Error`) correctly. A union or enum `TypePlan.Open` marks is documented as one a + response may extend, one `TypePlan.Inclusive` marks as one whose value may be several variants at + once, and every `PlannedError` and `PlannedResponse` as the body and headers it declares. An + enveloped response is documented as the `Type` a method returns and the `Envelope.Body` the wire + carries, and an operation as safe or idempotent where its `Effects` say so. No runtime `Policy` is + consulted — proving the IR/policy split pays off. - **Mock-server emitter (`emitters/mock`).** Reuses `PrimaryResponse`, the content `Return` elects and the IR `Examples` to synthesize canned responses, sending the whole `Envelope.Body` where a - response is enveloped, `Others` to answer as each declared response - does, the error taxonomy to simulate declared failures, `TypePlan.Open` to say whether a variant - the document does not list may be sent, the `BindingView` to route requests, and the visibility - projections to honor the same wire shapes an SDK sends/receives. Refine lowers into a - server-routing AST (routes from `HTTPBinding.URITemplate`, handlers validating against - constraints). Because the same plan feeds SDK and mock, the two are **wire-consistent by - construction** — the mock can *be* the interception target the SDK's wire-conformance harness - ([mandatory tests](testing/mandatory-tests.md)) is diffed against. + response is enveloped, `Others` to answer as each declared response does, the error taxonomy to + simulate declared failures, `TypePlan.Open` to say whether a variant or an enum value the document + does not list may be sent, the `BindingView` to route requests, and the visibility projections to + honor the same wire shapes an SDK sends/receives. Refine lowers into a server-routing AST (routes + from `HTTPBinding.URITemplate`, handlers validating against constraints). Because the same plan + feeds SDK and mock, the two are **wire-consistent by construction** — the mock can *be* the + interception target the SDK's wire-conformance harness ([mandatory + tests](testing/mandatory-tests.md)) is diffed against. - **Validation emitter (future).** Additionally consumes the validation-only constructs kept verbatim in `Unmodeled` (`not`/`if-then-else`/`dependentSchemas`, …) that SDK/docs emitters ignore. It selects them by `UnmodeledEntry.Reason == ir.ReasonValidationOnly` rather than by key diff --git a/docs/emitters/operations/client.md b/docs/emitters/operations/client.md index f4535ce5e..189869b98 100644 --- a/docs/emitters/operations/client.md +++ b/docs/emitters/operations/client.md @@ -40,3 +40,32 @@ operation whose `Servers` names a server of its own. authenticates per server, and a messaging target reads it. - **emit.** The URL template is expanded at request time from the chosen values; the template and each variable's constants are typed AST, the expansion a runtime helper. + +## Client settings + +A parameter an API's calls all send alike, such as an account in the path or an API version +header, is set once on the client when its service's hint names it (`ShapeHint.ClientParams`, +[shaping hints](../contract/shaping-hints.md)). Nothing makes one a setting by inference. Whether +every operation of a scope takes a parameter changes as the API grows, so a setting the plan chose +would come and go with operations, and each change would move the parameter into or out of every +method's signature: a breaking SDK change caused by an additive spec change. + +- **What a setting fills.** A `ParamKey` fills, in every operation, the parameter at its location + and wire name whose declaration lowers to one wire form with the others: the same root through + its scalar chain ([scalars](../refine/scalars.md)), and the same style and explode. Requiredness, + constraints and docs may differ, and each operation checks its own. A declaration of another wire + form stays a plain input, reported once by refine, naming the operation, since a shared name does + not make two parameters one. +- **How a call reads it.** The client takes the setting when it is built, a call may give its own, + and a call with neither for a required parameter is refused unsent, the error naming it. A + parameter a setting fills is not positional but an optional field of the operation's request type + ([request shaping](../refine/requests.md)), so a call can still name another account. +- **Candidates.** The plan reports, as an info `emitters/plan/client-param-candidate`, each + `ParamKey` that every operation of some group with more than one operation takes in one wire + form, with the number of operations in the document taking it so, so an SDK's author can name + it. On the golden spec (spec @319dc756, morphic @b206dc99), `account_id` in the path is taken by + 2,660 of 3,763 operations, each a string under one of 16 sets of constraints, and `zone_id` by + 698. One shared component parameter was weighed as the test instead and finds a parameter every + operation of a group shares in 52 of the 515 groups with more than one operation, against 465 by + location and name: the spec declares its parameters inline. + diff --git a/docs/emitters/operations/errors.md b/docs/emitters/operations/errors.md index 30b49d687..59e5ab368 100644 --- a/docs/emitters/operations/errors.md +++ b/docs/emitters/operations/errors.md @@ -24,9 +24,9 @@ spells it ([Go](../targets/go/errors.md#ratelimited-in-go), response decoder fills from the entry a response takes. The retry wiring reads `Retryable`/`Throttling` **from the IR fact** and falls back to `Policy.Retry` only where the spec is silent ([precedence](../contract/runtime-policy.md#precedence--declared-ir-facts-win-hard-rule)); -throttling selects the throttling backoff class. The error types are runtime boilerplate ([structure -and boilerplate](../emit/templates.md)): their kinds come from `Policy.Errors`, so a template -renders them and no spec changes them. +throttling selects the throttling backoff, `Retry.Throttled`. The error types are runtime +boilerplate ([structure and boilerplate](../emit/templates.md)): their kinds come from +`Policy.Errors`, so a template renders them and no spec changes them. ## The status tells failures apart diff --git a/docs/emitters/packages.md b/docs/emitters/packages.md index f4f70406b..d35e5630b 100644 --- a/docs/emitters/packages.md +++ b/docs/emitters/packages.md @@ -8,7 +8,8 @@ emitters/ ├── contract.go # Emitter, EmitterInput/Output, Artifact, Capabilities, TargetKey, registry ├── plan/ # SHARED, language-neutral — imports ir, ir/irfacts. Build(), Plan, OpPlan, TypePlan, ModelShape │ └── plantest/ # golden snapshots of Plan (language-independent corpus) -├── policy/ # runtime SDK-policy vocabulary + Defaults() + Merge(DeepPartial) (INV10) +├── policy/ # runtime SDK-policy vocabulary + Defaults() + Merge(DeepPartial) + Resolve(layers) (INV10) +├── embed/ # reads the policy and hints a document embeds as vendor extensions (contract/embedded-policy.md) ├── naming/ # SHARED naming engine: acronym/compound tables, the character-name table, │ # reserved-word set type, ID-keyed collision resolver. Casing FUNCTIONS are │ # per-target (naming.Policy impl) diff --git a/docs/emitters/plan/operations.md b/docs/emitters/plan/operations.md index b5b52f8ea..2292e8984 100644 --- a/docs/emitters/plan/operations.md +++ b/docs/emitters/plan/operations.md @@ -15,6 +15,8 @@ type OpPlan struct { Stream *StreamPlan // nil = neither direction streams (docs/ir/interaction-patterns.md) LRO *LROPlan // nil = not long-running Auth []AuthOption // OR-of-ANDs, priority-ordered (../operations/auth.md) + Effects Effects // what calling it may change: declared, else by method + // (policy.md) Bindings BindingView // per-protocol param/request/response view from OpBindings Inferred map[string]ir.Provenance // which fields above were heuristic (INV6) } @@ -40,6 +42,12 @@ type LROPlan struct { // maps ir.LongRunning (docs/ir/interactio ResultPath *ir.PropPath // where the final result lives in the terminal response } +type Effects struct { + Safe bool // asks for no change on the server (RFC 9110 §9.2.1) + Idempotent bool // a second call changes nothing the first did not + // (RFC 9110 §9.2.2); true of every safe operation +} + type AuthOption struct { // one option in the OR-of-ANDs; slice order is PRIORITY order Schemes []ir.SchemeUse // ALL must be satisfied together; empty = "no auth is acceptable" } @@ -55,10 +63,12 @@ lowered late. The content ranking (`json > form > multipart > binary`) is a shar helper (resolves [open question 3](../../ir/open-questions.md)). Some facts oagen's plan carries are not here, by the [plan-or-refine test](plan-or-refine.md). -Whether there is a body, whether the operation is one-way and its idempotency are stated by the IR, -so a refiner reads them there: `Operation.Request`, `Operation.OneWay` and `Operation.Idempotency`, -the last of which also names the idempotency-token parameter. And whether an operation's inputs are -passed positionally or inside an options object differs by language, so it is each target's +Whether there is a body and whether the operation is one-way are stated by the IR, so a refiner +reads them there, `Operation.Request` and `Operation.OneWay`, as it reads the idempotency-token +parameter `Operation.Idempotency` names. Whether the operation is safe or idempotent is `Effects`, +since the IR states it for few operations and a method states it for most ([plan +policy](policy.md#an-operations-effects-come-from-its-method)). And whether an operation's inputs +are passed positionally or inside an options object differs by language, so it is each target's ([request shaping](../refine/requests.md)); the neutral facts it is decided from are already here — `BindingView` says where each input binds, and `Parameter.Required` and `Payload.Required` which are required — so moving the decision out added no field. diff --git a/docs/emitters/plan/plan-or-refine.md b/docs/emitters/plan/plan-or-refine.md index a34e12438..11ab7211e 100644 --- a/docs/emitters/plan/plan-or-refine.md +++ b/docs/emitters/plan/plan-or-refine.md @@ -31,7 +31,8 @@ of the artifact that uses it — how an untagged union's variant is recognized o The procedure decided every existing field: 1. **Choices under `plan.Policy`** stay: `PrimaryResponse`, `RequestContent` and the content each - `ResponsePlan` elects, the unwrap behind `Return.Type`, `Open`, and `PagePlan`, an inferred one + `ResponsePlan` elects, the unwrap behind `Return.Type`, `Open` on a union and on an enum, + `Effects`, which a method sets where the IR states nothing, and `PagePlan`, an inferred one included. 2. **Derivations several artifacts expose** stay: `Return`, `Others`, `Errors`, `Auth`, `Bindings`, `Stream` and `LRO`; and `Flattened`, `Required`, `Undeclared`, `Unflattened`, `Union`, `Discrim`, @@ -43,15 +44,15 @@ The procedure decided every existing field: units](../refine/output-units.md)); SDKs and docs that disagreed on any of them would mark a member optional, name a field's type, hold one variant where another holds several, or place a type differently. -3. **Copies of what the IR states** are gone: whether there is a body, one-way-ness and idempotency - ([OpPlan](operations.md)), a model's `Base`, `Implements` and `Mixins`, an enum's openness, value - type, flags and fallback member, a union's variant list, and the operation-group tree - ([BindingView](bindings.md)). `Union` replaces a classification that copied the variant list and - a second discriminator beside the tag mode it derived, and `Scalar` replaces a bare root that - left the constraints it merged nowhere. `NeedShapes` is gone too: it listed again the lifecycles - `Shapes` is keyed by ([ModelShape](types.md#modelshape--visibility-projection)). A client's - servers are stated by the IR as well, so they get no field ([the - client](../operations/client.md)). +3. **Copies of what the IR states** are gone: whether there is a body, one-way-ness and the + idempotency token ([OpPlan](operations.md)), a model's `Base`, `Implements` and `Mixins`, an + enum's declared openness, value type, flags and fallback member, a union's variant list, and the + operation-group tree ([BindingView](bindings.md)). `Union` replaces a classification that copied + the variant list and a second discriminator beside the tag mode it derived, and `Scalar` replaces + a bare root that left the constraints it merged nowhere. `NeedShapes` is gone too: it listed + again the lifecycles `Shapes` is keyed by + ([ModelShape](types.md#modelshape--visibility-projection)). A client's servers are stated by the + IR as well, so they get no field ([the client](../operations/client.md)). 4. **The parameter-passing shape** fails the first test and moved to refine ([request shaping](../refine/requests.md)): a dynamic target's keyword arguments are additive as they stand, where a positional parameter added to a Go or Kotlin function breaks its callers, so the diff --git a/docs/emitters/plan/policy.md b/docs/emitters/plan/policy.md index 5d076719e..a6554cedf 100644 --- a/docs/emitters/plan/policy.md +++ b/docs/emitters/plan/policy.md @@ -17,6 +17,12 @@ type Policy struct { // fields describes the frame (streams.md) OpenUnions bool // default on: a union a client reads keeps a // variant it does not know (below) + OpenEnums bool // default on: an enum a client reads keeps a value + // it does not list (below) + ExclusiveMatch string // "first" (default) | "only": what a oneOf value two + // variants admit decodes to (../refine/unions.md) + MethodSemantics bool // default on: an operation stating no effects takes + // its method's (below) } type PrimaryRule string // "lowest_2xx" | "unconditional" | "default" @@ -66,6 +72,38 @@ has no field saying a union is open or closed (#664); once it does, a declared a plan records the choice so that docs and a mock agree with the SDK ([docs and mocks](../docs-and-mock.md)). +## An enum a client reads is open + +The reason unions are open holds for an enum: an API that adds a member must not break every SDK +already deployed on the first response carrying it. So under `OpenEnums` an enum is `Open` wherever +`Facts.Usage` marks it `Output`, marked `Inferred`, and a value it does not list is kept rather +than refused ([enums](../refine/enums.md)). An enum only ever written stays as the IR says, and +turning the policy off leaves every enum as the IR says. + +This does not override the declared `Enum.Closed`, which states that the members the document +lists are all it admits. The SDK honours that in what it sends: its constants name the listed +members alone. `Closed` says nothing of what a later version of the API will list, and that is what +a client reading a response has to survive. A document declaring an enum open, through a promoted +`x-extensible-enum` among others, makes it open in both directions. On the golden spec (spec +@319dc756, morphic @b206dc99) all 8,314 enums are closed, since an OpenAPI `enum` is a closed value +set, so without the policy each one a response carries would fail to decode the first time the API +added a member. + +## An operation's effects come from its method + +`OpPlan.Effects` ([OpPlan](operations.md)) says whether calling an operation is safe and whether it +is idempotent. A declared `Operation.Idempotency` decides, its token kind counting as idempotent +since a retry sends the token it first sent. Where the IR states none, `MethodSemantics` takes them +from the binding's method as RFC 9110 §9.2 defines them, marked `Inferred`: `GET`, `HEAD`, +`OPTIONS` and `TRACE` are safe, and those with `PUT` and `DELETE` are idempotent. `POST` and +`PATCH` state neither, and an operation no HTTP binding carries takes neither. + +A method says nothing of whether an operation destroys something: `DELETE` and `PUT` are +idempotent and may be destructive, so nothing derives a confirmation from `Effects`. On the golden +spec no operation declares its idempotency, and of its 3,763 operations 1,841 use a safe method and +2,693 an idempotent one. Without the policy no operation would be known idempotent, so no request +that got no response would ever be sent again. + ## Envelopes An envelope wraps an API's success bodies in members of its own, such as a failure flag, messages diff --git a/docs/emitters/plan/types.md b/docs/emitters/plan/types.md index 0e5ec427f..c927aac5a 100644 --- a/docs/emitters/plan/types.md +++ b/docs/emitters/plan/types.md @@ -12,8 +12,9 @@ type TypePlan struct { // cannot hold: each lowered beside the properties, or degraded Union UnionShape // a union's wire shape, one of four (../refine/unions.md); "" = not a union Discrim *DiscrimFacts // a union's or hierarchy's discriminator, as variant IDs - Open bool // a union or polymorphic base keeps a value no variant - // recognizes rather than refusing it (policy.md, ../refine/unions.md) + Open bool // a union, polymorphic base or enum keeps a value no variant or + // member recognizes rather than refusing it (policy.md, + // ../refine/unions.md, ../refine/enums.md) Inclusive bool // an any_of union two of whose variants can admit one value, // holding every variant a value admits (below); false otherwise Scalar *ScalarChain // a scalar's Base chain, walked once; nil = not a scalar diff --git a/docs/emitters/refine/enums.md b/docs/emitters/refine/enums.md index 446e2bf1f..481f270ca 100644 --- a/docs/emitters/refine/enums.md +++ b/docs/emitters/refine/enums.md @@ -7,3 +7,9 @@ a closed and a flags enum in forms of its own ([Go](../targets/go/types.md#enums the canonical name for serialization. The open/closed bit, value type, member values, and flags bit all survived from the IR to this point — Kiota's string-only-closed-enum mistake is designed out because the fact reached refine. + +An enum `TypePlan.Open` marks ([plan policy](../plan/policy.md#an-enum-a-client-reads-is-open)) +lowers to the target's open form whatever `Closed` says, so a response carrying a member a later +version added decodes and keeps its value. Its constants still name only the members the document +lists, which is all `Closed` asks of what the SDK sends. + diff --git a/docs/emitters/refine/requests.md b/docs/emitters/refine/requests.md index d63fa586b..4a9e6fb8c 100644 --- a/docs/emitters/refine/requests.md +++ b/docs/emitters/refine/requests.md @@ -13,8 +13,10 @@ options](../contract/options.md)). The three first targets share one default: pa positional, in the order the primary binding's URI template names them (`HTTPBinding.URITemplate`), never in the order `Operation.Params` holds them, which follows the source's declarations rather than the URL; and every other input — query, header and cookie parameters and the body — is a field -of one request type for the operation, which it takes when it has any such input. Each target -declares that type its own way ([Go](../targets/go/operations.md#the-request-type), +of one request type for the operation, which it takes when it has any such input. A parameter a +client setting fills ([client settings](../operations/client.md#client-settings)) is an optional +field of the request type, whatever its location, since a call may leave it to the client. Each +target declares that type its own way ([Go](../targets/go/operations.md#the-request-type), [Kotlin](../targets/kotlin/operations.md#the-request-type)). Two properties decide the rule: 1. **Only the wire may fix an order.** A path template orders its parameters and a parameter list diff --git a/docs/emitters/refine/steps.md b/docs/emitters/refine/steps.md index 75acaa4b4..4369a1fc6 100644 --- a/docs/emitters/refine/steps.md +++ b/docs/emitters/refine/steps.md @@ -12,11 +12,11 @@ | `LowerConstraints` | `Constraints` on properties/scalars/lists/maps | field-level validation metadata + a refine-built validation method structure | [constraints](../../ir/constraints.md) | | `LowerUnions` | `Union{Variants, Exclusive, WireTagged, Discriminator}`, a polymorphic base's `Discriminator` or `Abstract` with `Facts.Subtypes`, `TypePlan.Union`, `TypePlan.Open`, `TypePlan.Inclusive` | the target's closed sum, one member per variant or subtype + a fallback on an open one; an inclusive union one optional member per variant, each set when its variant admits the value ([unions](unions.md#a-value-several-variants-admit)) | [unions](../../ir/unions.md); INV2 | | `LowerDiscriminator` | `Discriminator{Property/PropertyName/Index, Mapping→TypeID, Default, Envelope}` | a tag-dispatched decoder + typed factories | [models](../../ir/models.md)–[unions](../../ir/unions.md) | -| `LowerOpenEnums` | `Enum.Closed`, `ValueType`, `Members`, `Flags`, `FallbackMember` | the target's closed, open and flags forms; an open one keeps an unknown value | [enums](../../ir/enums.md) | +| `LowerOpenEnums` | `Enum.Closed`, `TypePlan.Open`, `ValueType`, `Members`, `Flags`, `FallbackMember` | the target's closed, open and flags forms; an open one keeps an unknown value | [enums](../../ir/enums.md) | | `ExtractInterfaces` | `Implements`, `OperationGroup` | interfaces / client interfaces for testability | [models](../../ir/models.md) | | `MapExternalTypes` | `External{Identity, Package, MinVersion}`, `Ctx.Types` | the type table's row for the identity ([type table](../contract/type-table.md)): imported library type + dependency-manifest entry; no row → opaque strategy + diagnostic | [containers](../../ir/containers.md) | | `MaterializeValues` | `Value{Kind, Num BigVal, Ref, Ctor}` | literals; `Num` → bignum/decimal, never float64; `Ctor` → call; `symbol` → string explicitly; a default documented, never written onto a request ([values](values.md)) | [values](../../ir/values.md) | -| `BuildRequestShaping` | `plan.BindingView`, `RequestContent`, the target's parameter shape ([request shaping](requests.md)) | request builders, per-protocol param binding, each operation's request type | [bindings](../../ir/bindings.md) | +| `BuildRequestShaping` | `plan.BindingView`, `RequestContent`, the target's parameter shape ([request shaping](requests.md)), the service's `ShapeHint.ClientParams` ([client settings](../operations/client.md#client-settings)) | request builders, per-protocol param binding, each operation's request type | [bindings](../../ir/bindings.md) | | `LowerHeaders` | `ResponsePlan.Headers` | a type with a field per declared header, each optional, read from the raw headers by wire name; a `WithResponse` method on a success ([return shapes](../operations/returns.md)), the failure value's and an `OtherResponse`'s `Headers` ([failures](../operations/errors.md), [return shapes](../operations/returns.md)) | [responses](../../ir/responses.md) | | `LowerPagination` | `plan.PagePlan` (item unwrap, cursor paths) | an auto-pager; pacing left to policy | [interaction-patterns](../../ir/interaction-patterns.md) | | `LowerStreaming` | `plan.StreamPlan` (direction; each side's content, framing and element) | a method per shape the operation offers; a stream over an element of any kind; a `bytes` side lowered as a file; terminal ends | [interaction-patterns](../../ir/interaction-patterns.md) | diff --git a/docs/emitters/refine/unions.md b/docs/emitters/refine/unions.md index 48dc15ee7..a19f18d21 100644 --- a/docs/emitters/refine/unions.md +++ b/docs/emitters/refine/unions.md @@ -58,6 +58,13 @@ give a null member (#618) then lower alike. The golden spec's single-result enve `result` as `anyOf: [{type: object, nullable: true}, {type: string, nullable: true}]`: an object, a string or `null`, the last held by the reference. +A `oneOf` admits exactly one variant, so a value two of its variants admit breaks the document. What +it decodes to is `plan.Policy.ExclusiveMatch`'s ([plan policy](../plan/policy.md)): under `first`, +the default, the variant recognition finds first, as a lenient reader of a document whose variants +overlap needs; under `only`, a coded decode error, found by trying every variant whose kinds hold +the value, as an inclusive union does below. A `oneOf` whose variants the plan proves disjoint +([TypePlan](../plan/types.md#variants-that-can-share-a-value)) never meets the choice. + ## A value several variants admit An `anyOf` lets one value satisfy several variants, and what the union holds then turns on diff --git a/docs/emitters/targets/go/lowering.md b/docs/emitters/targets/go/lowering.md index d31982970..e1e05ad9d 100644 --- a/docs/emitters/targets/go/lowering.md +++ b/docs/emitters/targets/go/lowering.md @@ -18,7 +18,7 @@ Go's normative lowering of each construct of the [lowering table](../../refine/l | `Model` whose `Base` is no model, not a decorator | with no property, catch-all or discriminator of its own, lowered as the `Scalar` row lowers an alias over that base, its constraints, docs and examples binding the position ([scalars](../../../ir/scalars.md)); a union base beside properties distributed over the variants as [models](../../../ir/models.md#composition-beside-a-union) distributes a union co-declared with a composition; anything else a diagnosed degrade | | `AdditionalProps` / `Additional` | catch-all `map` as the embedded fallback; `closed`→`RejectUnknownMembers(true)`, pinned per model ([`User`](models.md#user-in-go)) | | `Union` (four wire shapes, [unions](../../refine/unions.md#the-wire-shapes)) | a struct holding a sealed interface + marker + a wrapper per variant + a tag-dispatch (de)serializer on the struct; `Open` → an `Unknown` fallback; an inclusive `anyOf` → a struct with a pointer per variant ([unions](unions.md#an-inclusive-union)) | -| `Enum` open/closed/flags | consts named by enum then member (`StatusActive`, [enums](types.md#enums)): closed→typed consts; open→string+consts+`Unknown`; flags→bitset; `FallbackMember` on decode | +| `Enum` open/closed/flags | consts named by enum then member (`StatusActive`, [enums](types.md#enums)): closed→typed consts; open, or `TypePlan.Open`→string+consts+`Unknown`; flags→bitset; `FallbackMember` on decode | | `List`/`MapT`/`Tuple` | `[]T`/`map[K]V`/positional struct (optional past `MinItems`, `Rest`→trailing slice); `List.Encoding`→serializer mode | | `Literal` | typed constant (incl. `symbol`) | | `External` | the type-table row for its `Identity` ([type table](../../contract/type-table.md)): imported library type + dep-manifest; no row → opaque handle, passthrough + diagnostic | diff --git a/docs/emitters/targets/go/operations.md b/docs/emitters/targets/go/operations.md index 8db473b3a..11db844ad 100644 --- a/docs/emitters/targets/go/operations.md +++ b/docs/emitters/targets/go/operations.md @@ -61,4 +61,6 @@ operations](../../operations/long-running.md). ## The client The client's settings ([the client](../../operations/client.md)) are functional options — -`NewClient(WithRegion(RegionEU))`, `WithServer(…)`, `WithBaseURL(u)`. +`NewClient(WithRegion(RegionEU))`, `WithServer(…)`, `WithBaseURL(u)`. A [client +setting](../../operations/client.md#client-settings) is one more, `WithAccountID("…")`, and the +request type's field for it a pointer, `AccountID *string`, nil taking the client's. diff --git a/docs/emitters/targets/go/types.md b/docs/emitters/targets/go/types.md index cb1274437..87b263f35 100644 --- a/docs/emitters/targets/go/types.md +++ b/docs/emitters/targets/go/types.md @@ -31,8 +31,9 @@ for one. ## Enums Go cannot express open enums, so `Closed=false` lowers to a named string type + typed consts + a -passthrough that preserves unknown values and honors `FallbackMember` on decode; `Closed=true` -lowers to typed consts with strict decode; `Flags=true` → bitset. A `ValueType` whose Go type has +passthrough that preserves unknown values and honors `FallbackMember` on decode, as does an enum +`TypePlan.Open` marks ([enums](../../refine/enums.md)); `Closed=true` lowers to typed consts with +strict decode; `Flags=true` → bitset. A `ValueType` whose Go type has no constant form, such as `date`, `uuid` or `decimal`, has no Go lowering yet (#1202). A Go constant is declared at package level, where every enum's constants meet, so Go names a diff --git a/docs/emitters/targets/kotlin/lowering.md b/docs/emitters/targets/kotlin/lowering.md index be46aac3d..d485c140a 100644 --- a/docs/emitters/targets/kotlin/lowering.md +++ b/docs/emitters/targets/kotlin/lowering.md @@ -20,7 +20,7 @@ where Multiplatform differs from the JVM ([Multiplatform common code](multiplatf | `Model` whose `Base` is no model, not a decorator | with no property, catch-all or discriminator of its own, lowered as the `Scalar` row lowers an alias over that base, its constraints, docs and examples binding the position ([scalars](../../../ir/scalars.md)); a union base beside properties distributed over the variants as [models](../../../ir/models.md#composition-beside-a-union) distributes a union co-declared with a composition; anything else a diagnosed degrade | | `AdditionalProps` / `Additional` | catch-all `Map` filled by a serializer that reads the whole object ([unknown members](models.md#unknown-members)); strict `Json`, `@JsonIgnoreUnknownKeys` on each other open model | | `Union` (four wire shapes, [unions](../../refine/unions.md#the-wire-shapes)) | sealed interface + one wrapper class per variant + a `KSerializer` per wire shape; `Open` → an `Unknown` fallback; KMP: not exhaustive in Swift; an inclusive `anyOf` → a class with a nullable property per variant ([unions](unions.md#an-inclusive-union)) | -| `Enum` open/closed/flags | closed → `enum class` with `@SerialName` members, `FallbackMember` on decode; open → value class over the value type with a constant per member, which keeps an unknown value; flags → value class over an integer | +| `Enum` open/closed/flags | closed → `enum class` with `@SerialName` members, `FallbackMember` on decode; open, or `TypePlan.Open` → value class over the value type with a constant per member, which keeps an unknown value; flags → value class over an integer | | `List`/`MapT`/`Tuple` | `List`/`Map`/a class with a positional serializer (nullable past `MinItems`, `Rest` → trailing `List`) | | `Literal` | `const val`, or an `object` for a non-primitive value | | `External` | the type-table row for its `Identity`, each platform's own; no row → diagnostic | diff --git a/docs/emitters/targets/kotlin/operations.md b/docs/emitters/targets/kotlin/operations.md index dbe06129f..1dd3c9107 100644 --- a/docs/emitters/targets/kotlin/operations.md +++ b/docs/emitters/targets/kotlin/operations.md @@ -15,4 +15,5 @@ Kotlin's request type ([request shaping](../../refine/requests.md)) is a class b ## The client The client's settings ([the client](../../operations/client.md)) are setters on the client's -builder. +builder. A [client setting](../../operations/client.md#client-settings) is one more, +`accountId("…")`, and the request type's property for it is nullable, null taking the client's. diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index c9bd52202..6c9829c2a 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -22,7 +22,12 @@ throughout these documents. silent `any` and never a panic, a fact outside M3's scope included ([what M3 lowers](../milestones.md#what-m3-lowers)). 8. **T-8 · Manifest ID-keying** — the entity→symbol map survives path/name churn. -9. **T-9 · Policy precedence** — declared `Retryable`/`Fault`/`Idempotency` override policy defaults. +9. **T-9 · Policy precedence** — declared `Retryable`/`Fault`/`Idempotency` override policy defaults + at a document-wide setting and at an operation's, in every layer. A project's document-wide + setting beats a per-operation one the document embeds, an operation's own setting beats its + layer's document-wide one, an operation ID no operation has is reported, and an embedded value a + higher layer replaces draws an info naming both + ([layers](../contract/runtime-policy.md#layers-and-operations)). 10. **T-10 · Step-trace golden** — the ordered refine step list produces a pinned, deterministic trace (guards implicit inter-step coupling). 11. **T-11 · Architecture import-graph** — the layering in [package layout](../packages.md). @@ -165,3 +170,28 @@ throughout these documents. `Except` lists and a failure body are read as they would be with no envelope, and the default policy unwraps nothing ([envelopes](../plan/responses.md#envelopes), [plan policy](../plan/policy.md#envelopes)). +32. **T-32 · Embedded policy and hints** — a mapping reading a key on the document, on an operation + and on a parameter configures the document, that operation and that parameter's hint; a key on + a node it cannot configure, a value of the wrong shape and a mapping naming a key of a reason + other than `vendor_extension` are each reported, the entry staying in `Unmodeled`; the default + mapping reads nothing, and `Disabled` reads nothing however the mapping is written + ([embedded policy](../contract/embedded-policy.md)). +33. **T-33 · Client settings** — on every target, a service hint naming a path parameter makes it + a client setting, which every operation declaring it in that wire form sends unless the call + gives its own; a call with neither is refused unsent; an operation declaring it at another + wire form keeps it as an input and is reported once; no parameter becomes a setting unless a + hint names it, and the plan reports each candidate as an info + ([client settings](../operations/client.md#client-settings)). +34. **T-34 · Effects** — a declared `Idempotency` of each kind beats the method, a `GET` with none + is safe and idempotent, a `PUT` and a `DELETE` idempotent, a `POST` neither, each derived one + marked `Inferred`; turning `MethodSemantics` off leaves every undeclared operation neither; and + on every target a request that got no response is sent again for an idempotent operation and + never for a `POST`, under the default `Retry.Connection` + ([effects](../plan/policy.md#an-operations-effects-come-from-its-method)). +35. **T-35 · Open enums and overlapping oneOfs** — on every target, a closed enum a response carries + decodes a value it does not list and re-encodes it as it came, its constants naming only the + listed members, while one only ever sent stays strict and turning `OpenEnums` off restores + strict decoding; a `oneOf` value two variants admit decodes into the first under `first` and is + a coded error under `only` ([plan policy](../plan/policy.md#an-enum-a-client-reads-is-open), + [unions](../refine/unions.md#variant-identity-and-recognition)). + From 5557ce93d75f2aff95b8a900d17a8444d99079fc Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:30:22 +0300 Subject: [PATCH 05/14] docs(emitters): encode every enum member as its JSON kind An enum whose value type has no Go constant form, a date, a uuid or a decimal, had no Go lowering, and the type table built an enum value by decoding its wire form, which no constant can be. Kotlin's enum class writes every member as its @SerialName string, so a boolean or numeric enum went out as a string its schema does not admit. A member is now written as the JSON kind its value type has, and decoded by comparing as JSON compares values, numbers by value. Go declares an enum over its value type's row, and over the wire text where the row has no constant, the codec writing the value type's JSON kind and a Value method giving the row's type. Kotlin generates a KSerializer for every enum that is not a string enum, and an entry of a mapped type holds its value as that type. T-36 pins every value type on every target. --- docs/emitters/contract/type-table.md | 6 +++-- docs/emitters/refine/enums.md | 13 ++++++++++ docs/emitters/refine/lowering-table.md | 2 +- docs/emitters/targets/go/lowering.md | 4 +-- docs/emitters/targets/go/types.md | 32 ++++++++++++++++++++---- docs/emitters/targets/kotlin/lowering.md | 2 +- docs/emitters/targets/kotlin/types.md | 8 ++++++ docs/emitters/testing/mandatory-tests.md | 7 ++++++ 8 files changed, 63 insertions(+), 11 deletions(-) diff --git a/docs/emitters/contract/type-table.md b/docs/emitters/contract/type-table.md index 4ddee05f8..bc27bef58 100644 --- a/docs/emitters/contract/type-table.md +++ b/docs/emitters/contract/type-table.md @@ -71,8 +71,10 @@ type Dependency struct { that type's literals, checks its constraints and knows whether it has methods. Any other type is known only through its row, and how it codes is the target's ([Go](../targets/go/types.md), [Kotlin](../targets/kotlin/types.md)). A literal of a mapped type (a default, a constant, an enum - value) is built by decoding the value's wire form through its codec, and a constraint on it is not - checked by the SDK, which the server still checks, reported once per key as an info + value) is built by decoding the value's wire form through its codec, an enum declared over the + wire form where the target's constants cannot hold the type + ([enums](../refine/enums.md#a-member-on-the-wire)), and a constraint on it is not checked by the + SDK, which the server still checks, reported once per key as an info `emitters//unchecked-constraint`. A row's codec goes with its type, so a declaration standing over the type keeps it by naming that type again rather than defining a new one ([scalars](../refine/scalars.md)), and a type only a user's row names is taken to have methods. diff --git a/docs/emitters/refine/enums.md b/docs/emitters/refine/enums.md index 481f270ca..abd9cedc1 100644 --- a/docs/emitters/refine/enums.md +++ b/docs/emitters/refine/enums.md @@ -13,3 +13,16 @@ lowers to the target's open form whatever `Closed` says, so a response carrying version added decodes and keeps its value. Its constants still name only the members the document lists, which is all `Closed` asks of what the SDK sends. +## A member on the wire + +A member's wire form is its `WireName` where it has one, else its `Value` as the JSON kind its +enum's `ValueType` has: a string for a `string`, `date` or `uuid` enum, a number for an `integer`, +`number` or `decimal` one, a boolean for a `bool` one. Every target writes that form and decodes by +comparing a value with the members as JSON compares values, which is what an `enum` keyword +compares, whatever `format` says: numbers by value, so `0.1` and `0.10` are one member, and +everything else by its text. A target whose enum form holds only some value types lowers the others +over the wire form ([Go](../targets/go/types.md#enums), [Kotlin](../targets/kotlin/types.md#enums)), +never over a form that would change the JSON kind, as a `bool` enum written as the string `"true"` +would. On the golden spec (spec @319dc756, morphic @b206dc99) 1,069 enums are `bool`, 35 `integer` +and 348 `number`, beside 6,862 `string` ones. + diff --git a/docs/emitters/refine/lowering-table.md b/docs/emitters/refine/lowering-table.md index a00371ac3..21ba2e37c 100644 --- a/docs/emitters/refine/lowering-table.md +++ b/docs/emitters/refine/lowering-table.md @@ -24,7 +24,7 @@ IR bug (INV9), not a compiler change. | `Model` whose `Base` is no model, not a decorator | — | | `AdditionalProps` / `Additional` | pattern channel | | `Union` (four wire shapes, [unions](unions.md)) | `WireName`/`WireID` per variant; an unknown variant's value; every variant an inclusive union's value admits | -| `Enum` open/closed/flags | member `WireName` | +| `Enum` open/closed/flags | member `WireName`; each member's JSON kind, compared by value ([a member on the wire](enums.md#a-member-on-the-wire)) | | `List`/`MapT`/`Tuple` | packed/expanded | | `Literal` | — | | `External` | — | diff --git a/docs/emitters/targets/go/lowering.md b/docs/emitters/targets/go/lowering.md index e1e05ad9d..e77430559 100644 --- a/docs/emitters/targets/go/lowering.md +++ b/docs/emitters/targets/go/lowering.md @@ -18,7 +18,7 @@ Go's normative lowering of each construct of the [lowering table](../../refine/l | `Model` whose `Base` is no model, not a decorator | with no property, catch-all or discriminator of its own, lowered as the `Scalar` row lowers an alias over that base, its constraints, docs and examples binding the position ([scalars](../../../ir/scalars.md)); a union base beside properties distributed over the variants as [models](../../../ir/models.md#composition-beside-a-union) distributes a union co-declared with a composition; anything else a diagnosed degrade | | `AdditionalProps` / `Additional` | catch-all `map` as the embedded fallback; `closed`→`RejectUnknownMembers(true)`, pinned per model ([`User`](models.md#user-in-go)) | | `Union` (four wire shapes, [unions](../../refine/unions.md#the-wire-shapes)) | a struct holding a sealed interface + marker + a wrapper per variant + a tag-dispatch (de)serializer on the struct; `Open` → an `Unknown` fallback; an inclusive `anyOf` → a struct with a pointer per variant ([unions](unions.md#an-inclusive-union)) | -| `Enum` open/closed/flags | consts named by enum then member (`StatusActive`, [enums](types.md#enums)): closed→typed consts; open, or `TypePlan.Open`→string+consts+`Unknown`; flags→bitset; `FallbackMember` on decode | +| `Enum` open/closed/flags | a named type over the value type's row, consts named by enum then member (`StatusActive`, [enums](types.md#enums)): closed→typed consts, strict decode; open, or `TypePlan.Open`→consts+`Unknown`; flags→bitset; `FallbackMember` on decode; a value type with no Go constant → a named `string` over the wire text, its codec writing the value type's JSON kind, + a `Value()` of the row's type | | `List`/`MapT`/`Tuple` | `[]T`/`map[K]V`/positional struct (optional past `MinItems`, `Rest`→trailing slice); `List.Encoding`→serializer mode | | `Literal` | typed constant (incl. `symbol`) | | `External` | the type-table row for its `Identity` ([type table](../../contract/type-table.md)): imported library type + dep-manifest; no row → opaque handle, passthrough + diagnostic | @@ -53,7 +53,7 @@ lowers to in Go. | `LowerConstraints` | field-level `validate` tags + a refine-built `Validate()` method structure | | `LowerUnions` | a struct holding a sealed interface + marker method + a wrapper per variant or subtype + a fallback on an open one; on an inclusive union, a struct with a pointer per variant | | `LowerDiscriminator` | tag-dispatched `UnmarshalJSONFrom` on the union's struct + typed factories | -| `LowerOpenEnums` | closed → typed consts; open → `string`+consts+`Unknown(raw)`; flags → bitset | +| `LowerOpenEnums` | closed → typed consts; open → consts+`Unknown(raw)`; flags → bitset; each over the value type's row, or the wire text where it has no constant | | `ExtractInterfaces` | Go interfaces / client interfaces for testability | | `LowerPagination` | auto-pager yielding `iter.Seq2[T, error]`; pacing left to policy | | `LowerStreaming` | a method per shape the operation offers; `Stream[T]` with `Recv()`/`Send()` over an element of any kind; a `bytes` side lowered as a file; terminal ends | diff --git a/docs/emitters/targets/go/types.md b/docs/emitters/targets/go/types.md index 87b263f35..752d6bdee 100644 --- a/docs/emitters/targets/go/types.md +++ b/docs/emitters/targets/go/types.md @@ -30,11 +30,33 @@ for one. ## Enums -Go cannot express open enums, so `Closed=false` lowers to a named string type + typed consts + a -passthrough that preserves unknown values and honors `FallbackMember` on decode, as does an enum -`TypePlan.Open` marks ([enums](../../refine/enums.md)); `Closed=true` lowers to typed consts with -strict decode; `Flags=true` → bitset. A `ValueType` whose Go type has -no constant form, such as `date`, `uuid` or `decimal`, has no Go lowering yet (#1202). +An enum is a named type over its `ValueType`'s Go row, with a typed constant per member. Go cannot +express open enums, so `Closed=false` lowers to that type + its consts + a passthrough that +preserves unknown values and honors `FallbackMember` on decode, as does an enum `TypePlan.Open` +marks ([enums](../../refine/enums.md)); `Closed=true` lowers to that type + its consts with strict +decode; `Flags=true` → bitset. + +Go declares constants of boolean, numeric and string types alone, so a `ValueType` whose row is none +of them — `date`, `time`, `datetime`, `datetime_offset`, `duration`, `uuid`, `bytes`, `decimal`, and +`integer` or `number` where a row maps them exactly — lowers over the member's wire form ([a member +on the wire](../../refine/enums.md#a-member-on-the-wire)): a named type over `string`, each constant +the member's wire text, whose codec writes that text as the value type's JSON kind, a number for a +`decimal`, and decodes by comparing as JSON does, so the constant of a decimal `0.1` is the value a +`0.10` decodes to. A method returns the value as the row's type, decoding the text through the row's +codec, as [the type table](../../contract/type-table.md) builds any literal of a mapped type: + +```go +type Day string // date, enum: ["2020-01-01"] + +const Day20200101 Day = "2020-01-01" + +func (d Day) Value() (sdk.Date, error) // the member as the row's type +``` + +Two designs lost to it. Package-level variables built through the codec cover every type, but a +`var` can be assigned, and `==` is wrong for `time.Time` and absent for a decimal over `math/big`, +so a strict decoder and a caller's `switch` would each need a comparison of their own. A named type +over the row's type has no constant to give a caller at all. A Go constant is declared at package level, where every enum's constants meet, so Go names a member's constant by its enum's identifier, as rendered, followed by the member's name diff --git a/docs/emitters/targets/kotlin/lowering.md b/docs/emitters/targets/kotlin/lowering.md index d485c140a..328addc9b 100644 --- a/docs/emitters/targets/kotlin/lowering.md +++ b/docs/emitters/targets/kotlin/lowering.md @@ -20,7 +20,7 @@ where Multiplatform differs from the JVM ([Multiplatform common code](multiplatf | `Model` whose `Base` is no model, not a decorator | with no property, catch-all or discriminator of its own, lowered as the `Scalar` row lowers an alias over that base, its constraints, docs and examples binding the position ([scalars](../../../ir/scalars.md)); a union base beside properties distributed over the variants as [models](../../../ir/models.md#composition-beside-a-union) distributes a union co-declared with a composition; anything else a diagnosed degrade | | `AdditionalProps` / `Additional` | catch-all `Map` filled by a serializer that reads the whole object ([unknown members](models.md#unknown-members)); strict `Json`, `@JsonIgnoreUnknownKeys` on each other open model | | `Union` (four wire shapes, [unions](../../refine/unions.md#the-wire-shapes)) | sealed interface + one wrapper class per variant + a `KSerializer` per wire shape; `Open` → an `Unknown` fallback; KMP: not exhaustive in Swift; an inclusive `anyOf` → a class with a nullable property per variant ([unions](unions.md#an-inclusive-union)) | -| `Enum` open/closed/flags | closed → `enum class` with `@SerialName` members, `FallbackMember` on decode; open, or `TypePlan.Open` → value class over the value type with a constant per member, which keeps an unknown value; flags → value class over an integer | +| `Enum` open/closed/flags | closed → `enum class` with `@SerialName` members on a `string` enum, a generated `KSerializer` writing each member's JSON kind on any other ([enums](types.md#enums)), `FallbackMember` on decode; open, or `TypePlan.Open` → value class over the value type with a constant per member, which keeps an unknown value; flags → value class over an integer | | `List`/`MapT`/`Tuple` | `List`/`Map`/a class with a positional serializer (nullable past `MinItems`, `Rest` → trailing `List`) | | `Literal` | `const val`, or an `object` for a non-primitive value | | `External` | the type-table row for its `Identity`, each platform's own; no row → diagnostic | diff --git a/docs/emitters/targets/kotlin/types.md b/docs/emitters/targets/kotlin/types.md index 802d3f4ca..a17049c39 100644 --- a/docs/emitters/targets/kotlin/types.md +++ b/docs/emitters/targets/kotlin/types.md @@ -26,3 +26,11 @@ the root's type, and its serializer, in every signature. An enum's forms are in the [lowering table](lowering.md). A Kotlin entry lives in its enum class and takes the member's name alone (`ACTIVE`, as the naming policy cases it). + +`kotlinx.serialization` writes an `enum class` as a string, its entry's `@SerialName`, which is a +member's wire form only for a `string` enum ([a member on the +wire](../../refine/enums.md#a-member-on-the-wire)). Any other closed enum is an `enum class` whose +`KSerializer` is generated: it writes each entry's wire form as the value type's JSON kind, a +boolean for `bool`, a number for `integer`, `number` and `decimal`, and decodes by comparing as JSON +does. An entry of a value type a row maps, such as a `date`, holds the member as that row's type, +`val value: LocalDate`, built through the row's codec when the class loads. diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index 6c9829c2a..19002002d 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -195,3 +195,10 @@ throughout these documents. a coded error under `only` ([plan policy](../plan/policy.md#an-enum-a-client-reads-is-open), [unions](../refine/unions.md#variant-identity-and-recognition)). +36. **T-36 · Enums over every value type** — on every target, an enum of each primitive a + `ValueType` can name, `bool`, `integer`, `number`, `decimal`, `date`, `uuid` and `bytes` among + them, closed and open, encodes each member as its value type's JSON kind and decodes it back to + the same constant, a number spelled with trailing zeros included; where the target's constants + cannot hold the type, the member's value is reachable as the row's type, and no member is ever + written as a string its value type does not admit + ([a member on the wire](../refine/enums.md#a-member-on-the-wire)). From 624f08e288a89c21ec92b3b9cb73a3c56a71d7ba Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:36:17 +0300 Subject: [PATCH 06/14] docs(emitters): state Kotlin's reserved sets for units Kotlin's naming document stated only the members it gives every type, so a Kotlin emitter had no names to reserve for a group's module or for the declarations a unit holds beside the spec's, and T-19 no fixture. Measured with Kotlin 2.4.0 and Gradle 9.8.2: a module named build sits in the root project's output, which clean deletes with the module's sources; one named buildsrc is taken for Gradle's buildSrc on a case-insensitive file system and fails the build; gradle and src share the wrapper's and the root sources' directories. Those four are the units set, and a group's module is placed at the build root so the set applies. Hard keywords need no reservation: backticked as package segments they compile on the JVM, JS and Native and link into a framework, and kotlin is refused only as the first segment, which the options name. The unit set is Client, a group's sub-client and the runtime names the root unit declares. --- docs/emitters/targets/kotlin/naming.md | 23 ++++++++++++++++++++++- docs/emitters/targets/kotlin/overview.md | 4 +++- docs/emitters/testing/mandatory-tests.md | 5 +++-- 3 files changed, 28 insertions(+), 4 deletions(-) diff --git a/docs/emitters/targets/kotlin/naming.md b/docs/emitters/targets/kotlin/naming.md index 3f3311fc3..aae66cbac 100644 --- a/docs/emitters/targets/kotlin/naming.md +++ b/docs/emitters/targets/kotlin/naming.md @@ -4,9 +4,30 @@ What each [scope](../../refine/reserved-names.md) reserves in Kotlin: +- **The units**: `build`, `buildsrc` in any letter case, `gradle` and `src`, the names Gradle gives + a meaning at a build's root, where a group's module sits ([output + units](overview.md#output-units)). `build` is the root project's output, so a `clean` of the root + deletes a module there, sources and all. `buildSrc` is Gradle's build-logic build, found without + regard to case on a case-insensitive file system such as macOS's, so a module named `buildsrc` + fails the build there. `gradle` holds the wrapper and the version catalog, and `src` the root + project's sources, so a module named either would share their directory. +- **A unit**: `Client` and, in a group's unit, its sub-client; and in the root unit every runtime + name the target declares there, which each runtime template declares beside it, as a function's + names are ([reserved names](../../refine/reserved-names.md)): `ApiFailure`, `ApiException`, + `ClientException`, `ServerException`, the exception of each kind the error taxonomy names + (`NotFoundException`), `Fault`, `OtherResponse`, `OtherResponseException`, `Response`, `RawBody`, + `Optional`, `Duration`, `Authenticator` and `Transport`, and on Multiplatform `Decimal` and + `OffsetDateTime`. - **A type**: `equals`, `hashCode`, `toString`, `Builder` and `Companion`. -The sets for the units and for a unit are not stated yet (#1187). +Neither a hard keyword nor `kotlin` is a units name. A hard keyword as a package segment takes the +escape below, which kotlinc accepts in a `package` and an `import` alike: `fun`, `object`, `when`, +`is`, `in` and `typealias`, backticked, compile for the JVM, JS and macOS Native and link into a +framework, and those with `class`, `val`, `interface`, `as`, `true`, `null`, `this`, `super`, +`package`, `return`, `throw` and `try` on the JVM. kotlinc refuses `kotlin` only as a package's +first segment ("Only the Kotlin standard library is allowed to use the 'kotlin' package"), which is +the root's and so the options' to name. Each of these was measured with Kotlin 2.4.0 and Gradle +9.8.2 on darwin/arm64. ## The escape diff --git a/docs/emitters/targets/kotlin/overview.md b/docs/emitters/targets/kotlin/overview.md index 27dd76408..4e035a4c0 100644 --- a/docs/emitters/targets/kotlin/overview.md +++ b/docs/emitters/targets/kotlin/overview.md @@ -35,7 +35,9 @@ Codecs are generated `KSerializer`s, the counterpart of Go's A Kotlin [unit](../../refine/output-units.md) is a package, and also a Gradle module of its own unless it is the root, since kotlinc compiles a module whole and a Multiplatform framework exports -what its modules declare ([scale](scale.md)). A group's unit is `.`. +what its modules declare ([scale](scale.md)). A group's unit is `.`, and its module is +the directory `` at the build's root, beside the root project's own `src`, so the names +Gradle gives a meaning there are reserved ([naming](naming.md#reserved-names)). Kotlin's limit counts a Gradle module's declarations, and its default is set when the Kotlin targets are first measured ([open questions](../../open-questions.md)). diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index 19002002d..698821a91 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -68,8 +68,9 @@ throughout these documents. 19. **T-19 · Reserved scopes** — a fixture naming its properties, parameters, path parameters, enum members and union variants after every name each scope of [reserved names](../refine/reserved-names.md) reserves, after characters alone and after names that begin - with a digit, and its groups after the standard-library packages its files import, generates - code that compiles on every target, with every template's declared names held to what it binds. + with a digit, and its groups after the standard-library packages its files import and after + every units name, generates code that compiles on every target, a `clean` of a Kotlin build + leaving every module's sources, with every template's declared names held to what it binds. 20. **T-20 · Recursive types** — a model that holds itself directly, through a base, through a tuple and through another model, and one whose cycle passes through a list or a union, generate code that compiles on every target, and exactly the references on a cycle Go holds by value are held From 8d3481e52d35b00aa05bcf50b1f0ef88488af868 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:45:15 +0300 Subject: [PATCH 07/14] docs(emitters): define what a generation keeps for the next one The manifest recorded hashes, a file list and the entity-to-symbol map, too little for a regeneration to honor what the last one did: an edited generated file was overwritten, a no-op rewrote every file, a reformatted spec looked new, and keep regions were found by path. The manifest now records the document's digest instead of the spec's, a normalized digest per written file, the options version, the steps that ran, files written once, each entity's correlation key, base name and symbol, and each keep region by its entity, under a schema version migrated forward. Regions follow their entity to its new file. Every ID-keyed input that matches nothing is reported with its likely new home. A README is merged as blocks the user writes around. Collisions now read the prior manifest: a shipped symbol keeps its name while its ID and correlation key agree, and only newcomers take a suffix, so adding an endpoint renames nothing. The prior manifest joins the inputs the purity rule names. A spike at Go 1.27.3 chose dst for the Go merge adapter: go/ast nodes moved between files lost their doc comments, one into a function body, while dst and a text splice were both correct, and only dst reaches inside a declaration. Claims that IDs survive any rename are narrowed to renames that keep the pointer. --- docs/architecture.md | 5 +- docs/emitters/README.md | 2 +- docs/emitters/contract/emitter.md | 13 ++- docs/emitters/contract/runtime-policy.md | 3 +- docs/emitters/contract/shaping-hints.md | 10 +- docs/emitters/milestones.md | 3 +- docs/emitters/open-questions.md | 4 +- docs/emitters/packages.md | 6 +- docs/emitters/plan/overview.md | 10 +- docs/emitters/refine/naming.md | 43 ++++++- docs/emitters/surface-verification.md | 12 +- docs/emitters/targets/go/emit.md | 12 +- docs/emitters/targets/go/overview.md | 24 +++- docs/emitters/testing/mandatory-tests.md | 21 +++- docs/emitters/write-integrate.md | 141 ++++++++++++++++++++++- docs/ir/identity.md | 11 +- docs/prior-art.md | 6 +- docs/reference-learnings.md | 4 +- 18 files changed, 276 insertions(+), 54 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 0d48a32ca..a505d72c2 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -415,8 +415,9 @@ though both are post-milestone scope: - **Write/integrate** — regenerating into a live repo alongside hand-written code requires: deterministic file paths, a generation manifest (spec/emitter/config hashes, sorted file - list, and an entity → generated-symbol map **keyed by IR stable IDs**, which survive path - and name churn), file-header provenance that gates pruning (never delete a file lacking the + list, and an entity → generated-symbol map **keyed by IR stable IDs**, which survive a rename + of what they are not built from, a correlation key carrying the rest (#679)), file-header + provenance that gates pruning (never delete a file lacking the generated-by header), ignore-region markers for hand-written islands, and additive-only merging. Additive-only writers also need a staleness check — (previous-revision entities − current entities) ∩ files-on-disk — to surface dead code that pruning cannot touch. diff --git a/docs/emitters/README.md b/docs/emitters/README.md index 257c84afa..96c8d3dbb 100644 --- a/docs/emitters/README.md +++ b/docs/emitters/README.md @@ -235,7 +235,7 @@ The Kotlin targets read the same plan and lower the same facts as Go. |---|---| | 1 IR is the ABI | `EmitterInput.Doc` is the only structural input; `emitters/*` imports `ir`, `ir/irfacts` + shared contract, never `compiler`/`engine`/a sibling target ([the contract](contract/emitter.md), [package layout](packages.md)) | | 2 Lossless, lowered late | all lowering is in `refine` steps building a fresh target AST; the IR is never mutated; unions/enums/discriminators/visibility/pagination/streaming/LRO arrive un-lowered and the step list is the acceptance test ([refine](refine/pipeline.md), [operations](operations/returns.md)) | -| 3 Stable IDs, names presentation | plan, AST `Origin`, manifest, surface, hints, and the collision table are all keyed by IR stable ID; renames are lookups; collisions never touch identity; a name with no words is named from its own characters alone, and placement reads no count, so neither moves when something else changes ([the plan](plan/overview.md), [naming](refine/naming.md), [output units](refine/output-units.md), [write / integrate](write-integrate.md)) | +| 3 Stable IDs, names presentation | plan, AST `Origin`, manifest, surface, hints, and the collision table are all keyed by IR stable ID; a rename that keeps an entity's pointer is a lookup, and one that moves it the correlation key's; collisions never touch identity; a name with no words is named from its own characters alone, and placement reads no count, so neither moves when something else changes ([the plan](plan/overview.md), [naming](refine/naming.md), [output units](refine/output-units.md), [write / integrate](write-integrate.md)) | | 4 Names neutral, emitter cases | `Ident` starts as the `Naming.Canonical` word sequence, the source spelling beside each word for its letter case only; `RenderCasing`/`EscapeReservedWords` are the only casing steps; struct tags come from `WireName`/`WireID`, a separate channel ([naming](refine/naming.md), [emit](emit/printing.md)) | | 5 Pure, typed diagnostics | `Generate` is pure/reentrant/no-stderr/no-I/O; every un-lowerable construct → coded `ir.Diagnostic`; the engine decides fatality ([the contract](contract/emitter.md), [contract rules](contract/emitter.md#contract-rules-the-type-system-and-tests-enforce)) | | 6 Heuristics are policy | `plan.Policy`, each target's versioned options (its type table and naming tables among them), `verify.Severity`, method-name derivation, and shaping hints are injectable and disable-able; the type table's width-picking rows, the acronyms, the character names and the leading words are defaults a configuration replaces; inferred outputs marked `Inferred` ([target options](contract/options.md), [type table](contract/type-table.md), [plan policy](plan/policy.md), [naming](refine/naming.md), [shaping hints](contract/shaping-hints.md), [surface verification](surface-verification.md)) | diff --git a/docs/emitters/contract/emitter.md b/docs/emitters/contract/emitter.md index e0db201b5..6e07d646e 100644 --- a/docs/emitters/contract/emitter.md +++ b/docs/emitters/contract/emitter.md @@ -60,7 +60,8 @@ type Artifact struct { Header HeaderPlacement // where the generated-by provenance header goes (top/after-shebang/none) SkipIfExists bool // scaffolding written once (config, .gitignore) — never overwritten Overwrite bool // regenerate but preserve ignore-regions (default is additive) - Integrate *IntegrateHint // path-prefix stripping / tree-shake root for live-repo merge (../write-integrate.md) + Integrate *IntegrateHint // path-prefix stripping / tree-shake root for live-repo merge, and + // whether the file is merged as blocks (../write-integrate.md) Origins []EntityID // IR IDs whose emission produced this file (drives ID-keyed prune) } ``` @@ -100,10 +101,12 @@ the document, which determines it. ## Contract rules the type system and tests enforce -- **Purity / determinism.** No I/O, clock, randomness, or stderr inside `Generate`. Same - `(Doc, Plan, Policy, Hints, Scope, Options)`, under a context that stays live, ⇒ byte-identical - `Artifacts`; `Facts` is a function of `Doc` and adds nothing to the tuple. Maps are walked in - sorted-key order, slices in source order ([T-6](../testing/mandatory-tests.md)). +- **Purity / determinism.** No I/O, clock, randomness, or stderr inside `Generate`. Same `(Doc, + Plan, Policy, Hints, Scope, Prior, Options)`, under a context that stays live, ⇒ byte-identical + `Artifacts`, the prior manifest included since the names a generation shipped are kept + ([naming](../refine/naming.md#collisions-and-the-names-a-generation-shipped)); `Facts` is a + function of `Doc` and adds nothing to the tuple. Maps are walked in sorted-key order, slices in + source order ([T-6](../testing/mandatory-tests.md)). - **Diagnostics are the only input-failure channel.** A construct the emitter cannot lower emits `ir.Diagnostic{Severity, Code:"emitters//…", Message, Provenance}` — never a silent fallback, never a panic ([T-7](../testing/mandatory-tests.md)). Codes live in a stable diff --git a/docs/emitters/contract/runtime-policy.md b/docs/emitters/contract/runtime-policy.md index d4b615fa6..07a8a7dcf 100644 --- a/docs/emitters/contract/runtime-policy.md +++ b/docs/emitters/contract/runtime-policy.md @@ -106,7 +106,8 @@ per-operation settings outvote a project-wide choice, which then could not be ma naming every operation. A key naming no operation of the document is reported, never ignored, since an operation whose ID -changed would otherwise lose its settings in silence. Where a higher layer replaces what the +changed would otherwise lose its settings in silence ([a key that matches +nothing](../write-integrate.md#a-key-that-matches-nothing)). Where a higher layer replaces what the document embeds, an info diagnostic names the setting, both values and the operation, so the document's author can see it was overruled. diff --git a/docs/emitters/contract/shaping-hints.md b/docs/emitters/contract/shaping-hints.md index 5f1db0f34..778e2f1d7 100644 --- a/docs/emitters/contract/shaping-hints.md +++ b/docs/emitters/contract/shaping-hints.md @@ -2,8 +2,11 @@ oagen's `OperationHint` / rename / remount / split-union-body / url-builder machinery is *SDK ergonomics, not API semantics* — the same category as runtime policy. It lives in -`EmitterInput.Hints`, keyed by **IR stable ID** (never `"METHOD /path"`), so a path rename never -drops a hint (INV3): +`EmitterInput.Hints`, keyed by **IR stable ID** (never `"METHOD /path"`), so a rename of anything +the ID is not built from, such as an `operationId`, never drops a hint (INV3). A path rename gives +the operation a new ID, and its hint is then [a key that matches +nothing](../write-integrate.md#a-key-that-matches-nothing), reported with the entity its correlation +key now names: ```go type ShapeHint struct { @@ -36,7 +39,8 @@ so one IR document drives different compat baselines per language. Hints come in layers, as runtime policy does ([layers](runtime-policy.md#layers-and-operations)): the hints a document embeds ([embedded policy](embedded-policy.md)), then the project's, then the emitter's, each field taking the highest layer's value. An ID no entity of the document has is -reported, never ignored. +reported, never ignored ([a key that matches +nothing](../write-integrate.md#a-key-that-matches-nothing)). `AlsoMount` answers an operation a document exposes under two names, which one `Remount` cannot. Each sub-client it names gets a method calling the operation, keyed in the manifest under the diff --git a/docs/emitters/milestones.md b/docs/emitters/milestones.md index 8c9a0ebfe..e4fa90934 100644 --- a/docs/emitters/milestones.md +++ b/docs/emitters/milestones.md @@ -17,7 +17,8 @@ Extends architecture.md §6 milestone 3 (the first emitters) and beyond. 4. **M3d — docs + mock emitters.** Prove reuse of `Doc + Plan` over a second and third target AST, and that docs/mock agree with the SDK by construction. 5. **M4 — write/integrate + surface verification.** Manifest, header-gated pruning, ignore regions, - additive `go/ast` merge, staleness; neutral surface diff with injectable per-language severity. + additive merge (`dst` for Go), regions that follow their entity, README blocks, staleness; + neutral surface diff with injectable per-language severity. 6. **M5 — a further language family (TS or Python).** Proves the plan holds beyond the first targets' statically typed languages: a new target is a new target AST + printer + refine pipeline reusing the shared plan and step contract, forcing **no** IR or plan change. The shared plan diff --git a/docs/emitters/open-questions.md b/docs/emitters/open-questions.md index 427959a27..7c2135591 100644 --- a/docs/emitters/open-questions.md +++ b/docs/emitters/open-questions.md @@ -19,8 +19,8 @@ RPC/messaging targets are the real test. Revisit when the second-protocol emitter lands (tracks [open question 6](../ir/open-questions.md)). 5. **Manifest granularity for merge.** Whether the entity→symbol map keys at property level (not just - type/op level) to support field-level additive merge, or leaves sub-type merge to the `go/ast` - adapter. Decide when write/integrate (M4) is built. + type/op level) to support field-level additive merge, or leaves sub-type merge to the + merge adapter. Decide when write/integrate (M4) is built. 6. **Surface behavioral channel scope.** Which behavioral changes beyond default-value changes (e.g. a `Retryable` flip, an idempotency change) belong in the separate behavioral channel vs the structural one. Decide when surface verification (M4) is built against real cross-revision diffs. diff --git a/docs/emitters/packages.md b/docs/emitters/packages.md index d35e5630b..034e7c31d 100644 --- a/docs/emitters/packages.md +++ b/docs/emitters/packages.md @@ -25,7 +25,7 @@ emitters/ │ ├── refine/ # IR+Plan → goast — the ordered lowering pipeline (refine/pipeline.md) │ ├── emit/ # goast → []byte via printer + go/format; templates/ (boilerplate only) │ ├── naming/ # Go reserved words + default acronym table (naming.Policy impl) -│ └── mergeadapter.go# go/ast additive-merge adapter (registers into manifest driver) +│ └── mergeadapter.go# dst additive-merge adapter (registers into manifest driver) ├── kotlin/ # ◀ FIRST TARGETS (TargetKeys "kotlin-jvm" and "kotlin-multiplatform") │ ├── emitter.go # NewJVM() and NewMultiplatform(): one pipeline, two Platform profiles │ ├── platform.go # Platform: the built-in type table, identifier rules, transport @@ -51,7 +51,9 @@ its own, as a compiler's packages do, and none may be exempt, which only tooling is. The entries the first packages add assert: `emitters/golang` and `emitters/kotlin` may each import `emitters`, `emitters/plan`, `emitters/policy`, `emitters/naming`, `emitters/typemap`, `emitters/refine`, `emitters/manifest`, `emitters/verify`, `ir` and `ir/irfacts`; neither may import -`compilers/*`, `engine`, or the other. +`compilers/*`, `engine`, or the other. `emitters/golang` may also import `github.com/dave/dst`, for +its merge adapter, and no other package may +([merging](targets/go/overview.md#merging-and-surface-extraction)). **Switch-completeness tests** live beside `goast` and `ktast` (over each one's `NodeKind`) and beside every refiner (over every `ir.TypeKind`): adding an IR kind fails the test of every dispatch that must handle it (the `assertNever` obligation). diff --git a/docs/emitters/plan/overview.md b/docs/emitters/plan/overview.md index 46da6a7d0..c4d417b63 100644 --- a/docs/emitters/plan/overview.md +++ b/docs/emitters/plan/overview.md @@ -23,10 +23,12 @@ const Revision = 1 // bumped with every change to what Buil ``` Every key is an IR stable ID — never `"METHOD /path"`, never a rendered name (INV3; the oagen -fragility, prior-art §1). A spec rename never orphans a plan entry: it is "same ID, new symbol", -a lookup. Every inferred field carries `Inferred` provenance, propagated into diagnostics so it is -auditable and disable-able (INV6). The plan's own diagnostics are coded `emitters/plan/…`, reported -once for every target rather than once by each. +fragility, prior-art §1). A rename of anything an ID is not built from, such as an `operationId` or +a `title`, never orphans a plan entry: it is "same ID, new symbol", a lookup. A rename of what an ID +is built from, a path or a component name, gives the entity a new ID, and only its correlation key +(#679) carries it across the revision. Every inferred field carries `Inferred` provenance, +propagated into diagnostics so it is auditable and disable-able (INV6). The plan's own diagnostics +are coded `emitters/plan/…`, reported once for every target rather than once by each. The plan **classifies; it does not pick target types and it never cases anything.** It records what it derives from the IR ("this union is discriminated, and its tag values select the variants with diff --git a/docs/emitters/refine/naming.md b/docs/emitters/refine/naming.md index b4c047d73..91439e92e 100644 --- a/docs/emitters/refine/naming.md +++ b/docs/emitters/refine/naming.md @@ -38,12 +38,43 @@ there is no lossy runtime re-splitting of a cased string (oagen's `splitWords` c `EscapeReservedWords` runs after casing and **never touches `WireName`/`WireID`** — struct tags are built from the wire channel, so wire correctness never depends on the rendered identifier ([T-4](../testing/mandatory-tests.md)). Collisions are resolved deterministically in `Ctx.Syms`, -which is keyed by IR ID: adding one endpoint never renames an existing method (the oagen -collision-cascade counterexample; INV3). Anonymous hoisted types get an emitter-chosen name from -`Naming.Hint`. Per-service presentation renames (`Service.Renames`) change how a shape is presented -in a service without changing its `TypeID` or its own `Naming`. Enum members are keyed by -`EnumMember.ID`, which is derived from the value a member holds, so two members whose names render -alike are two entries to disambiguate, and reordering members of distinct values renames none. +which is keyed by IR ID, and against the names the prior generation shipped, so adding one endpoint +never renames an existing method (the oagen collision-cascade counterexample; INV3; +[collisions](#collisions-and-the-names-a-generation-shipped)). Anonymous hoisted types get an +emitter-chosen name from `Naming.Hint`. Per-service presentation renames (`Service.Renames`) change +how a shape is presented in a service without changing its `TypeID` or its own `Naming`. Enum +members are keyed by `EnumMember.ID`, which is derived from the value a member holds, so two members +whose names render alike are two entries to disambiguate, and reordering members of distinct values +renames none. + +## Collisions, and the names a generation shipped + +Names that render alike in one scope collide, and `Ctx.Syms` tells them apart. Keying by ID and +being deterministic are both needed and neither is enough: no rule reading only the current document +can tell which of two colliding operations shipped first, so any such rule can rename the incumbent +when a newcomer joins. A renamer that renames every member of a collision from its own identity is +deterministic and order-independent, and still renames `get` to `zonesGetZone` the day +`zonesGetZoneSettings` joins its group, a breaking SDK change. Only the prior generation knows the +incumbent, so the prior manifest ([write / +integrate](../write-integrate.md#what-the-manifest-records)) is an input to the rule: + +1. **A shipped name keeps its symbol.** An entity whose prior entry has its ID and its correlation + key, in the same unit, with the same `Base`, takes the entry's `Symbol` again, a suffix it was + given included. Two such symbols were apart in the prior generation and stay apart. +2. **Every other member, in ID order**, takes its rendered name where nothing in the scope holds it, + else that name with the lowest numeric suffix, `2` up, that is free. + +Only the choice is pinned, never the name: an entity whose own rendered name changed, as a renamed +`operationId` does, gets a new `Base`, so rule 1 does not hold for it and it is named afresh. An ID +alone pins nothing, since an ID built from an array index can name another entity in the next +revision (#678): the correlation key must agree too. On a first generation, or one run without its +prior manifest, rule 2 decides alone; ID order is independent of declaration order, so the two-order +oracle holds it. What the promise rests on is stated with it: a name stays only while its entity's +ID and correlation key survive the revision, and an ID derived from a pointer changes when the +pointer does (#419), until the correlation key carries it across. + +The prior manifest is an input like any other, so a generation is still a pure function of its +inputs ([contract rules](../contract/emitter.md#contract-rules-the-type-system-and-tests-enforce)). ## The source spelling, for letter case only diff --git a/docs/emitters/surface-verification.md b/docs/emitters/surface-verification.md index 1cd0c56ee..fc7cb1574 100644 --- a/docs/emitters/surface-verification.md +++ b/docs/emitters/surface-verification.md @@ -4,11 +4,13 @@ Post-milestone. `EmitterOutput.Surface` is the neutral API-surface projection enums, methods, fields as neutral records — built by the **same walk** over `Origin`-tagged AST nodes that emit performs. -- **Neutral projection with IR IDs.** Both the freshly-generated SDK and an extracted - existing SDK (via a per-language extractor) project into one `verify.Surface`; each record carries - its **IR stable ID** (threaded from `Origin`), so cross-language findings correlate without - name-matching, and rename detection is an ID lookup rather than oagen's Jaccard-similarity - heuristics. `Generate` returns the generated side directly; only the extractor parses source. +- **Neutral projection with IR IDs.** Both the freshly-generated SDK and an extracted existing SDK + (via a per-language extractor) project into one `verify.Surface`; each record carries its **IR + stable ID** (threaded from `Origin`), so cross-language findings correlate without name-matching, + and a rename that keeps the entity's pointer is an ID lookup rather than oagen's + Jaccard-similarity heuristics. One that moves the pointer, such as a path renamed, is found by the + correlation key (#679) the manifest records. `Generate` returns the generated side directly; only + the extractor parses source. - **Injectable per-language severity.** The differ emits **neutral change records**; a per-target `verify.Severity` function decides breaking/additive (parameter names are public API in PHP, arity in Go, ~nothing in JS) — heuristics-as-policy (INV6). diff --git a/docs/emitters/targets/go/emit.md b/docs/emitters/targets/go/emit.md index d56490d14..b66f76904 100644 --- a/docs/emitters/targets/go/emit.md +++ b/docs/emitters/targets/go/emit.md @@ -41,11 +41,13 @@ type Iface struct{} // any ## Why a domain AST → text → `go/format`, not `go/ast` directly `go/ast` is awkward to *synthesize* (it wants a `token.FileSet` and real positions and cannot carry -generator intent like a field's `Origin` IR ID or a `Docs.Refs` span). The domain AST is -smaller, holds provenance and wire metadata on its nodes, and prints to text that `go/format` then -makes canonical. `go/ast` is used in exactly one place — re-parsing *existing* files for additive -merge ([write / integrate](../../write-integrate.md)) — where its parser is the right tool. The two -Go trees are kept aligned by ID-keying: "a symbol I generated" ↔ "a symbol I parsed" is a lookup. +generator intent like a field's `Origin` IR ID or a `Docs.Refs` span). The domain AST is smaller, +holds provenance and wire metadata on its nodes, and prints to text that `go/format` then makes +canonical. A syntax tree of Go is used in exactly one place — re-parsing *existing* files for +additive merge ([write / integrate](../../write-integrate.md)) — and that one is `dst`'s, which +keeps comments on their nodes where `go/ast` cannot +([merging](overview.md#merging-and-surface-extraction)). The two Go trees are kept aligned by +ID-keying: "a symbol I generated" ↔ "a symbol I parsed" is a lookup. ## Runtime templates diff --git a/docs/emitters/targets/go/overview.md b/docs/emitters/targets/go/overview.md index fb925593a..a55212f1a 100644 --- a/docs/emitters/targets/go/overview.md +++ b/docs/emitters/targets/go/overview.md @@ -39,8 +39,28 @@ check](../../refine/output-units.md#the-size-check) reports and its split by gro ## Merging and surface extraction -The Go [merge adapter](../../write-integrate.md) (`emitters/golang/mergeadapter.go`) uses `go/ast` -to parse existing files and append only new top-level symbols, never delete. +The Go [merge adapter](../../write-integrate.md) (`emitters/golang/mergeadapter.go`) parses an +existing file with `github.com/dave/dst`, a decorated syntax tree, and appends only new top-level +symbols, never deleting one. `dst` keeps each comment on the node it describes, so a declaration +moved from the new generation's file into the old one carries its doc comment and a +`@morphic:keep` marker stays where its region is; `go/ast` keeps comments in a list of their own by +byte offset, so a node moved between two files leaves them behind. + +A spike settled it, at Go 1.27.3 with `dst` v0.28.0. A generated file holding a keep region gained +a type and a method from the next generation, three ways: + +- **`go/ast` nodes moved, printed by `go/printer`:** the moved declarations lost their doc + comments, which landed above `GetZone`, and one of them inside `GetZone`'s body. +- **`go/ast` to find the declarations, their source text spliced in, `astutil.AddImport`:** correct, + the region untouched and the import added. It reaches top-level declarations alone, so adding a + field to a struct, which a merge at property level would need ([open + questions](../../open-questions.md), 5), would be a splice inside one with no tree to check it. +- **`dst`:** correct, imports managed by its restorer, and a file using generics, a range-over-func + iterator and `min` round-trips byte for byte. + +`dst` is MIT-licensed, at v0 and maintained in bursts, its last release in 2026. So its version is +pinned exactly, `emitters/golang` alone may import it ([package layout](../../packages.md)), and an +upgrade reruns the spike's cases as tests of the adapter. Go's extractor ([surface verification](../../surface-verification.md)) reads the existing package's exported API, as `apidiff` does. diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index 698821a91..31d3dca55 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -21,7 +21,9 @@ throughout these documents. 7. **T-7 · Diagnostic-not-silence** — every un-lowerable construct emits a coded diagnostic, never a silent `any` and never a panic, a fact outside M3's scope included ([what M3 lowers](../milestones.md#what-m3-lowers)). -8. **T-8 · Manifest ID-keying** — the entity→symbol map survives path/name churn. +8. **T-8 · Manifest ID-keying** — the entity→symbol map survives a rename of what an ID is not + built from, such as an `operationId`, and, through the correlation key, a path or component + rename. 9. **T-9 · Policy precedence** — declared `Retryable`/`Fault`/`Idempotency` override policy defaults at a document-wide setting and at an operation's, in every layer. A project's document-wide setting beats a per-operation one the document embeds, an operation's own setting beats its @@ -203,3 +205,20 @@ throughout these documents. cannot hold the type, the member's value is reachable as the row's type, and no member is ever written as a string its value type does not admit ([a member on the wire](../refine/enums.md#a-member-on-the-wire)). +37. **T-37 · Shipped names** — on every target, a generation whose operation collides with nothing, + then a revision adding one that collides with it, regenerated against the first generation's + manifest, keeps the first symbol unchanged and gives the newcomer a suffix; the first + generation is byte-identical in both declaration orders; a renamed `operationId` renames its + symbol; and an ID that now names another entity, its correlation key differing, is not pinned + ([collisions](../refine/naming.md#collisions-and-the-names-a-generation-shipped)). +38. **T-38 · Manifest and integration** — a regeneration whose inputs all match writes nothing; a + spec reformatted into the same IR is the same generation; a generated file edited outside its + regions is reported and not overwritten, while one checked out with other line endings is not + an edit; a region follows its entity into the file a rename moves it to, and one whose entity + is gone is reported and its file kept; a file written once is never written again; a hint, an + override and an overlay key matching nothing each draw `emitters/unmatched-key`, naming the + entity of the same correlation key; a README's text outside its blocks and its frozen blocks + survive byte for byte, and its table of contents lists the user's headings; and a manifest of + an older `Schema` is migrated, one of a newer refused + ([write / integrate](../write-integrate.md)). + diff --git a/docs/emitters/write-integrate.md b/docs/emitters/write-integrate.md index e6e889ff7..6f6852d77 100644 --- a/docs/emitters/write-integrate.md +++ b/docs/emitters/write-integrate.md @@ -4,14 +4,16 @@ Post-milestone (architecture.md §2.3), but the contract is designed so it bolts change** (INV9). The `manifest.Manifest` and `Artifact` fields returned from `Generate` are the seams; the `Origin`-tagged AST makes the manifest a byproduct of the emit walk. -- **Generation manifest keyed by IR stable IDs.** Records spec/plan/emitter/config **hashes**, a +- **Generation manifest keyed by IR stable IDs.** Records what the generation was built from, a **sorted** file list (deterministic diffs), and an **entity → generated-symbol map keyed by `TypeID`/`OpID`/`PropID`** — assembled from the `Origin` on every emitted node. This is the - biggest win of stable-ID keying: a spec rename is "same ID, new symbol" — a lookup, not oagen's - structural rename-inference machinery. A declaration derived from an entity keys under it, as - `User.ID#create` does ([`User`](targets/go/models.md#user-in-go)), a union's wrappers and fallback - among them, so no two symbols share a key. Write/integrate and surface verification - (architecture.md §2.3) both rest on it. + biggest win of stable-ID keying: a rename of anything an ID is not built from is "same ID, new + symbol" — a lookup, not oagen's structural rename-inference machinery — and a rename of what it is + built from, a path or a component name, is carried by the correlation key each entry records. A + declaration derived from an entity keys under it, as `User.ID#create` does + ([`User`](targets/go/models.md#user-in-go)), a union's wrappers and fallback among them, so no two + symbols share a key. Write/integrate and surface verification (architecture.md §2.3) both rest on + it. - **File-header provenance gates pruning.** `Artifact.Header` writes a generated-by header; the writer **never deletes a file lacking it** (hand-edited). First adoption (no prior manifest) skips pruning, writes a baseline, emits a notice. @@ -24,3 +26,130 @@ seams; the `Origin`-tagged AST makes the manifest a byproduct of the emit walk. - **Staleness check.** `(prior-manifest entities − current entities) ∩ files-on-disk` surfaces dead code additive writers cannot prune — cheap because both manifests are ID-keyed and the prior IR/plan snapshot is serializable (INV7). + +## What the manifest records + +A manifest holds the choices a later generation must honor and cannot recompute from the spec: the +names it shipped, the behavior it pinned, the files a user now owns. So it records more than hashes: + +```go +package manifest + +type Manifest struct { + Schema int // this format's version; an older one is migrated forward + Target string // the emitter's TargetKey + Options int // OptionsVersion(), the behavior pinned for this SDK + // (contract/options.md) + Inputs Inputs // what the generation was built from + Files []File // every file written, sorted by path + Entities map[EntityID]Entry // every emitted entity and each declaration derived from one + Steps map[string]int // each refine step and template that ran → its version + Regions []Region // the hand-written regions found, each by its entity (below) +} + +type Inputs struct { + Doc string // the document's digest, which Plan.Version takes + // (plan/overview.md) + Plan string // Plan.Version + Emitter string // the emitter's own version + Config string // SHA-256 of the decoded options, resolved policy and resolved + // hints, each as its canonical JSON +} + +type File struct { + Path string + Digest string // SHA-256 over the bytes written, normalized (below) + Origins []EntityID // as the Artifact lists them + Once bool // written when absent, the user's thereafter +} + +type Entry struct { + Key string // the entity's correlation key, from the pass #679 plans + Unit string // the unit the symbol was placed in + Base string // its rendered name before a collision was resolved + Symbol string // the name the generation gave it (refine/naming.md) + Kind string // the target-AST node kind it was declared as + File string // the path it was written to +} +``` + +- **The document, not the spec text.** `Inputs.Doc` is the IR's digest, canonical JSON with + `sources` left out. Reformatting a spec, or moving it to another path, changes the spec's bytes + and leaves the IR whole, so it is no new generation. +- **A file as written.** `Digest` covers the bytes after formatting, with a byte-order mark + stripped, line endings made `\n` and trailing newlines dropped, so a formatter run or a checkout + that converts line endings is not mistaken for an edit. A file carrying the generated header + whose bytes on disk no longer match was edited outside its regions. The writer leaves it as it is + and reports it as a coded diagnostic, whose severity the engine decides, rather than overwrite + the edit. +- **Nothing to do.** When every `Inputs` digest and every file's digest match the prior manifest, + the engine writes nothing. +- **Behavior and steps.** `Options` is the version whose defaults this SDK keeps until its owner + moves it ([target options](contract/options.md)), and `Steps` names the version of each step + that shaped it, which [surface verification](surface-verification.md) reads to say what a + regeneration changed and why. +- **Written once.** A file a target generates for the user to own, such as a test to fill in, is + `Once`: written while absent, never rewritten or pruned. It is matched to a later generation by + its origins' correlation keys, so a renamed entity's file is not written a second time under the + new name. +- **Its own version.** Unlike the IR, a manifest holds choices that cannot be recomputed, so an + older `Schema` is never refused: each version step has a migration, applied in order, and a + manifest from a newer Morphic is refused, since its fields cannot be read. + +## Regions follow their entity + +A `@morphic:keep ` region belongs to an entity: the one whose declaration encloses it, or, at +a file's top level, the one whose declaration it follows, else the file's first origin. The +manifest records each as a `Region`, keyed by that entity's ID, its correlation key and the region's +name, with the region's text. So a region follows its entity: when a rename moves the entity to +another file, the next generation writes the region after the entity's declaration there, and the +old file, emptied of generated symbols, is pruned as any other. + +```go +type Region struct { + Entity EntityID // the entity it belongs to + Key string // that entity's correlation key + Name string // the name its marker gives it, unique within the entity + File string // where it was found + Text string // what it held, markers excluded +} +``` + +A region whose entity no longer exists is reported and kept: its file is not pruned while it holds +one, and the staleness check lists it. Merging stays additive. A mode that rewrote generated code a +user edited would take ordinary three-way merge semantics, the prior generation as the base and +conflicts written with the markers merge tools read, never a merge of its own invention. + +## A key that matches nothing + +Every input keyed by an IR ID can outlive its entity: a hint ([shaping +hints](contract/shaping-hints.md)), a runtime-policy override +([layers](contract/runtime-policy.md#layers-and-operations)), a per-target naming overlay, an +overlay, the prior manifest. A spec change that removes or renames the entity leaves the key +matching nothing, and ignoring it would drop the user's customization in silence. So every such key +is reported, as a warning `emitters/unmatched-key` whose severity the caller controls and which +never aborts the run. It names the key and, where one exists, the entity whose correlation key +equals the one recorded for it, as the likely new home. A prior manifest's entry for a removed +entity is not such a key: the staleness check reports it. + +## A document the user writes around + +A README is mostly hand-written, with generated sections: installation, authentication, an +example per feature, a table of contents. A keep region protects hand-written islands in generated +code; a README is the reverse, so it is integrated as blocks. The docs emitter +([docs and mocks](docs-and-mock.md)) returns each generated section as a block with a stable ID, +between `` and ``, in an artifact whose +`Integrate.Blocks` is set. The engine's integrate step then rewrites only those blocks in the file +on disk: + +- a block found there is replaced by the new one of its ID, and everything outside the markers is + kept byte for byte; +- a block marked `` is the user's, and kept as it is; +- a block new to this generation goes after the last block it follows in the generated order, or + at the end; +- a block the generation no longer has is removed, unless frozen, and reported; +- the table of contents is a block of its own, rebuilt last from the headings of the merged file, + so it lists the user's sections too. + +A file with no markers yet takes the generated blocks after its own text, once, and a first +generation writes the whole artifact. diff --git a/docs/ir/identity.md b/docs/ir/identity.md index b03f8a463..a824002d7 100644 --- a/docs/ir/identity.md +++ b/docs/ir/identity.md @@ -10,11 +10,12 @@ type ServiceID, GroupID, ChannelID, MessageID, AuthID, PropID, ParamID, EnumMemb type ServerID string ``` -IDs are opaque to consumers but constructed deterministically by compilers from the source -pointer of the defining occurrence, so the same input always yields the same IDs (stable -snapshots, cachable, diffable across spec revisions). IDs are never derived from display names -and never rewritten by renames. The `dedup` pass may alias two structurally identical anonymous -types; aliases are recorded so both IDs stay resolvable. +IDs are opaque to consumers but constructed deterministically by compilers from the source pointer +of the defining occurrence, so the same input always yields the same IDs (stable snapshots, +cachable, diffable across revisions that keep each entity's pointer; one that renames a path or a +component gives what is beneath it new IDs, #419). IDs are never derived from display names and +never rewritten by renames. The `dedup` pass may alias two structurally identical anonymous types; +aliases are recorded so both IDs stay resolvable. ## The ID grammar diff --git a/docs/prior-art.md b/docs/prior-art.md index 85681386d..8d6cf796d 100644 --- a/docs/prior-art.md +++ b/docs/prior-art.md @@ -44,7 +44,8 @@ and the differ/compat/verify subsystems added since the original survey). symbol map), file-header provenance that gates pruning (never delete a file lacking the generated-by header), ignore-region markers for hand-written islands, and additive-only AST merges. Manifests should key entities by stable ID — oagen's `"METHOD /path"` keys break - on path renames. + on path renames. So do pointer-derived IDs, which is why a correlation key is planned beside + them (#679). - **Wire-conformance smoke testing with a spec-only offline baseline** — the expected request shape (method, path, query, body keys) is derived from the spec alone and diffed against the generated SDK running under HTTP interception; request-side mismatches block, response-side @@ -89,7 +90,8 @@ and a self-correcting overlay loop. Three structural lessons: model even has a dual-key design (`id` + name) intended for rename detection — but `id` is derived from the name, so the rename branch is dead. Cross-language change rollup finally reinvents a spec-level identity (`conceptualChangeId`). Morphic's pointer-derived IDs make - rename detection a lookup; the compat experience adds one refinement — when both pointer + detecting a rename that keeps the pointer a lookup; the compat experience adds one + refinement — when both pointer *and* name churn, a third *structural* correlation tier (content-hash / field-set similarity, shared with the `dedup` pass) is the fallback (see [open question 5](./ir/open-questions.md)). diff --git a/docs/reference-learnings.md b/docs/reference-learnings.md index 6f3e4da5c..157f08e85 100644 --- a/docs/reference-learnings.md +++ b/docs/reference-learnings.md @@ -178,7 +178,9 @@ touches some slice of it: if its content starts with the auto-generated header*, else preserve + report as hand-edited; **skip pruning entirely on first adoption**. Adopt this schema directly. Morphic's one improvement (already specced): key the entity→symbol map by **IR stable ID**, not oagen's `"METHOD /path"` - string, so it survives path/name churn. Record `service` per op (oagen needs it for scoped runs). + string, so it survives a rename of anything the ID is not built from, and record a correlation + key (#679) beside it for a path or name rename. Record `service` per op (oagen needs it for scoped + runs). - **Scoped regeneration NEVER filters the spec** (oagen `orchestrator.ts:37`, `generate-files.ts:44`; fastapi `--specify-tags`; openapi-python full-rmtree). Placement/dedup/shared-files/barrels are computed over the *full* (or `selected ∪ already-on-disk`) surface so shared files stay byte- From 2a1112a801b63c8e49a97027c42c6e7c64f23003 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:49:49 +0300 Subject: [PATCH 08/14] docs(emitters): design the docs emitter and its shared examples The docs emitter reused the SDK's primary response and content, had no design for URLs, links, collisions or versions, spelled usage examples it could only re-derive, and left documentation extensions and agent documents unaddressed. Every artifact would also synthesize its own example values. The docs now render every response, error case and content, the primary only ordering them, and each type's request and response shapes apart. A route plan derives every URL from an ID-keyed catalog through templates, refuses any two entities on one URL after the templates apply, resolves links as ID requests, and keeps one entity's identity across version snapshots through route tables the engine passes in. Agent Markdown and llms.txt share those routes and escape spec text. SDK emitters return a usage sample per operation for the docs, and the plan holds one set of examples, declared first and otherwise seeded from each entity's correlation key, for docs, samples, tests and mocks. Documentation extensions are read by mapping, internal markers left to the filter pass, and a CLI is planned as an extension point built from IR facts. --- docs/architecture.md | 7 +- docs/emitters/contract/emitter.md | 15 +++ docs/emitters/contract/registry.md | 1 + docs/emitters/docs-and-mock.md | 138 ++++++++++++++++++++++- docs/emitters/emit/templates.md | 2 +- docs/emitters/milestones.md | 4 + docs/emitters/plan/overview.md | 8 ++ docs/emitters/plan/plan-or-refine.md | 4 +- docs/emitters/plan/policy.md | 8 ++ docs/emitters/testing/mandatory-tests.md | 15 +++ 10 files changed, 194 insertions(+), 8 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index a505d72c2..b0e9e3462 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -389,9 +389,10 @@ Passes operate on the IR only; they know nothing about source formats or target ### 2.3 Emitters (IR → artifacts) -Emitters are plugins (SDK-per-language, docs, mock servers…). Out of scope for this session -except for the boundary they impose on the IR; the internal shape mirrors what Kiota and oagen -converged on independently: +Emitters are plugins (SDK-per-language, docs, mock servers, and a command-line client planned as an +extension point, `docs/emitters/docs-and-mock.md`…). Out of scope for this session except for the +boundary they impose on the IR; the internal shape mirrors what Kiota and oagen converged on +independently: 1. **Plan** — compute language-*neutral* per-operation and per-type decisions once (is-paginated, error taxonomy, primary-content/response selection, return-shape classification (a success body diff --git a/docs/emitters/contract/emitter.md b/docs/emitters/contract/emitter.md index 6e07d646e..e9620b0e5 100644 --- a/docs/emitters/contract/emitter.md +++ b/docs/emitters/contract/emitter.md @@ -35,6 +35,11 @@ type EmitterInput struct { Scope *Scope // nil = whole document; else "emit only these" — full Doc still // consumed so global decisions stay byte-identical Prior *PriorState // prior manifest + prior surface for write/integrate; nil first run + Samples map[TargetKey]map[ir.OpID]Sample // each SDK target's usage samples, for a docs target + // (../docs-and-mock.md#samples-come-from-the-sdk-emitters) + Routes map[string]RouteTable // every version snapshot's routes, keyed by version, for a + // target whose Capabilities.CrossVersion is set + // (../docs-and-mock.md#routes-and-links) Options TargetOptions // the target's own typed options, decoded and validated before // Generate runs, its type table and naming tables among them // (options.md, type-table.md, ../refine/naming.md) @@ -45,8 +50,18 @@ type EmitterOutput struct { Diagnostics []ir.Diagnostic // typed, coded, provenance-bearing; engine decides fatality Manifest *manifest.Manifest // entity→symbol map keyed by IR stable IDs (../write-integrate.md) Surface *verify.Surface // neutral API-surface projection of the generated output (../surface-verification.md) + Samples map[ir.OpID]Sample // an SDK target's usage sample per operation, for the docs } +// Sample is one operation's call as an SDK target spells it. +type Sample struct { + Language string // the fence's language, such as "go" or "kotlin" + Code string // the call, its values the plan's examples +} + +// RouteTable is one snapshot's routes: each entity's page path and anchor. +type RouteTable map[EntityID]string + // EntityID is the string form of any IR stable ID (TypeID, OpID, PropID, ServiceID, …). // Every IR ID is a `type X string`, so one keyspace addresses them all. type EntityID = string diff --git a/docs/emitters/contract/registry.md b/docs/emitters/contract/registry.md index fe0069ca4..ff280576f 100644 --- a/docs/emitters/contract/registry.md +++ b/docs/emitters/contract/registry.md @@ -39,6 +39,7 @@ type Capabilities struct { LowersKinds []ir.TypeKind // the kinds this emitter has an emit path for WriteIntegrate bool // supports additive merge into a live repo (../write-integrate.md) SurfaceVerify bool // produces a verify.Surface (../surface-verification.md) + CrossVersion bool // reads every version snapshot's routes (../docs-and-mock.md) Protocols []string // "http", "grpc", "message", "graphql", "otp" Lifecycle Lifecycle // "hidden" | "experimental" | "stable" | "sunset"; required Successor TargetKey // the target replacing a sunset one; "" when none does diff --git a/docs/emitters/docs-and-mock.md b/docs/emitters/docs-and-mock.md index bfee42546..86ba7661c 100644 --- a/docs/emitters/docs-and-mock.md +++ b/docs/emitters/docs-and-mock.md @@ -3,8 +3,9 @@ Both are `Emitter` implementations consuming the **same `Doc + Plan`** — the payoff of computing the plan once: docs and mocks agree with the SDK by construction, not by re-derivation. -- **Docs emitter (`emitters/docs`).** Reuses the shared plan verbatim (primary response/content, - pagination, return-shape, error taxonomy — exactly what docs must describe) and most of the refine +- **Docs emitter (`emitters/docs`).** Reuses the shared plan verbatim (pagination, return shape, + error taxonomy — what docs must describe; [what the docs render](#what-the-docs-render) says + which responses) and most of the refine step library, but lowers into a `docast` (pages, operation blocks, schema tables, examples) and runs a **reduced pipeline**: no `EscapeReservedWords`, and casing is a presentation choice (SDK-style identifiers or spec-faithful source names, set in its options). The spans @@ -34,5 +35,138 @@ plan once: docs and mocks agree with the SDK by construction, not by re-derivati `UnmodeledEntry.Provenance` — the keyword's own position, not the enclosing schema's. The contract already carries all of that; no IR change is needed (INV9). +- **CLI emitter (future).** A command-line client is the third surface APIs ship beside SDKs and + docs, and almost all of it is IR and plan facts ([a command-line target](#a-command-line-target)). + Adding any of these is a registry entry over one `Doc + Plan` — no change to the ABI, no change to the IR. + +## What the docs render + +The primary response and content are an SDK's ergonomic choice: a method returns one. A reader of +the docs needs every answer the API gives, so the docs emitter renders every `Response`, every +`ErrorCase` and every `Content` the operation declares, the primary response first and the elected +content first within each, and those choices order the page and decide nothing else. A type is +rendered once per direction it travels, through its `ModelShape` per lifecycle +([TypePlan](plan/types.md#modelshape--visibility-projection)): a request shape lists what a caller +sends and a response shape what it receives, each member's requiredness read in that direction, so +a write-only property never appears in a response's table, nor a read-only one in a request's. + +## Routes and links + +Every docs target needs URLs, links between pages, an answer for two entities wanting one URL, and +a way to show one entity across API versions. Each is decided by data, never by a renderer: + +- **A catalog first, URLs second.** The emitter builds a catalog — services, groups, operations, + and the types that warrant a page — holding no URL. A route plan then derives every URL from the + catalog alone, through templates in the docs emitter's options, one per kind of page + (`"/{service}/{group}/{operation}"`). A slug is presentation (INV3), rendered from names by the + naming policy and marked `Inferred` where a default derived it. +- **A total route plan.** Every `OpID` and every type with a page gets a route, and every `PropID` + an anchor on its type's page. Anything left without one is a coded diagnostic, and a recursive + type is linked by its ID, never inlined to a depth and cut. +- **Collisions are refused.** Two entities whose routes resolve to one URL are an error + diagnostic naming both, never a page that overwrites another and never a page dropped. The check + runs on the URLs the templates produce, over every kind of route in one table, so no template can + reintroduce a collision the catalog did not have. +- **Links are requests.** A renderer never formats a URL. It writes a link request, an entity's ID + with the projection it wants (a page, an anchor, the agent document of one target), and the + route plan answers it. A spec's [`Docs.Refs`](../ir/annotations.md#references-in-prose) resolve + the same way, so a link follows its target wherever its route moves. +- **Versions keep identity.** The emitter receives one version's snapshot ([contract + rules](contract/emitter.md#contract-rules-the-type-system-and-tests-enforce)), and an entity's + page is identified by its ID, never by the version, so a version switcher links one entity + across versions. The engine computes each snapshot's route table, a pure function of the snapshot + and the options, and hands every snapshot's docs run the tables of all of them in + `EmitterInput.Routes`, for a target whose `Capabilities.CrossVersion` is set. A link to an entity + a version lacks resolves to that version's page saying so. Slugs may differ between versions; IDs + do not. + +## Documents for agents + +Beside its pages, the docs emitter writes the same content as plain Markdown for coding agents, from +the same `Doc + Plan` and through the same route plan, so an agent document and a page never +disagree about a URL: + +- **One document per operation**, holding its method and path, parameters and schemas as nested + lists, examples, code samples and links, as one aggregate document and one per SDK target, so an + agent reads only the language it writes. +- **`llms.txt` indexes** at site, service, version and target scope, each a Markdown index linking + the agent documents of its scope rather than the pages. +- **Facts, not guesses.** An operation is marked safe, or safe to retry, from `OpPlan.Effects` + ([OpPlan](plan/operations.md)). A warning that it destroys something comes only from a declared + fact, a hint or a promoted extension, since a method cannot say it: `DELETE` and `PUT` are + idempotent and may destroy. +- **Spec text is escaped.** A heading or a list item built from a summary or a description is + escaped for Markdown, and raw HTML in a description is rendered as text, never passed through. + +## Samples come from the SDK emitters + +How a call is spelled in one language — the client accessor, the method name, the arguments, a +model's constructor — is that SDK emitter's refine output, and an emitter never sees another +(INV1). A docs emitter that spelled calls itself would re-derive them and could disagree with the +SDK it documents. So each SDK emitter returns a usage sample per operation in +`EmitterOutput.Samples`, and the engine, which runs the SDK targets first, hands the docs emitter +every target's samples in `EmitterInput.Samples`. A sample a target cannot render is a coded +diagnostic of that target's, never an empty code block on the page. Each sample's values are the +plan's examples (below). + +## One set of examples + +READMEs, samples, generated tests and mock responses all need example values, and each would +synthesize its own where the document declares none, so the docs would show values the SDK's own +tests never sent, and every regeneration could change them. So the plan holds one set, +`Plan.Examples`, keyed by the ID of each type, property, parameter and operation: + +- **Declared first.** An entity's `Examples` are its set, labelled authored in the docs. +- **Synthesized only where none is declared**, under `plan.Policy.Examples`: a value valid against + the entity's type and constraints, bounded in depth and in the items it lists, marked `Inferred` + and labelled generated. Each choice the synthesizer makes, such as which enum member or which + variant, is seeded from the entity's correlation key, so the value is the same on every + regeneration until the entity's schema or the synthesizer's step version changes. A value it + cannot make valid, as a `pattern` it cannot satisfy, is reported and left out, never emitted + invalid. +- **Shared** by the docs emitter, the samples, each SDK's tests and the mock server. + +The manifest need not persist them: a seeded synthesizer reproduces each value from its inputs, and +a value that changes because its schema did is one that had to. + +## Documentation extensions + +Documents carry presentation as vendor extensions: a logo, a tag's display name, groups of tags, +code samples on an operation, a marker on an internal operation or property. They stay verbatim in +`Unmodeled`, and the docs emitter reads those its options name, as the [embedded +reader](contract/embedded-policy.md) reads policy: a key to a presentation field, the value decoded +strictly and a wrong shape reported, the entry left in place, and an empty default mapping, since +each key is one tool's spelling. A code sample a document declares is shown as authored beside the +samples the SDK emitters return. An internal marker is the filter pass's to read (architecture.md +§2.2), never the docs emitter's: hiding an operation in the docs alone would document an SDK +different from the one generated. A search index over the pages, keyed by ID, is an optional +artifact of the same emitter. + +## A command-line target + +A CLI is a registry entry over `Doc + Plan`, planned as an extension point beyond the milestones +([milestones](milestones.md)) rather than a milestone target, since nothing in the first targets +depends on it. What it needs is almost all IR and plan facts: + +| CLI behavior | Read from | +|---|---| +| `--dry-run` on a call that changes something; no confirmation on a safe one | `OpPlan.Effects` | +| A confirmation before a destructive call | a declared fact only: a hint or a promoted extension | +| Streamed output for a list | `OpPlan.Page` | +| Choices and defaults on a flag | `Enum`, `Parameter.Default`, `Property.Default` | +| Body fields as flags, with the path to put each back | the request `Payload`'s model, by `PropID` | +| Flags that exclude or require each other | the branches of a request body's `Union` | +| A flag whose value is never echoed | `Property.Secret`, `TypeCommon.Sensitive` | +| Completing an ID from a list call | `ResourceInfo` and its collection operations | + +Command names, positional arguments, `@file` inputs and hidden commands are presentation, so they +are [shaping hints](contract/shaping-hints.md) keyed by ID. Flag constraints are derived at refine +from the union's branches by `PropID`, never by a property's leaf name: a leaf name derives no +conflict between `{origin: {host}}` and `{proxy: {host}}`, and lets two branches' `host` merge into +one flag. Flag names collide as identifiers do and are told apart by the same ID-keyed resolution +([naming](refine/naming.md)), so `foo_bar` and a nested `foo.bar` do not both become `--foo-bar`. A +body field nested at any depth, or with `.` in its name, is a flag like any other, and a union +beside properties, or a top-level `anyOf` body, still yields flags, its branches' constraints among +them. diff --git a/docs/emitters/emit/templates.md b/docs/emitters/emit/templates.md index 20810fbe3..e075a709d 100644 --- a/docs/emitters/emit/templates.md +++ b/docs/emitters/emit/templates.md @@ -6,7 +6,7 @@ |---|---|---| | Models, enums, unions, discriminators, operations, clients, serializers | **Typed AST nodes** | Spec-derived *structure*; correctness must be compiler-checked; must be uniform across languages | | Runtime core: HTTP transport, retry loop, backoff+jitter, pagination iterator runtime, a stream reader per framing, the error and response types, auth injectors, logging hooks | **Templates**, one small file each, parameterised by `Policy` | Policy-driven *boilerplate* whose shape does not vary with the spec | -| Package docs, README, per-op usage examples | **Docs emitter** (`docast`, [docs and mocks](../docs-and-mock.md)) | Prose with holes | +| Package docs, README, per-op usage examples | **Docs emitter** (`docast`, [docs and mocks](../docs-and-mock.md)), each usage sample returned by the SDK emitter whose call it spells | Prose with holes; a call is spelled by the target that generates it | A retry loop is the same code for every API; only numbers and header names vary. A typed AST there buys nothing and costs readability. But a template with an `if` that inspects the IR is a design bug diff --git a/docs/emitters/milestones.md b/docs/emitters/milestones.md index e4fa90934..08ecdca76 100644 --- a/docs/emitters/milestones.md +++ b/docs/emitters/milestones.md @@ -24,6 +24,10 @@ Extends architecture.md §6 milestone 3 (the first emitters) and beyond. reusing the shared plan and step contract, forcing **no** IR or plan change. The shared plan first pays back at M3, where three targets read it. +Beyond the milestones, a command-line client is a documented extension point rather than a +scheduled target: a registry entry over `Doc + Plan`, as docs and mocks are, whose inputs and flag +rules [docs and mocks](docs-and-mock.md#a-command-line-target) sets out. No milestone depends on it. + ## What M3 lowers M3a to M3c lower the IR facts the compilers produce, and degrade the rest with a code. This scopes diff --git a/docs/emitters/plan/overview.md b/docs/emitters/plan/overview.md index c4d417b63..ca8b82aa9 100644 --- a/docs/emitters/plan/overview.md +++ b/docs/emitters/plan/overview.md @@ -15,11 +15,19 @@ func Build(doc *ir.Document, facts *irfacts.Index, pp Policy) (*Plan, []ir.Diagn type Plan struct { Ops map[ir.OpID]OpPlan // one entry per operation Types map[ir.TypeID]TypePlan // one entry per registry type + Examples map[string][]Example // by the ID of each type, property, parameter and + // operation: its declared examples, else one + // synthesized (../docs-and-mock.md#one-set-of-examples) Version string // the plan's identity (below), which the plan cache // and the generation manifest key on } const Revision = 1 // bumped with every change to what Build returns + +type Example struct { + Value ir.Example // the example, as the IR spells one + Authored bool // declared by the document; false = synthesized, Inferred +} ``` Every key is an IR stable ID — never `"METHOD /path"`, never a rendered name (INV3; the oagen diff --git a/docs/emitters/plan/plan-or-refine.md b/docs/emitters/plan/plan-or-refine.md index 11ab7211e..3bcf198ba 100644 --- a/docs/emitters/plan/plan-or-refine.md +++ b/docs/emitters/plan/plan-or-refine.md @@ -32,8 +32,8 @@ The procedure decided every existing field: 1. **Choices under `plan.Policy`** stay: `PrimaryResponse`, `RequestContent` and the content each `ResponsePlan` elects, the unwrap behind `Return.Type`, `Open` on a union and on an enum, - `Effects`, which a method sets where the IR states nothing, and `PagePlan`, an inferred one - included. + `Effects`, which a method sets where the IR states nothing, `Examples`, synthesized where none + is declared, and `PagePlan`, an inferred one included. 2. **Derivations several artifacts expose** stay: `Return`, `Others`, `Errors`, `Auth`, `Bindings`, `Stream` and `LRO`; and `Flattened`, `Required`, `Undeclared`, `Unflattened`, `Union`, `Discrim`, `Inclusive`, `Scalar`, `Shapes`, `Alias` and `Reach`. `Required`, `Undeclared` and `Unflattened` diff --git a/docs/emitters/plan/policy.md b/docs/emitters/plan/policy.md index a6554cedf..4c3c736bf 100644 --- a/docs/emitters/plan/policy.md +++ b/docs/emitters/plan/policy.md @@ -23,6 +23,14 @@ type Policy struct { // variants admit decodes to (../refine/unions.md) MethodSemantics bool // default on: an operation stating no effects takes // its method's (below) + Examples ExamplePolicy // default on, depth 3, 2 items: what may be synthesized + // where no example is declared (../docs-and-mock.md) +} + +type ExamplePolicy struct { + Synthesize bool // off leaves an entity with none declared without one + MaxDepth int // how deep a synthesized value nests + MaxItems int // how many items a synthesized list or map holds } type PrimaryRule string // "lowest_2xx" | "unconditional" | "default" diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index 31d3dca55..612f0e177 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -221,4 +221,19 @@ throughout these documents. survive byte for byte, and its table of contents lists the user's headings; and a manifest of an older `Schema` is migrated, one of a newer refused ([write / integrate](../write-integrate.md)). +39. **T-39 · Docs** — the docs emitter renders every response, error case and content an operation + declares, the primary first, and a type's request and response shapes apart, a write-only + property in no response table; every operation, page-worthy type and property has a route or + anchor, and two entities resolving to one URL is an error naming both, also when the URL + templates are changed; every link and `Docs.Refs` span resolves through the route plan; a + version switcher links one entity across snapshots and an entity a version lacks to that + version's page saying so; agent documents and `llms.txt` share the pages' URLs, escape spec + text and pass no raw HTML; and a sample comes from each SDK target, a missing one a diagnostic + ([docs and mocks](../docs-and-mock.md)). +40. **T-40 · One set of examples** — a declared example is used everywhere and labelled authored; + where none is declared, the value synthesized is valid against its constraints, within the + policy's depth and item bounds, marked `Inferred`, identical across regenerations and in both + declaration orders, and the same in the docs, the samples, an SDK's tests and the mock; one the + synthesizer cannot make valid is reported and left out + ([one set of examples](../docs-and-mock.md#one-set-of-examples)). From 9238d63173c30c0d6cad848f1ea3982fe8798a93 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:51:57 +0300 Subject: [PATCH 09/14] docs(emitters): give refiners the resolved runtime policy EmitterInput.Policy became policy.Resolved when policy gained layers and per-operation overrides, but the refiner context still held a single policy.Policy, so a step had no way to read an operation's own settings. Ctx.Policy now holds the resolved policy the emitter is given. --- docs/emitters/refine/pipeline.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/emitters/refine/pipeline.md b/docs/emitters/refine/pipeline.md index 8d2dca0e8..7c697c727 100644 --- a/docs/emitters/refine/pipeline.md +++ b/docs/emitters/refine/pipeline.md @@ -29,7 +29,8 @@ type Ctx struct { Facts *irfacts.Index // read-only derived facts, the engine's one index (../contract/emitter.md) Plan *plan.Plan // read-only shared decisions Types typemap.Table // the merged type table (../contract/type-table.md), built from Options - Policy policy.Policy // read-only SDK runtime policy (../contract/runtime-policy.md) + Policy policy.Resolved // read-only SDK runtime policy, resolved per operation + // (../contract/runtime-policy.md#layers-and-operations) Naming naming.Policy // the target's casing engine over its options' naming tables // (INV4; naming.md) Hints map[emitter.EntityID]emitter.ShapeHint From 996728b160e400a01d497e7a5c73a47b4a0a56bc Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:52:07 +0300 Subject: [PATCH 10/14] docs(emitters): say the policy input is resolved per operation --- docs/emitters/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/emitters/README.md b/docs/emitters/README.md index 257c84afa..60ecb1585 100644 --- a/docs/emitters/README.md +++ b/docs/emitters/README.md @@ -242,4 +242,4 @@ The Kotlin targets read the same plan and lower the same facts as Go. | 7 Serializable/deterministic | `Plan` + manifest are JSON-serializable, and `Facts` is rebuilt from the document; maps walked sorted, slices source order; identical input ⇒ identical bytes ([who computes the plan](contract/emitter.md#who-computes-the-plan), [the plan](plan/overview.md), [determinism](emit/printing.md#determinism-and-the-velocity-answer), [mandatory tests](testing/mandatory-tests.md)) | | 8 Optionality ≠ nullability | `LowerOptionality` renders all four states plus protobuf `Presence` as a third axis; collapses are diagnosed ([optionality](refine/models.md#optionality--nullability--presence-inv8)) | | 9 Capability surface designed against every format | every `ir.TypeKind` and every operation facet (pagination/streaming/LRO/one-way/messaging) has a lowering path or diagnosed degrade ([step list](refine/steps.md), [lowering table](refine/lowering-table.md)); docs/mock/validation are registry entries over the same `Doc + Plan`; a new language forces no IR or plan change ([docs and mocks](docs-and-mock.md), [M5](milestones.md)) | -| 10 Runtime policy separate | `policy.Policy` tree in `EmitterInput.Policy`, never on `ir.Document`; it parameterises only boilerplate; declared IR facts win over its defaults ([runtime policy](contract/runtime-policy.md)) | +| 10 Runtime policy separate | `policy.Policy` trees, resolved per operation, in `EmitterInput.Policy`, never on `ir.Document`; it parameterises only boilerplate; declared IR facts win over its defaults ([runtime policy](contract/runtime-policy.md)) | From 96d3dfa785fb76363a8b4d62382752f890775cb1 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:52:31 +0300 Subject: [PATCH 11/14] docs(emitters): give refiners the prior generation Collision resolution now keeps the names the prior generation shipped, read from its manifest, but the refiner context carried no prior state, so the naming step had nothing to read it from. Ctx.Prior carries the PriorState the emitter is given. --- docs/emitters/refine/pipeline.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/emitters/refine/pipeline.md b/docs/emitters/refine/pipeline.md index 7c697c727..4722df535 100644 --- a/docs/emitters/refine/pipeline.md +++ b/docs/emitters/refine/pipeline.md @@ -34,6 +34,8 @@ type Ctx struct { Naming naming.Policy // the target's casing engine over its options' naming tables // (INV4; naming.md) Hints map[emitter.EntityID]emitter.ShapeHint + Prior *emitter.PriorState // read-only prior generation, whose shipped names naming keeps + // (naming.md); nil on a first run Options emitter.TargetOptions // decoded options (../contract/options.md); a step asserts its own target's type Out TargetAST // the per-language AST being built (e.g. *goast.Module) Syms *SymbolTable // ID-keyed; collision resolution never touches IR identity From 7e823559c4920b54f26b48ca8238f4559e3b61f6 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:52:49 +0300 Subject: [PATCH 12/14] docs(emitters): give refiners the samples and route tables The docs emitter now receives every SDK target's usage samples and every version snapshot's route table, but the shared refiner context carried neither, so a docs refine step could not read them. Ctx holds both. --- docs/emitters/refine/pipeline.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/emitters/refine/pipeline.md b/docs/emitters/refine/pipeline.md index 4722df535..98f9d3208 100644 --- a/docs/emitters/refine/pipeline.md +++ b/docs/emitters/refine/pipeline.md @@ -36,6 +36,10 @@ type Ctx struct { Hints map[emitter.EntityID]emitter.ShapeHint Prior *emitter.PriorState // read-only prior generation, whose shipped names naming keeps // (naming.md); nil on a first run + Samples map[emitter.TargetKey]map[ir.OpID]emitter.Sample // read-only, a docs target's: each + // SDK target's samples (../docs-and-mock.md) + Routes map[string]emitter.RouteTable // read-only, a cross-version target's: every + // snapshot's routes (../docs-and-mock.md#routes-and-links) Options emitter.TargetOptions // decoded options (../contract/options.md); a step asserts its own target's type Out TargetAST // the per-language AST being built (e.g. *goast.Module) Syms *SymbolTable // ID-keyed; collision resolution never touches IR identity From 15c0c5a7cafe391606c1f9e79f627d977753ce60 Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 06:55:03 +0300 Subject: [PATCH 13/14] docs(emitters): verify a generation before, during and after it Nothing required generated output to build, a target found the constructs it could not lower one at a time inside Generate, a regeneration could not say what changed or why, nothing checked the manifest, and surface verification's records lacked the facts its verdicts turn on. The engine now checks a generation at three points. Before it, the capability-matrix rows a document exercises, a new derived fact, are compared with the rows a target declares, and each one unhonored is reported. During it, every refine step and template carries a version the manifest records. After it, a verify stage builds, lints and tests the staged output and compiles the usage samples, mapping each toolchain message to the entity whose declaration holds it, and the manifest is checked for exact coverage and for symbols that resolve in the build. Change records now carry direction, response optionality and bindings, the seeded severity table gives one verdict per channel, structural for callers' code and behavioral for deployed SDKs, and the version bump follows from them. apidiff is the test oracle for Go's structural verdicts, neither the differ nor the severity. --- docs/emitters/README.md | 2 +- docs/emitters/contract/emitter.md | 8 ++ docs/emitters/contract/registry.md | 3 + docs/emitters/refine/pipeline.md | 3 + docs/emitters/surface-verification.md | 54 +++++++++++++ docs/emitters/targets/go/overview.md | 16 +++- docs/emitters/testing/mandatory-tests.md | 20 +++++ docs/emitters/verification.md | 99 ++++++++++++++++++++++++ docs/ir/derived-facts.md | 11 +++ 9 files changed, 214 insertions(+), 2 deletions(-) create mode 100644 docs/emitters/verification.md diff --git a/docs/emitters/README.md b/docs/emitters/README.md index 5df634e39..70a73acc8 100644 --- a/docs/emitters/README.md +++ b/docs/emitters/README.md @@ -97,7 +97,7 @@ its own, and answers every construct of the [lowering table](refine/lowering-tab **Other artifacts, and how the work is held** - [Docs and mock-server emitters](docs-and-mock.md), [write / integrate](write-integrate.md), - [surface verification](surface-verification.md). + [verifying a generation](verification.md), [surface verification](surface-verification.md). - [Package layout](packages.md), [mandatory tests](testing/mandatory-tests.md), the [behavior server](testing/behavior-server.md), the [milestone plan](milestones.md) and the [open questions](open-questions.md). diff --git a/docs/emitters/contract/emitter.md b/docs/emitters/contract/emitter.md index e9620b0e5..aae9f9ec1 100644 --- a/docs/emitters/contract/emitter.md +++ b/docs/emitters/contract/emitter.md @@ -53,6 +53,12 @@ type EmitterOutput struct { Samples map[ir.OpID]Sample // an SDK target's usage sample per operation, for the docs } +// Span is the lines one top-level declaration of a file holds, 1-based and inclusive. +type Span struct { + From, To int + Origin EntityID +} + // Sample is one operation's call as an SDK target spells it. type Sample struct { Language string // the fence's language, such as "go" or "kotlin" @@ -78,6 +84,8 @@ type Artifact struct { Integrate *IntegrateHint // path-prefix stripping / tree-shake root for live-repo merge, and // whether the file is merged as blocks (../write-integrate.md) Origins []EntityID // IR IDs whose emission produced this file (drives ID-keyed prune) + Spans []Span // the lines each top-level declaration holds, by its Origin, so a + // toolchain's message maps to an entity (../verification.md) } ``` diff --git a/docs/emitters/contract/registry.md b/docs/emitters/contract/registry.md index ff280576f..c4d021bf4 100644 --- a/docs/emitters/contract/registry.md +++ b/docs/emitters/contract/registry.md @@ -40,6 +40,9 @@ type Capabilities struct { WriteIntegrate bool // supports additive merge into a live repo (../write-integrate.md) SurfaceVerify bool // produces a verify.Surface (../surface-verification.md) CrossVersion bool // reads every version snapshot's routes (../docs-and-mock.md) + Rows []string // the capability-matrix rows it honors, by key + // (../verification.md) + Verify Verify // how its output is built and checked (../verification.md) Protocols []string // "http", "grpc", "message", "graphql", "otp" Lifecycle Lifecycle // "hidden" | "experimental" | "stable" | "sunset"; required Successor TargetKey // the target replacing a sunset one; "" when none does diff --git a/docs/emitters/refine/pipeline.md b/docs/emitters/refine/pipeline.md index 98f9d3208..da15ad218 100644 --- a/docs/emitters/refine/pipeline.md +++ b/docs/emitters/refine/pipeline.md @@ -21,6 +21,9 @@ type Refiner interface { type Step interface { Name() string // stable, for diagnostics + golden step-trace tests + Version() int // bumped with every change to what it builds for some input; + // the manifest records it + // (../verification.md#during-steps-carry-versions) Apply(*Ctx) []ir.Diagnostic // appends typed nodes to the target AST under construction; idempotent } diff --git a/docs/emitters/surface-verification.md b/docs/emitters/surface-verification.md index fc7cb1574..24b27bd47 100644 --- a/docs/emitters/surface-verification.md +++ b/docs/emitters/surface-verification.md @@ -21,3 +21,57 @@ that emit performs. tracked in a distinct channel from a signature change — detectable without re-parsing because the plan already carries defaults as typed `Value`s. Every finding carries drift provenance (which stage caused it). + +## What a change record carries + +Some classifications turn on facts beyond the symbol that changed, so a neutral record carries them, +and a target's severity reads them rather than guessing: + +- **Direction.** Each record names the usages of the entity it concerns, from `Facts.Usage` + ([derived facts](../ir/derived-facts.md#usage)): an enum or a union gaining a member is additive + for a value the SDK sends and changes what a deployed SDK reading a response meets, unless the + type is `Open` ([plan policy](plan/policy.md)). A type used both ways is judged both ways, and the + more severe verdict counts. +- **Optionality in a response.** A response member going from required to optional breaks a typed + target, Go's `string` becoming `*string`, and is easy to miss, since the same change in a request + is additive. The record says which side the member is read on. +- **Bindings.** A changed method, path, parameter location or serialization style breaks a call + without changing a type, so it is a record of the behavioral channel, beside a changed default. + +Two channels answer two questions. The structural channel asks whether a caller's code still +builds against the regenerated SDK. The behavioral channel asks whether a deployed SDK, built +against the old document, still works against the API the new one describes, which a new member of +a closed response enum does not. + +## Severity, seeded + +Each target's `verify.Severity` starts from the rules every typed target shares, one verdict per +channel, and states where it departs ([Go](targets/go/overview.md#merging-and-surface-extraction), +[Kotlin](targets/kotlin/overview.md#merging-and-surface-extraction)): + +| Change | Structural: callers' code | Behavioral: deployed SDKs | +|---|---|---| +| An entity removed, or renamed with no correlation key carrying it | breaking | breaking | +| A request member or parameter added | additive if optional, breaking if required | breaking if required | +| A request member made required | breaking | breaking | +| A response member added | additive | additive | +| A response member made optional | breaking on a typed target | breaking | +| A union or enum gaining a member, read in a response | the target's: breaking where a match over it must be exhaustive, as a Kotlin `when` over an `enum class` | breaking unless `Open` | +| A union or enum gaining a member, only sent | additive | additive | +| A success status added | breaking where it changes a return ([return shapes](operations/returns.md)) | breaking | +| A failure status added | additive | additive | +| A media type removed | breaking | breaking | +| A binding changed: method, path, parameter location or style | none | breaking | +| A default changed | none | reported | + +## The version a regeneration takes + +A regeneration's version bump follows from what changed, never from a digest of the spec's text. +The structural channel sets it, since an SDK's version is a promise to its callers' code: a +breaking record makes it major, an additive one minor. A behavioral record makes it at least minor, +since the regenerated SDK behaves differently from the last. A regeneration with none of those that +still changed an artifact, through a step whose version moved +([verification](verification.md#during-steps-carry-versions)) or a document whose wording did, +takes a patch, and one that changed no artifact takes none, which the manifest's digests show +([write / integrate](write-integrate.md#what-the-manifest-records)). + diff --git a/docs/emitters/targets/go/overview.md b/docs/emitters/targets/go/overview.md index a55212f1a..84808e070 100644 --- a/docs/emitters/targets/go/overview.md +++ b/docs/emitters/targets/go/overview.md @@ -63,4 +63,18 @@ pinned exactly, `emitters/golang` alone may import it ([package layout](../../pa upgrade reruns the spike's cases as tests of the adapter. Go's extractor ([surface verification](../../surface-verification.md)) reads the existing package's -exported API, as `apidiff` does. +exported API with `go/types`, and each record it builds carries the IR ID its declaration's +`Origin` names, which no Go tool records. + +`golang.org/x/exp/apidiff` compares two Go packages and labels each change compatible or not, which +is the Go half of the structural channel's question. It is the test oracle for Go's severity, not +the differ and not the severity function: over the surface corpus, a differential test holds each +structural verdict Go's `verify.Severity` gives to apidiff's on the same two generated packages, and +a disagreement is either a severity bug or a divergence the severity's documentation states. It +cannot be the differ, since its records carry no IR ID, nor the severity, since it answers only +whether a caller's Go code builds: given an enum gaining a constant it says compatible, which it is +for a caller's code, while the behavioral channel still reports what a deployed SDK reading a closed +response enum meets ([surface +verification](../../surface-verification.md#what-a-change-record-carries)). As a test-only +dependency it needs no compatibility promise from `x/exp`, which gives none; it was under active +development, BSD-3-Clause, at the October 2026 revision measured. diff --git a/docs/emitters/testing/mandatory-tests.md b/docs/emitters/testing/mandatory-tests.md index 612f0e177..8864bb87b 100644 --- a/docs/emitters/testing/mandatory-tests.md +++ b/docs/emitters/testing/mandatory-tests.md @@ -236,4 +236,24 @@ throughout these documents. declaration orders, and the same in the docs, the samples, an SDK's tests and the mock; one the synthesizer cannot make valid is reported and left out ([one set of examples](../docs-and-mock.md#one-set-of-examples)). +41. **T-41 · Unhonored rows** — a document exercising a row a target does not honor draws + `emitters//unhonored-row` naming the row and its locations before `Generate` runs; a row + the target honors draws nothing; and every key of the capability matrix has a detector in + `irfacts.Rows` or a stated reason for none ([verification](../verification.md)). +42. **T-42 · The verify stage** — every golden output of every target builds, lints and passes its + tests in the stage, its samples compiled in the snippet project; a planted compile error is + reported at the entity whose declaration holds it, through `Artifact.Spans`; a missing or old + toolchain skips that target's checks and says so, never reading as a pass; and a failing stage + leaves the destination untouched unless the caller lets the failure through + ([verification](../verification.md#after-the-output-builds)). +43. **T-43 · The manifest checked** — every operation and type the plan emits has exactly one + manifest entry, a dropped and a doubled entry each an error; every symbol the manifest names + exists in the built output, a planted rename that skips the manifest reported + ([verification](../verification.md#after-the-manifest-says-what-the-output-holds)). +44. **T-44 · Severity and versions** — each row of the seeded severity table, on every target and in + both channels; an enum gaining a member read in a response is behavioral breaking and, on Go, + structurally additive as apidiff agrees; Go's structural verdicts equal apidiff's over the + surface corpus; and a regeneration's bump follows the rule, a moved step version that changed an + artifact a patch and one that changed nothing none + ([surface verification](../surface-verification.md#the-version-a-regeneration-takes)). diff --git a/docs/emitters/verification.md b/docs/emitters/verification.md new file mode 100644 index 000000000..f8f9e94e0 --- /dev/null +++ b/docs/emitters/verification.md @@ -0,0 +1,99 @@ +# Verifying a generation + +The mandatory tests pin bytes and behavior ([mandatory tests](testing/mandatory-tests.md)), and a +golden that does not compile is a golden nonetheless. So the engine checks a generation at three +points: before it, that each target honors what the document uses; while it runs, which versioned +steps shaped it; and after it, that the output builds and the manifest says what the output holds. +Each finding is a coded diagnostic whose severity the engine decides (INV5), so a check never fails +a run the caller did not ask it to fail. + +## Before: what a target honors + +A target that does not implement pagination, long-running operations, open enums, event streams, +file arrays in multipart or cookie parameters finds out construct by construct inside `Generate`, +and the user only after generating. So the engine compares, before generation, what the document +uses with what the target declares: + +- **What the document uses** is `irfacts.Rows`, derived on demand and never stored, as every + `irfacts` fact is ([derived facts](../ir/derived-facts.md)): each row of the [capability + matrix](../ir-spec-matrix.md) the document exercises, keyed by the row's stable key, with the + locations that exercise it. A test holds the detectors to the matrix's keys, as the conformance + corpus is held to them, so a row added to the matrix has a detector or a stated reason for none. +- **What the target honors** is `Capabilities.Rows` ([the registry](contract/registry.md)), the + keys it lowers. A row it degrades with a code (T-7) is not honored. +- **The difference** is reported per target and per row, a warning + `emitters//unhonored-row` naming the row and its first locations, before `Generate` + runs. The same two sets give a target's support matrix for its docs, generated rather than kept. + +## During: steps carry versions + +A generator changes between releases, and a user regenerating needs to know whether their SDK +changed, why, and what that does to their version number. A digest of the emitter says that +something changed and moves every SDK, whether or not the change touched anything it uses. So each +refine step and each boilerplate template carries a version, an integer bumped whenever what it +produces changes for some input, held by the step-trace golden as `plan.Revision` is held by the +plan's ([the plan's version](plan/overview.md#the-plans-version), T-10): + +- the step trace of a run says which steps fired, and the manifest records each one's version + ([what the manifest records](write-integrate.md#what-the-manifest-records)); +- a regeneration's release notes are the changelog entries of the steps it used whose versions + moved, and a moved step that changed an artifact sets at least a patch bump + ([surface verification](surface-verification.md#the-version-a-regeneration-takes)); +- a construct no step lowers is a T-7 diagnostic, as it always was. + +## After: the output builds + +`Generate` performs no I/O, so building what it returned is an engine stage of its own, run over +the generated files in a staging directory before the writer moves them into place. A generation +that fails it therefore never lands in the user's tree unless the caller lets that failure through. +Each target declares how its output is checked, in `Capabilities.Verify`: + +```go +type Verify struct { + Toolchains []Toolchain // what the checks need installed + Checks []Check // run in order, in the staged output root +} + +type Toolchain struct { + Name string // "go", "gradle" + Probe []string // the version probe, such as ["go", "version"] + Version string // a pattern capturing the version from the probe's output + Minimum string // the oldest version the generated code supports + Install string // guidance shown when it is missing or older +} + +type Check struct { + Name string // "build", "lint", "test" + Command []string // such as ["go", "vet", "./..."] + Needs []string // the toolchains it runs on + Reports string // how its messages give a position: "go", "kotlinc", "gradle" +} +``` + +- **A toolchain missing or too old** is reported with its install guidance, and that target's checks + are skipped and said to be: a check that could not run never reads as one that passed. +- **A failure maps to its entity.** The printer records which lines each top-level declaration + spans, `Artifact.Spans`, so a compiler message's file and line resolve to the entity whose + declaration holds it, and the diagnostic carries that entity's IR provenance, not just a path in + the output. +- **Samples are built too.** The usage samples each SDK target returns + ([docs and mocks](docs-and-mock.md#samples-come-from-the-sdk-emitters)) are compiled in a snippet + project the stage writes beside the SDK and never ships, so a sample the docs show is one that + builds against the SDK it documents. +- **CI runs the stage over every golden output**, so a golden that does not build fails the change + that produced it. + +## After: the manifest says what the output holds + +A manifest that is incomplete, or that names a symbol the built output lacks, is worse than none: +write/integrate, surface verification and every consumer mapping operations to SDK calls trust it. +It is built from `Origin` rather than inferred from source, so it is checked exactly: + +- **Coverage.** Every `OpID` and `TypeID` the plan says a target emits has exactly one manifest + entry, and every entry names an entity of the document. A miss or a duplicate is an error + diagnostic of the target's, since the manifest is the target's to build, with no tolerance. +- **Resolution.** After the build, every symbol the manifest names exists in the built output. The + check is the per-target extractor that [surface verification](surface-verification.md) reads an + SDK through, so it is that machinery's smallest use. A symbol missing, as a collision rename that + never reached the manifest or a keep region shadowing a generated symbol leaves one, is an error + diagnostic naming the entry. diff --git a/docs/ir/derived-facts.md b/docs/ir/derived-facts.md index 94ed246b7..f3aa1fcda 100644 --- a/docs/ir/derived-facts.md +++ b/docs/ir/derived-facts.md @@ -90,3 +90,14 @@ value of exactly that type can hold, at any depth, one of the same shape — and the types whose shapes share its strongly connected component. A reference to a polymorphic base counts every subtype it may hold, so a subtype holding its own discriminated base is recursive. A cycle through composition alone, which the IR permits, is reported too. + +## Capability rows + +`Rows()` answers which rows of the [capability matrix](../ir-spec-matrix.md) a document exercises: +for each row's stable key, the locations of the nodes that exercise it, sorted, such as every +operation whose `Pagination` is set for `pagination`. One detector per row reads the facts the row +names, and a test holds the detectors to the matrix's keys, as the conformance corpus is held to +them: a row added to the matrix has a detector, or a reason no IR fact shows it. The engine reads +`Rows` to tell a user, before generation, which rows a target does not honor +([verification](../emitters/verification.md#before-what-a-target-honors)). + From e80fae50fb45be27ef0fadee443410bcce95b75c Mon Sep 17 00:00:00 2001 From: OmarAlJarrah Date: Mon, 2 Nov 2026 07:01:16 +0300 Subject: [PATCH 14/14] docs(emitters): put the oneOf match rule under recognition The rule saying what a oneOf decodes a value two variants admit to landed at the end of the section on a variant's null, which it has nothing to do with. It belongs with how a variant is recognized, which is also where T-35 links it. --- docs/emitters/refine/unions.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/emitters/refine/unions.md b/docs/emitters/refine/unions.md index a19f18d21..7f920a56d 100644 --- a/docs/emitters/refine/unions.md +++ b/docs/emitters/refine/unions.md @@ -47,6 +47,13 @@ below, which keeps its bytes as they arrived — a degrade, never a whole-docume ([T-7](../testing/mandatory-tests.md)). That graceful-degradation margin is exactly what the ABI seam buys and the fused-pipeline generator cannot have. +A `oneOf` admits exactly one variant, so a value two of its variants admit breaks the document. What +it decodes to is `plan.Policy.ExclusiveMatch`'s ([plan policy](../plan/policy.md)): under `first`, +the default, the variant recognition finds first, as a lenient reader of a document whose variants +overlap needs; under `only`, a coded decode error, found by trying every variant whose kinds hold +the value, as an inclusive union does below. A `oneOf` whose variants the plan proves disjoint +([TypePlan](../plan/types.md#variants-that-can-share-a-value)) never meets the choice. + ## A null a variant admits A variant admits `null` when its reference is `Nullable` or its type is the `null` literal. `null` @@ -58,13 +65,6 @@ give a null member (#618) then lower alike. The golden spec's single-result enve `result` as `anyOf: [{type: object, nullable: true}, {type: string, nullable: true}]`: an object, a string or `null`, the last held by the reference. -A `oneOf` admits exactly one variant, so a value two of its variants admit breaks the document. What -it decodes to is `plan.Policy.ExclusiveMatch`'s ([plan policy](../plan/policy.md)): under `first`, -the default, the variant recognition finds first, as a lenient reader of a document whose variants -overlap needs; under `only`, a coded decode error, found by trying every variant whose kinds hold -the value, as an inclusive union does below. A `oneOf` whose variants the plan proves disjoint -([TypePlan](../plan/types.md#variants-that-can-share-a-value)) never meets the choice. - ## A value several variants admit An `anyOf` lets one value satisfy several variants, and what the union holds then turns on