Repository navigation
docs(ir): answer what a non-emitter consumer relies on - #1351
Merged
Merged
Conversation
An anyOf lets one value satisfy several variants, and the emitter design lowered every untagged union to a closed sum that holds exactly one. Where two variants admit the same value, holding the first misnames it and drops the members only the others declare. The plan now proves, once per union, whether two variants of an anyOf can admit one value (TypePlan.Inclusive), by JSON kind, by finite value sets and by a member one model requires that the other cannot hold. A union it proves disjoint lowers as a closed sum, as a oneOf does. One it cannot is inclusive: it holds an optional member per variant, set for each variant the value admits, decodes into every candidate variant, and encodes the merge, a disagreement being an encode error. A null a nullable variant admits is held by the reference, never by a variant. Go lowers an inclusive union to a struct with a pointer per variant and Kotlin to a class with a nullable property per variant. T-29 pins the round trip of a value two open models admit.
A positional model is an array on the wire, so the kinds rule now says so and the member rule reads only models laid out by name. A stream that ends on a terminal variant now ends on an element holding one, among the several an inclusive union holds. A null literal variant now lowers as a nullable variant does, so either spelling of a null member reaches the reference. Inclusive joins the plan fields the plan-or-refine test keeps, and the docs emitter says when a value may be several variants at once.
plan.Policy held a function, PrimaryResponse, and Plan.Version hashed the policy. A function has no stable bytes: json/v2 refuses it, its printed form is an address that changes every run, and skipping it gives two policies that pick different responses one version. The policy is now data from a closed vocabulary: PrimaryResponse is an ordered list of rules and DefaultFails says whether an unpicked Default is the catch-all failure. Version hashes the document's digest without sources, a new plan.Revision bumped with every change to Build's output, and the policy's deterministic JSON. EnvelopeDetect named a type nothing defined. plan.Policy.Envelopes now lists envelopes a configuration names, none by default. Each matches a body by the bases it reaches or by the members it declares, and says where the value sits and which flag value means the call failed. A matched success returns that member, keeps the envelope reachable through WithResponse, and turns a 2xx whose flag says it failed into a failure with no fault or kind, decoded as the operation's matching error body. Failures and pages are read whole. T-30 pins the plan's identity and T-31 the envelope reads.
…into docs/emitters-envelopes-policy-data # Conflicts: # docs/emitters/docs-and-mock.md # docs/emitters/testing/mandatory-tests.md
Runtime policy was one tree per project, so an operation needing its own timeout or retry had nowhere to say so, and a document's own SDK configuration, kept verbatim as vendor extensions, would have been parsed by each emitter its own way. Policy and hints are now layers, each holding a document-wide override and overrides by operation: what a document embeds, the project's, the emitter's. A setting takes the highest layer that sets it, an operation's value over its layer's document-wide one, and a declared IR fact beats every layer. A new emitters/embed reader turns vendor extensions a configuration maps into the lowest layer of each, decoding each value strictly and reporting what it cannot read. The vocabulary gains connection-error retries, Retry-After, a throttling backoff and redirects. Whether an operation is safe or idempotent is a plan fact, OpPlan.Effects: declared, else from its HTTP method. An enum a client reads keeps a value it does not list, as a union does, and a oneOf value two variants admit decodes as plan.Policy.ExclusiveMatch says. A service hint makes a parameter a client setting; nothing is hoisted by inference, since a shared parameter stops being shared as an API grows, and the plan only reports candidates.
An enum whose value type has no Go constant form, a date, a uuid or a decimal, had no Go lowering, and the type table built an enum value by decoding its wire form, which no constant can be. Kotlin's enum class writes every member as its @SerialName string, so a boolean or numeric enum went out as a string its schema does not admit. A member is now written as the JSON kind its value type has, and decoded by comparing as JSON compares values, numbers by value. Go declares an enum over its value type's row, and over the wire text where the row has no constant, the codec writing the value type's JSON kind and a Value method giving the row's type. Kotlin generates a KSerializer for every enum that is not a string enum, and an entry of a mapped type holds its value as that type. T-36 pins every value type on every target.
Kotlin's naming document stated only the members it gives every type, so a Kotlin emitter had no names to reserve for a group's module or for the declarations a unit holds beside the spec's, and T-19 no fixture. Measured with Kotlin 2.4.0 and Gradle 9.8.2: a module named build sits in the root project's output, which clean deletes with the module's sources; one named buildsrc is taken for Gradle's buildSrc on a case-insensitive file system and fails the build; gradle and src share the wrapper's and the root sources' directories. Those four are the units set, and a group's module is placed at the build root so the set applies. Hard keywords need no reservation: backticked as package segments they compile on the JVM, JS and Native and link into a framework, and kotlin is refused only as the first segment, which the options name. The unit set is Client, a group's sub-client and the runtime names the root unit declares.
The manifest recorded hashes, a file list and the entity-to-symbol map, too little for a regeneration to honor what the last one did: an edited generated file was overwritten, a no-op rewrote every file, a reformatted spec looked new, and keep regions were found by path. The manifest now records the document's digest instead of the spec's, a normalized digest per written file, the options version, the steps that ran, files written once, each entity's correlation key, base name and symbol, and each keep region by its entity, under a schema version migrated forward. Regions follow their entity to its new file. Every ID-keyed input that matches nothing is reported with its likely new home. A README is merged as blocks the user writes around. Collisions now read the prior manifest: a shipped symbol keeps its name while its ID and correlation key agree, and only newcomers take a suffix, so adding an endpoint renames nothing. The prior manifest joins the inputs the purity rule names. A spike at Go 1.27.3 chose dst for the Go merge adapter: go/ast nodes moved between files lost their doc comments, one into a function body, while dst and a text splice were both correct, and only dst reaches inside a declaration. Claims that IDs survive any rename are narrowed to renames that keep the pointer.
The docs emitter reused the SDK's primary response and content, had no design for URLs, links, collisions or versions, spelled usage examples it could only re-derive, and left documentation extensions and agent documents unaddressed. Every artifact would also synthesize its own example values. The docs now render every response, error case and content, the primary only ordering them, and each type's request and response shapes apart. A route plan derives every URL from an ID-keyed catalog through templates, refuses any two entities on one URL after the templates apply, resolves links as ID requests, and keeps one entity's identity across version snapshots through route tables the engine passes in. Agent Markdown and llms.txt share those routes and escape spec text. SDK emitters return a usage sample per operation for the docs, and the plan holds one set of examples, declared first and otherwise seeded from each entity's correlation key, for docs, samples, tests and mocks. Documentation extensions are read by mapping, internal markers left to the filter pass, and a CLI is planned as an extension point built from IR facts.
EmitterInput.Policy became policy.Resolved when policy gained layers and per-operation overrides, but the refiner context still held a single policy.Policy, so a step had no way to read an operation's own settings. Ctx.Policy now holds the resolved policy the emitter is given.
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
Collision resolution now keeps the names the prior generation shipped, read from its manifest, but the refiner context carried no prior state, so the naming step had nothing to read it from. Ctx.Prior carries the PriorState the emitter is given.
…docs/emitters-docs-artifacts
The docs emitter now receives every SDK target's usage samples and every version snapshot's route table, but the shared refiner context carried neither, so a docs refine step could not read them. Ctx holds both.
Nothing required generated output to build, a target found the constructs it could not lower one at a time inside Generate, a regeneration could not say what changed or why, nothing checked the manifest, and surface verification's records lacked the facts its verdicts turn on. The engine now checks a generation at three points. Before it, the capability-matrix rows a document exercises, a new derived fact, are compared with the rows a target declares, and each one unhonored is reported. During it, every refine step and template carries a version the manifest records. After it, a verify stage builds, lints and tests the staged output and compiles the usage samples, mapping each toolchain message to the entity whose declaration holds it, and the manifest is checked for exact coverage and for symbols that resolve in the build. Change records now carry direction, response optionality and bindings, the seeded severity table gives one verdict per channel, structural for callers' code and behavioral for deployed SDKs, and the version bump follows from them. apidiff is the test oracle for Go's structural verdicts, neither the differ nor the severity.
The rule saying what a oneOf decodes a value two variants admit to landed at the end of the section on a variant's null, which it has nothing to do with. It belongs with how a variant is recognized, which is also where T-35 links it.
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
…docs/emitters-docs-artifacts
…to docs/emitters-verification
A catalog, a linter or a diff reads the IR as an input format, and the design documents answered its questions only for emitters. This states the answers where such a consumer will look: - identity across revisions: an ID holds only while its pointer does, and an array index can hand an old ID to another entity, so matching across revisions rests on a correlation key derived from what the document names an entity by (designed in derived-facts.md); - what a key in Unmodeled promises: a vendor extension's name is the source's, every other part of a key is the compiler's spelling, and a change to it moves IRVersion as a renamed symbol does; - what to pin, and when a stored document must be recompiled; - why the compiler has no provenance-trimmed mode, and that a slice of a document is the planned filter pass. Open question 5 now records the answer and leaves only the structural tier open. The emitter documents cite the new sections in place of the issues they close.
Building the correlation keys against the golden spec showed three rows of the design wrong. Servers there share URL templates, so a server is keyed by the scope naming it as well. A header carries a property ID of its own, so it is an entity to key, not only a type. And two declarations of one wire name in one composed model would leave both unkeyed, though the flattening takes one, so the key follows the flattening. A variant has no ID, so the table now keys its type rather than the variant.
This was referenced Nov 2, 2026
…ata' into docs/emitters-policy-layers
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
…docs/emitters-docs-artifacts
…to docs/emitters-verification
… docs/ir-consumer-contract
5 of 6 tasks
…into docs/emitters-envelopes-policy-data
…ata' into docs/emitters-policy-layers
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
…docs/emitters-docs-artifacts
…to docs/emitters-verification
… docs/ir-consumer-contract
This branch stood on docs/emitters-verification (#1339), the top of the emitter design stack, because five of its edits rewrote lines #1336 wrote. Its own changes need nothing from that stack, so this merge takes main's side of every file the stack changed and keeps only this branch's changes, applied to main: - The citations of #679, #678 and #419 it replaced in plan/overview.md, refine/naming.md, surface-verification.md and write-integrate.md exist only on #1336's branch, so those edits drop out. #1336 can link the new sections once this lands. - identity.md's opening paragraph is rewritten from main's wording, not #1336's, and "Across revisions" no longer cites the prior manifest's section, which only #1336 adds. - The correlation keys section ends derived-facts.md, which on main has no capability rows section before it, and the README row no longer names one. - The #429 citation in embedded-policy.md is replaced in main's wording of the paragraph.
#1327 moved a path item's own summary and description to openapi:pathItem/summary and openapi:pathItem/description, at IRVersion 0.30.0. Compiling a path item that declares both, with two operations, puts both keys on each of them, which is where the table now names them.
This was referenced Nov 5, 2026
The design keyed an operation by its operationId, and the golden spec's own history shows that failing both ways. Between revisions c92b9b0f and 319dc756, nine operationIds moved to new endpoints while the old endpoints kept their routes and IDs under new names, so the key matched each old endpoint to another; nineteen operations kept route and ID under a renamed operationId, which the key never matched to themselves. An operation is now keyed by its method and route, and by its declared name only where no HTTP binding places it or another operation shares its route; callbacks stay keyed through the operation declaring them. The key is derived by code, not stored in the document, so nothing said when two keys are comparable. It is opaque and compared only with a key the same derivation produced, and irfacts.CorrelationVersion names the derivation for a consumer that stores keys. Also: - key a multipart part's headers, which no row covered; - name the type key's namespace as TypeCommon.Namespace, not the ID's; - state that a key passing through an array index moves with the array, as an allOf branch's and a tag's do, and that retiring a compiler-spelled key moves IRVersion; - count IDs through type and enum arrays too: 10,122 types, not 9,957; - list Property.Examples, which holds most examples on the golden spec; - say a consumer comparing two documents compiles both with one build, since a fix within one generation can lower a source differently; - prune a filter pass's slice by what its kept nodes reach: usage alone skips an encoding's wire type and leaves it dangling.
# Conflicts: # docs/ir/identity.md
The emitter documents described the correlation key as an operationId that carries an operation across a path rename. An operation's key is now its method and route (ir/derived-facts.md#correlation-keys), so what it carries across is a renamed path variable, and a renamed path changes it as a renamed component does. The plan, naming, surface verification, write/integrate, T-8, T-39 and T-40, and the prior-art and reference-learnings notes now say so. The manifest records irfacts.CorrelationVersion as Keys, the derivation its keys came from, and a prior manifest of another derivation is named as a first generation, as one run with key matching turned off is. Two statements the key's design no longer supports go: that a key need not be unique in its document, and that a pass computes it.
- The named-type row no longer spells a key segment: keys are opaque, and the operation row had already dropped its own. - What to pin says IRVersion moves with the Unmodeled keys a compiler spells, as ir-version.md does; a vendor extension's name it writes moves only with the source. - What a key promises names a coordinate as the source's: a declaration's pointer, which moves with the declaration. - Naming states the INV6 switch for key matching as an engine option, where it had named a pass that the derived key no longer has.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The IR's design documents answer every question as an emitter would ask it. A catalog, a linter or a diff that takes the IR as its input format asks other ones: does an ID mean the same thing in the next revision, can I read this
Unmodeledkey next month, what do I pin, and why is the document this large. This PR answers them where such a consumer will look, and settles how an entity is matched across revisions.docs/ir/identity.md#across-revisions). An ID identifies an entity within one document, and across revisions only while its pointer holds still. Renaming a position gives new IDs. A reordered array is worse: it hands an old ID to a different entity, so a consumer matching by ID silently attaches one entity's state to another. On the golden spec, 10,122 of 41,697 types, 13,466 of 32,291 enum members, 5,871 of 49,434 properties and all 8 servers have an ID that passes through an array index, counted by thejqcommand the section quotes.docs/ir/derived-facts.md#correlation-keys). What carries an entity across revisions: a key built from what the document names the entity by, derived on demand inirfactsand never stored, and none where two entities share one, because a wrong match is the failure the key exists to prevent. Three decisions in it:operationIdonly where no HTTP binding places it or two operations share a route. On the golden spec's own history anoperationIdkey matches an endpoint to a different one where a revision reassigns the name, and refuses an unchanged route under a renamedoperationId; the route does neither (measurements below).irfacts.CorrelationVersion. A consumer that stores keys records it beside them, and a stored key from another derivation matches nothing.IRVersionis not that version: it refuses a document, and a change to the derivation changes none.Unmodeledkey promises (docs/ir/unmodeled.md#what-a-key-promises,docs/ir/ir-version.md). A vendor extension's name and a name the document gives are the source's. An array index in a key, as inopenapi:allOf/1or a tag'sopenapi:tags/0/x-mark, is the source's position: it moves with the array and can then name another construct's entry. Every other part of a key is the compiler's spelling, so renaming, retiring or moving one movesIRVersion, as a renamed symbol does. Ano_ir_homeentry leaves in the change that adds its field, and that generation's history entry names the key. No test holds the vocabulary yet: test: pin the Unmodeled keys a compiler spells to IRVersion #1350 is filed for a pin like the name pin.docs/ir/consumers.md). What a reader finds and where, including what has no typed home yet (ir: decide whether a path item's summary and description get a typed home #1319, ir: response links have no home #669), and examples onProperty.Examples, which holds most of the golden spec's. What to pin, and when to recompile: two builds of one generation read each other's documents, but a fix between them can lower one source differently, so a diff compiles both sides with one build. The size decision: no provenance-trimmed mode, since a document stripped of provenance still passesirverifywhile proving less, and a slice is the planned filter pass. That pass must prune by what its kept nodes reach. Usage alone skips a reference like an encoding's wire type and leaves it dangling.operationIdkeeps its key. They now say what the route carries, a renamed path variable, and that a renamed path changes the key as a renamed component does: the plan, naming, surface verification, write/integrate, T-8, T-39 and T-40, and the prior-art and reference-learnings notes. The manifest recordsKeys, the derivation of its keys. A prior manifest of another derivation is named as a first generation.Why the route
Between golden-spec revisions c92b9b0f (2026-07-27) and 319dc756 (2026-10-30), both compiled by morphic @4ac3031d, keys derived by #1354 @e9f17079:
operationIdkeyoperationIdreassigned, the old endpoint still present under a new nameFrom ef715956 (2026-04-26) to c92b9b0f, 48 operations gained their first
operationIdon an unchanged route, which moves anoperationIdkey and every key built on it. Route keys were unique in each revision, as OpenAPI requires.Closes #418
Closes #419
Closes #429
Closes #430
#678 stays open. It asks for two things: state the hazard in
identity.mdandopen-questions.md, which this PR does, and have whatever keys state by ID across revisions check the correlation key too. The manifest onmainnow does that, but a shaping hint or a runtime-policy key written against an ID through an array index is still applied to whatever entity takes that ID next.#432 is answered except its first question: no release is tagged, so the pin is a commit, resolved by Go as a pseudo-version. Cutting a tag is a release decision this PR does not take, so #432 stays open for it.
#1354 builds the key and is stacked on this PR. Before it merges it follows this design: the operation key, the row keying a multipart part's headers, and
irfacts.CorrelationVersion.The branch has more than one commit, so it lands under its title, which fits within 72 characters with the number appended. The commit subjects, which the squash replaces, each fit within 72 without it.
Test plan
jqcommandidentity.mdquotes, run as printed. The sizes come from the compact and gzip-6 encodings ofmorphic compile -ooutput and of the source. The ID and provenance shares are the value bytes of theidandprovenancemembers in the type registry.operationId, by route and by ID. Every entity's key from feat(irfacts): derive a correlation key for every entity #1354 @e9f17079 was compared across c92b9b0f and 319dc756.Property.Examplesholds 8,429 of the golden spec's 17,749 example-holding nodes. Aformat: byteproperty's wire type has usage 0, and pruning every type of usage 0 leavesir/dangling-type-ref.testdata/conformance/openapi/schema-dialects.yamlcompiles tomax: "100"at b06b880 andexclusiveMax: "100"at 4ac3031, both 0.32.0. A multipart part's header, its type and its members get no key from feat(irfacts): derive a correlation key for every entity #1354's derivation. The golden spec writesopenapi:allOf/1201 times. A literal type is minted atenum/N, and the corpus mints types attype/N.main: a path item with a summary, a description and two operations, an extension, a response link, a multipart part's extension and aninfo.contactextension compile to the keys the documents cite, and thesortparameter's schema tot/anon/paths/~1things/get/parameters/0/schema.testdata/golden/openapi/petstore.yamland the golden spec, stripped of everyprovenancemember and decoded again, give noirverifyfinding either way.main@c3e52444:go test ./internal/archtestpasses,skills_lintis clean, and the push checklist finds no artifact, no conflict and no subject past 72.