Repository navigation
docs(emitters): define what a generation keeps for the next one - #1336
Merged
Merged
Conversation
An anyOf lets one value satisfy several variants, and the emitter design lowered every untagged union to a closed sum that holds exactly one. Where two variants admit the same value, holding the first misnames it and drops the members only the others declare. The plan now proves, once per union, whether two variants of an anyOf can admit one value (TypePlan.Inclusive), by JSON kind, by finite value sets and by a member one model requires that the other cannot hold. A union it proves disjoint lowers as a closed sum, as a oneOf does. One it cannot is inclusive: it holds an optional member per variant, set for each variant the value admits, decodes into every candidate variant, and encodes the merge, a disagreement being an encode error. A null a nullable variant admits is held by the reference, never by a variant. Go lowers an inclusive union to a struct with a pointer per variant and Kotlin to a class with a nullable property per variant. T-29 pins the round trip of a value two open models admit.
A positional model is an array on the wire, so the kinds rule now says so and the member rule reads only models laid out by name. A stream that ends on a terminal variant now ends on an element holding one, among the several an inclusive union holds. A null literal variant now lowers as a nullable variant does, so either spelling of a null member reaches the reference. Inclusive joins the plan fields the plan-or-refine test keeps, and the docs emitter says when a value may be several variants at once.
plan.Policy held a function, PrimaryResponse, and Plan.Version hashed the policy. A function has no stable bytes: json/v2 refuses it, its printed form is an address that changes every run, and skipping it gives two policies that pick different responses one version. The policy is now data from a closed vocabulary: PrimaryResponse is an ordered list of rules and DefaultFails says whether an unpicked Default is the catch-all failure. Version hashes the document's digest without sources, a new plan.Revision bumped with every change to Build's output, and the policy's deterministic JSON. EnvelopeDetect named a type nothing defined. plan.Policy.Envelopes now lists envelopes a configuration names, none by default. Each matches a body by the bases it reaches or by the members it declares, and says where the value sits and which flag value means the call failed. A matched success returns that member, keeps the envelope reachable through WithResponse, and turns a 2xx whose flag says it failed into a failure with no fault or kind, decoded as the operation's matching error body. Failures and pages are read whole. T-30 pins the plan's identity and T-31 the envelope reads.
…into docs/emitters-envelopes-policy-data # Conflicts: # docs/emitters/docs-and-mock.md # docs/emitters/testing/mandatory-tests.md
Runtime policy was one tree per project, so an operation needing its own timeout or retry had nowhere to say so, and a document's own SDK configuration, kept verbatim as vendor extensions, would have been parsed by each emitter its own way. Policy and hints are now layers, each holding a document-wide override and overrides by operation: what a document embeds, the project's, the emitter's. A setting takes the highest layer that sets it, an operation's value over its layer's document-wide one, and a declared IR fact beats every layer. A new emitters/embed reader turns vendor extensions a configuration maps into the lowest layer of each, decoding each value strictly and reporting what it cannot read. The vocabulary gains connection-error retries, Retry-After, a throttling backoff and redirects. Whether an operation is safe or idempotent is a plan fact, OpPlan.Effects: declared, else from its HTTP method. An enum a client reads keeps a value it does not list, as a union does, and a oneOf value two variants admit decodes as plan.Policy.ExclusiveMatch says. A service hint makes a parameter a client setting; nothing is hoisted by inference, since a shared parameter stops being shared as an API grows, and the plan only reports candidates.
An enum whose value type has no Go constant form, a date, a uuid or a decimal, had no Go lowering, and the type table built an enum value by decoding its wire form, which no constant can be. Kotlin's enum class writes every member as its @SerialName string, so a boolean or numeric enum went out as a string its schema does not admit. A member is now written as the JSON kind its value type has, and decoded by comparing as JSON compares values, numbers by value. Go declares an enum over its value type's row, and over the wire text where the row has no constant, the codec writing the value type's JSON kind and a Value method giving the row's type. Kotlin generates a KSerializer for every enum that is not a string enum, and an entry of a mapped type holds its value as that type. T-36 pins every value type on every target.
Kotlin's naming document stated only the members it gives every type, so a Kotlin emitter had no names to reserve for a group's module or for the declarations a unit holds beside the spec's, and T-19 no fixture. Measured with Kotlin 2.4.0 and Gradle 9.8.2: a module named build sits in the root project's output, which clean deletes with the module's sources; one named buildsrc is taken for Gradle's buildSrc on a case-insensitive file system and fails the build; gradle and src share the wrapper's and the root sources' directories. Those four are the units set, and a group's module is placed at the build root so the set applies. Hard keywords need no reservation: backticked as package segments they compile on the JVM, JS and Native and link into a framework, and kotlin is refused only as the first segment, which the options name. The unit set is Client, a group's sub-client and the runtime names the root unit declares.
The manifest recorded hashes, a file list and the entity-to-symbol map, too little for a regeneration to honor what the last one did: an edited generated file was overwritten, a no-op rewrote every file, a reformatted spec looked new, and keep regions were found by path. The manifest now records the document's digest instead of the spec's, a normalized digest per written file, the options version, the steps that ran, files written once, each entity's correlation key, base name and symbol, and each keep region by its entity, under a schema version migrated forward. Regions follow their entity to its new file. Every ID-keyed input that matches nothing is reported with its likely new home. A README is merged as blocks the user writes around. Collisions now read the prior manifest: a shipped symbol keeps its name while its ID and correlation key agree, and only newcomers take a suffix, so adding an endpoint renames nothing. The prior manifest joins the inputs the purity rule names. A spike at Go 1.27.3 chose dst for the Go merge adapter: go/ast nodes moved between files lost their doc comments, one into a function body, while dst and a text splice were both correct, and only dst reaches inside a declaration. Claims that IDs survive any rename are narrowed to renames that keep the pointer.
4 of 5 tasks
EmitterInput.Policy became policy.Resolved when policy gained layers and per-operation overrides, but the refiner context still held a single policy.Policy, so a step had no way to read an operation's own settings. Ctx.Policy now holds the resolved policy the emitter is given.
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
Collision resolution now keeps the names the prior generation shipped, read from its manifest, but the refiner context carried no prior state, so the naming step had nothing to read it from. Ctx.Prior carries the PriorState the emitter is given.
The rule saying what a oneOf decodes a value two variants admit to landed at the end of the section on a variant's null, which it has nothing to do with. It belongs with how a variant is recognized, which is also where T-35 links it.
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
This was referenced Nov 2, 2026
…into docs/emitters-envelopes-policy-data
…ata' into docs/emitters-policy-layers
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
This was
linked to
issues
Nov 4, 2026
The base branches this one was stacked on have landed on main, so main's side stands wherever they and this branch both changed a line, and this branch's own sentences are re-applied over it. Main's T-37 and T-38 took the numbers this branch gave its two tests, which follow as T-39 and T-40.
The key planned in #679 is an operation's operationId, else its method and path, and a component's name. A path renamed under an unchanged operationId keeps its key, so the key carries it across; a component renamed changes its ID and its key together, which only the structural tier of docs/ir/open-questions.md, question 5, could carry. Several statements said a component rename was carried by the key too, and prior-art.md still said IDs were durable across path renames. Each now says which renames the key carries. architecture.md's summary of the manifest named spec hashes, which the manifest no longer records, and milestones.md now says that M4 needs the key, since shipped names, regions, files written once and unmatched keys all match on it.
A prior entry was matched within its unit, but names collide within a scope: a client's methods, a type's members, a unit's declarations. By default every group's client is in the root unit, so an operation moved to another group kept its shipped get beside the get that group's client had shipped, and the generation declared two. An entry now records the scope its name was resolved in, and that is what matches. Matching by ID alone also left a path renamed under an unchanged operationId named afresh, though the manifest says the correlation key carries that rename. Where an entity's ID is new, its entry is the one recording its key under an ID the document no longer has, unless two new entities share that key. A symbol the scope now reserves is not taken again. On Multiplatform a declaration's Objective-C name is resolved in a namespace every unit shares, and Swift callers depend on it, so it is an entry of its own. T-39's newcomer now sorts before the incumbent in ID order: one that sorted after it kept the incumbent's name by ID order alone, so the test passed with the prior manifest ignored. It also covers the moved operation, the path rename and the Objective-C name.
A file's digest covered its keep regions, so an edit inside one, which regions exist for, read as an edit outside them, and the writer stopped regenerating the file. The digest now leaves each region's text out. An edited file keeps its prior entry, so the next generation reports it again rather than taking the edit as written. The document's digest left out sources but not diagnostics, unlike the Plan.Version digest it claims to be. A reformatted spec that draws one warning moves that warning's position, so it was a new generation: compiled twice, the documents differ with sources left out and match with diagnostics left out too. The emitter was named by a version no document defines. It is now the Morphic version and Go release from the build information, the same on every machine that shares the manifest, and a version naming no one tree never matches, so a change being developed is never skipped. A scoped run keeps the prior entries it leaves out, so the next whole run does not lose the names and files it did not touch. A region moved with its entity was always written after the declaration, which takes a member a user added inside a class out of it. A region records where it stood, and two of one name in one entity are reported. Regions are sorted, as files are. In a README, a block new to a generation and first in its order went to the end. It now goes beside its neighbours. A block ID built from an IR ID can hold a space or "-->", as a tag's does, so it is written percent-encoded. T-6 and printing.md name the prior manifest among the inputs output is a function of, Steps cites #702, which plans step versions, and plan/overview.md no longer defers to #713, which this change closes.
Three codes reported one condition, an ID an input names outliving its entity: emitters/unmatched-key for hints and overlays, and, since this branch was cut, emitters/policy/unknown-operation for a runtime-policy key and emitters/plan/envelope-unused for an envelope's Except or Bases entry. Naming where the entity went, as #716 asks, needs the prior manifest, which neither Resolve nor plan.Build takes. The engine holds every input and the prior manifest, so it matches each such ID against the document once, before any stage reads it, and reports each that matches nothing as emitters/unmatched-key, naming the entity that now holds its correlation key. Resolve and Build pass over such an ID. envelope-unused keeps an envelope no body matches, a different fact. The reasoning both codes gave for a warning, that one configuration may serve several documents, carries over. A Remount or Mount group and a ParamKey are checked the same way.
Rule 1 took a shipped symbol again only where its scope does not reserve it, but rule 2's suffixes had no such check, and a suffix can make a reserved name: two groups rendering com on Kotlin would give the second com2, a Windows device name Kotlin's units reserve. Rule 2 now takes the lowest suffix the scope neither holds nor reserves, and T-39 covers it. prior-art.md said a path rename is the correlation key's to carry, and shaping-hints.md that a hint keyed by a renamed path is reported with its new home. An operation with no operationId is keyed by its method and path, so its key changes with the path; both now say where the key holds. surface-verification.md and prior-art.md's oagen paragraph are reflowed where earlier edits left them ragged.
A symbol derived from an entity, such as an operation's request type, records the entity's correlation key. Once the operation's path is renamed, the old ID's own entry and its derived entries share that key, so "the entry recording its key" named several, and the renamed incumbent lost get and GetRequest to a newcomer sorting before it. An entity is now matched by key among entity entries alone, and a symbol derived from it takes the entry derived the same way from its match. naming.md defines this prior entry once, before the rules, and a keep region and a file written once follow their entities by it, without the scope and Base conditions that only govern keeping a symbol. T-39 covers the derived symbols. An unmatched key was said never to abort a run, but a caller whose threshold counts warnings fails the run, as architecture.md §4 says, and the issue asks for a severity the caller controls. It now says so. A file written once is an artifact with SkipIfExists, which the manifest records as Once; the two were never tied together. The manifest's file and entity lists hold what a scoped run or an edited file keeps from the prior manifest, not only what this run wrote, and docs-and-mock.md says the docs emitter's README is integrated as blocks.
The staleness check subtracts the current entities from the prior manifest's by ID, so an operation whose path was renamed under an unchanged operationId, its old ID gone though the correlation key carries it, was listed as dead code. A prior entity whose entry is a current entity's prior entry now counts as current, and T-40 says so. The README's "everything cross-run correlates by IR stable ID" names the correlation key too. Most of write-integrate's reports had no severity, and the severity decides whether a run fails. Each now has one, by the rule that an error is kept for lost meaning that decides behaviour: an edited file, a region whose entity is gone and a second region of one name in one entity are warnings, since nothing is lost, and a README block removed is an info. A manifest from a newer Morphic is an error with nothing written, since a generation without it would rename what was shipped. Two regions of one name stay where they are, their file kept, as no key can say which goes where.
The previous commit left the sentence's first line past the documents' 100-column width.
OmarAlJarrah
added a commit
that referenced
this pull request
Nov 5, 2026
This branch stood on docs/emitters-verification (#1339), the top of the emitter design stack, because five of its edits rewrote lines #1336 wrote. Its own changes need nothing from that stack, so this merge takes main's side of every file the stack changed and keeps only this branch's changes, applied to main: - The citations of #679, #678 and #419 it replaced in plan/overview.md, refine/naming.md, surface-verification.md and write-integrate.md exist only on #1336's branch, so those edits drop out. #1336 can link the new sections once this lands. - identity.md's opening paragraph is rewritten from main's wording, not #1336's, and "Across revisions" no longer cites the prior manifest's section, which only #1336 adds. - The correlation keys section ends derived-facts.md, which on main has no capability rows section before it, and the README row no longer names one. - The #429 citation in embedded-policy.md is replaced in main's wording of the paragraph.
OmarAlJarrah
added a commit
that referenced
this pull request
Nov 5, 2026
Its base left the emitter design stack and stands on main, so this branch now builds on main too. The citations of #679 this branch linked in architecture.md, prior-art.md and reference-learnings.md exist only on #1336's branch, so those edits drop out, and the two correlation entries join main's list of reachability roots.
This was referenced Nov 5, 2026
A README integrated as blocks fell under the rule for every written file, so a user's edit around the blocks, which is the point of them, read as an edit and stopped the file being written. Such a file carries no generated header, is never pruned, and its digest covers its unfrozen blocks alone. An edit inside an unfrozen block is replaced as the block is, freezing being how a user keeps one, and is reported, so it is not lost unseen. A scoped run removed a README block, or pruned a file, it had not selected; it now removes and prunes nothing outside its scope. A prior entry found by correlation key was "the entry recording its key", but a key need not be unique: a property's is its wire name. The match now needs exactly one such entry and one new entity with the key, and an unmatched key names its likely new home only where exactly one entity has the key.
A file written once is the user's to edit, but the edit check and the no-op test compared every file's digest, so a filled-in test file drew a warning on every run and kept every run from being a no-op. Such a file is never checked for an edit, and the no-op test leaves it aside. A README block built from an entity's ID now follows the entity as a region does: after a rename the new block takes the old one's place, and a block frozen before it stands for the entity, where it was left beside a second, generated section for the same operation.
A file a user marks @morphic:ignore-file is theirs whole, as a file written once is, but only files written once were left out of the edit check and the no-op test, so such a file drew a warning on every run, and header-gated pruning could delete code its user had taken over. It is now never written, pruned or checked for an edit. A file written once was written "while absent", so one its user deleted came back on the next run, though T-40 and the contract's SkipIfExists both say it is written once. The manifest records that it was written, so it is written by the first generation that finds it absent and never again.
A file edited outside its regions was left as it is, but when a rename moved its entity to another file, the old one, emptied of generated symbols, was pruned as any other, and the edit went with it. It is now never pruned. The entity is still written in its new place, since the generation's callers name it there and leaving it behind would not compile, and its regions stay in the kept file rather than being written into both, which would declare their code twice.
Nothing said what an empty key does. Read as written, a removed and an added entity that both lacked one were matched by their equal empty keys, so an unrelated newcomer inherited the removed one's suffixed name, list2 where list was free, and kept it from then on. Pinning such an entity by ID alone is no fix: its ID is what #678 shows can name another entity, and #583 asked for the ID and the key to agree. An entity with no key, or whose entry has none, now has no prior entry, so rule 2 names it as on a first generation and no region, file or block follows it by key. Write/integrate relies on the keys, so the pass computing them is one a caller cannot turn off. An unmatched key names its likely new home only where the prior manifest records a key for it and exactly one entity has that key.
The last change said the pass computing correlation keys is one a caller cannot turn off. But taking two revisions' entities for one because their names agree is inferred, not declared, and the design holds every such heuristic to being disable-able (INV6). Turning the pass off now turns the carrying off with it: every generation is named as a first one, and the engine says so once, as a warning, where a prior manifest is given, since the names it shipped may change. The emitters README's INV6 row lists the heuristic.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What one generation has to leave for the next, so that a regeneration honors what was shipped and edited. These issues are one contract, the generation manifest, and the rename claims they all rest on, so they land together.
What the manifest records (#713) (
docs/emitters/write-integrate.md#what-the-manifest-records):sourcesanddiagnosticsleft out, the digestPlan.Versionalready takes. Reformatting or moving a spec moves only where its reports point, so it is no new generation.\nline endings, trailing newlines dropped). An edit inside a region is not an edit. A file with the header whose bytes no longer match was edited outside its regions: it is a warning, never overwritten or pruned, and keeps its prior entry so the next generation reports it again. An entity a rename moves out of it is still written in its new place, since callers name it there, and its regions stay in the kept file rather than being written twice.(devel),+dirty) never matches, so a change being developed is never skipped.SkipIfExists, followed through their origins, never written again even after their user deletes one, never checked for an edit and left out of the no-op test, as a file marked@morphic:ignore-fileis), and each entity's correlation key, scope, base name and symbol.Regions follow their entity (#714). A keep region belongs to the entity it sits in, follows or precedes, and is recorded by ID, correlation key and name, with where it stood. When a rename moves the entity to another file, the region is written in the same place relative to the declaration, so one inside a class body stays inside it. Two regions of one name in one entity are a warning and stay where they are. A region whose entity is gone is reported and its file kept. A merge mode beyond additive would take standard three-way merge semantics, never one of its own.
A key that matches nothing (#716). Every ID an input names (a hint's key in any layer, a
RemountorMountgroup, a per-target overlay's key, a runtime-policy operation key, an envelope'sExceptandBasesentries) is matched against the document once, by the engine, before any stage reads it. Each that matches nothing is a warningemitters/unmatched-keynaming, where the prior manifest records its key and exactly one entity now has it, that entity as the likely new home. It is a warning, not an error, since one configuration may serve several documents, so it fails a run only at a threshold the caller sets to count warnings.This changes two codes that landed after this branch was cut.
emitters/policy/unknown-operation(runtime policy) and theExcept/Baseshalf ofemitters/plan/envelope-unused(plan policy) reported the same condition under codes of their own. Neitherpolicy.Resolvenorplan.Buildtakes the prior manifest, so neither could name where the entity went. The engine holds every input and the manifest, so it reports each once and those stages pass over such an ID.envelope-unusedkeeps "an envelope no body matches", which is a different fact, and T-32 follows. Keeping each reader's code was weighed: it overturns nothing, but leaves three codes for one condition and the new home in a second diagnostic.Shipped names (#583) (
docs/emitters/refine/naming.md#collisions-and-the-names-a-generation-shipped).getandGetRequestto a newcomer. Keep regions and files written once follow their entities by the same prior entry. An entity with no correlation key, or whose entry has none, has no prior entry: pinning it by ID alone would bring back ir: an ID derived from an array index names a different entity after the array changes #678, and two keyless entities would be taken for one (a removedlist2handed to an unrelated newcomer whilelistwas free). Carrying a name by key is inferred, so it can be turned off with the key pass (INV6); every generation is then named as a first one, with one warning where a prior manifest is given. The scope, not the unit, is compared: by default every group's client sits in the root unit, so an operation moved to another group would otherwise keepgetbeside thegetthat client shipped. A program of the rules shows it: comparing units gives twogetin one client, and matching by ID alone renames a path-renamed operation fromgettoget2.comwould givecom2, a Windows device name. A shipped symbol the scope now reserves is not taken again either.operationIdstill renames. An ID alone pins nothing (ir: an ID derived from an array index names a different entity after the array changes #678), as the issue's comment required.printing.mdnow name the prior manifest.README blocks (#705). The docs emitter returns sections as blocks with stable IDs between markers (
docs-and-mock.mdnow says so), each ID percent-encoded, since one built from a tag can hold a space or-->. Integration replaces only those blocks: text outside them is kept byte for byte, frozen blocks are kept, a block built from an entity's ID follows it across a rename as a region does, a new block lands beside its neighbours in the generated order, removals are reported, and the table of contents is rebuilt last from the merged file's headings. Such a file carries no generated header and its digest covers its unfrozen blocks alone, so an edit around them is no edit, and an edit inside an unfrozen block is replaced and reported, freezing being how a user keeps one.The Go merge adapter uses
dst(#794). A spike at Go 1.27.3 anddstv0.28.0 added a type and a method from a new generation to a file holding a keep region, three ways:go/astnodes moved and printed: every moved doc comment was misplaced, and one landed insideGetZone's body.go/astto locate declarations, source text spliced in,astutil.AddImport: correct, but it cannot reach inside a declaration, as a field-level merge (open question 5) would need.dst: correct, imports managed, and a file with generics, a range-over-func iterator andminround-trips byte for byte.dstis MIT-licensed, v0 and maintained in bursts, last released in 2026. The documents pin it exactly, allow it inemitters/golangalone (packages.mdstates the archtest entry the package will need), and make the spike's cases the adapter's tests.IDs and renames (#677). The statements across
docs/that #677 lists promised that IDs survive any rename, though an operation's ID is built from its path and a component's from its name. Each now says what holds: a rename of what the ID is not built from (anoperationId, atitle) is "same ID, new symbol", and one of what it is built from is carried by the correlation key (#679) only where the key's own source holds, as anoperationIddoes when its path is renamed. A component renamed changes its ID and its key together, which only the structural tier ofdocs/ir/open-questions.mdquestion 5 could carry. That coversarchitecture.md,ir/identity.md,prior-art.md(three sites),reference-learnings.md, and inemitters/:plan/overview.md,shaping-hints.md,write-integrate.md,surface-verification.md, T-8 and the README's INV3 row.milestones.mdnow says M4 needs the key, the staleness check counts an entity the key carried as live rather than listing it as dead, and the README's "everything cross-run correlates by IR stable ID" names the key too. #419 (stating the answer indocs/ir/open-questions.md) and #679 (the correlation-key pass) stay open.T-39 pins shipped names, its newcomer sorting before the incumbent so that ID order alone cannot pass it, and T-40 the manifest and integration rules.
Closes #713
Closes #714
Closes #716
Closes #583
Closes #705
Closes #794
Closes #677
Test plan
dstv0.28.0:go/astnodes moved and printed lost their doc comments, dropped the new import and placed the new declarations inside the keep region;dstkept all three, and round-trips generics, range-over-func andminbyte for byte.dstlicence (MIT, from its LICENSE), latest tag (v0.28.0, 2026-09-10, from the module proxy) and repository state checked.go vet.sourcesleft out; one drawing a warning differs untildiagnosticsis left out too.-->and a space verbatim (g/tags/Zones --> DNS).getandGetRequestunder the derived-entry rule and loses both to a newcomer when matched by key alone.go test ./internal/archtest -run TestDocumentsover the new anchors and links; a planted broken anchor fails it.