Repository navigation
docs(emitters): design the docs emitter and its shared examples - #1337
Merged
Merged
Conversation
An anyOf lets one value satisfy several variants, and the emitter design lowered every untagged union to a closed sum that holds exactly one. Where two variants admit the same value, holding the first misnames it and drops the members only the others declare. The plan now proves, once per union, whether two variants of an anyOf can admit one value (TypePlan.Inclusive), by JSON kind, by finite value sets and by a member one model requires that the other cannot hold. A union it proves disjoint lowers as a closed sum, as a oneOf does. One it cannot is inclusive: it holds an optional member per variant, set for each variant the value admits, decodes into every candidate variant, and encodes the merge, a disagreement being an encode error. A null a nullable variant admits is held by the reference, never by a variant. Go lowers an inclusive union to a struct with a pointer per variant and Kotlin to a class with a nullable property per variant. T-29 pins the round trip of a value two open models admit.
A positional model is an array on the wire, so the kinds rule now says so and the member rule reads only models laid out by name. A stream that ends on a terminal variant now ends on an element holding one, among the several an inclusive union holds. A null literal variant now lowers as a nullable variant does, so either spelling of a null member reaches the reference. Inclusive joins the plan fields the plan-or-refine test keeps, and the docs emitter says when a value may be several variants at once.
plan.Policy held a function, PrimaryResponse, and Plan.Version hashed the policy. A function has no stable bytes: json/v2 refuses it, its printed form is an address that changes every run, and skipping it gives two policies that pick different responses one version. The policy is now data from a closed vocabulary: PrimaryResponse is an ordered list of rules and DefaultFails says whether an unpicked Default is the catch-all failure. Version hashes the document's digest without sources, a new plan.Revision bumped with every change to Build's output, and the policy's deterministic JSON. EnvelopeDetect named a type nothing defined. plan.Policy.Envelopes now lists envelopes a configuration names, none by default. Each matches a body by the bases it reaches or by the members it declares, and says where the value sits and which flag value means the call failed. A matched success returns that member, keeps the envelope reachable through WithResponse, and turns a 2xx whose flag says it failed into a failure with no fault or kind, decoded as the operation's matching error body. Failures and pages are read whole. T-30 pins the plan's identity and T-31 the envelope reads.
…into docs/emitters-envelopes-policy-data # Conflicts: # docs/emitters/docs-and-mock.md # docs/emitters/testing/mandatory-tests.md
Runtime policy was one tree per project, so an operation needing its own timeout or retry had nowhere to say so, and a document's own SDK configuration, kept verbatim as vendor extensions, would have been parsed by each emitter its own way. Policy and hints are now layers, each holding a document-wide override and overrides by operation: what a document embeds, the project's, the emitter's. A setting takes the highest layer that sets it, an operation's value over its layer's document-wide one, and a declared IR fact beats every layer. A new emitters/embed reader turns vendor extensions a configuration maps into the lowest layer of each, decoding each value strictly and reporting what it cannot read. The vocabulary gains connection-error retries, Retry-After, a throttling backoff and redirects. Whether an operation is safe or idempotent is a plan fact, OpPlan.Effects: declared, else from its HTTP method. An enum a client reads keeps a value it does not list, as a union does, and a oneOf value two variants admit decodes as plan.Policy.ExclusiveMatch says. A service hint makes a parameter a client setting; nothing is hoisted by inference, since a shared parameter stops being shared as an API grows, and the plan only reports candidates.
An enum whose value type has no Go constant form, a date, a uuid or a decimal, had no Go lowering, and the type table built an enum value by decoding its wire form, which no constant can be. Kotlin's enum class writes every member as its @SerialName string, so a boolean or numeric enum went out as a string its schema does not admit. A member is now written as the JSON kind its value type has, and decoded by comparing as JSON compares values, numbers by value. Go declares an enum over its value type's row, and over the wire text where the row has no constant, the codec writing the value type's JSON kind and a Value method giving the row's type. Kotlin generates a KSerializer for every enum that is not a string enum, and an entry of a mapped type holds its value as that type. T-36 pins every value type on every target.
Kotlin's naming document stated only the members it gives every type, so a Kotlin emitter had no names to reserve for a group's module or for the declarations a unit holds beside the spec's, and T-19 no fixture. Measured with Kotlin 2.4.0 and Gradle 9.8.2: a module named build sits in the root project's output, which clean deletes with the module's sources; one named buildsrc is taken for Gradle's buildSrc on a case-insensitive file system and fails the build; gradle and src share the wrapper's and the root sources' directories. Those four are the units set, and a group's module is placed at the build root so the set applies. Hard keywords need no reservation: backticked as package segments they compile on the JVM, JS and Native and link into a framework, and kotlin is refused only as the first segment, which the options name. The unit set is Client, a group's sub-client and the runtime names the root unit declares.
The manifest recorded hashes, a file list and the entity-to-symbol map, too little for a regeneration to honor what the last one did: an edited generated file was overwritten, a no-op rewrote every file, a reformatted spec looked new, and keep regions were found by path. The manifest now records the document's digest instead of the spec's, a normalized digest per written file, the options version, the steps that ran, files written once, each entity's correlation key, base name and symbol, and each keep region by its entity, under a schema version migrated forward. Regions follow their entity to its new file. Every ID-keyed input that matches nothing is reported with its likely new home. A README is merged as blocks the user writes around. Collisions now read the prior manifest: a shipped symbol keeps its name while its ID and correlation key agree, and only newcomers take a suffix, so adding an endpoint renames nothing. The prior manifest joins the inputs the purity rule names. A spike at Go 1.27.3 chose dst for the Go merge adapter: go/ast nodes moved between files lost their doc comments, one into a function body, while dst and a text splice were both correct, and only dst reaches inside a declaration. Claims that IDs survive any rename are narrowed to renames that keep the pointer.
The docs emitter reused the SDK's primary response and content, had no design for URLs, links, collisions or versions, spelled usage examples it could only re-derive, and left documentation extensions and agent documents unaddressed. Every artifact would also synthesize its own example values. The docs now render every response, error case and content, the primary only ordering them, and each type's request and response shapes apart. A route plan derives every URL from an ID-keyed catalog through templates, refuses any two entities on one URL after the templates apply, resolves links as ID requests, and keeps one entity's identity across version snapshots through route tables the engine passes in. Agent Markdown and llms.txt share those routes and escape spec text. SDK emitters return a usage sample per operation for the docs, and the plan holds one set of examples, declared first and otherwise seeded from each entity's correlation key, for docs, samples, tests and mocks. Documentation extensions are read by mapping, internal markers left to the filter pass, and a CLI is planned as an extension point built from IR facts.
EmitterInput.Policy became policy.Resolved when policy gained layers and per-operation overrides, but the refiner context still held a single policy.Policy, so a step had no way to read an operation's own settings. Ctx.Policy now holds the resolved policy the emitter is given.
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
Collision resolution now keeps the names the prior generation shipped, read from its manifest, but the refiner context carried no prior state, so the naming step had nothing to read it from. Ctx.Prior carries the PriorState the emitter is given.
…docs/emitters-docs-artifacts
The docs emitter now receives every SDK target's usage samples and every version snapshot's route table, but the shared refiner context carried neither, so a docs refine step could not read them. Ctx holds both.
The rule saying what a oneOf decodes a value two variants admit to landed at the end of the section on a variant's null, which it has nothing to do with. It belongs with how a variant is recognized, which is also where T-35 links it.
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
…ts' into docs/emitters-generations
…docs/emitters-docs-artifacts
…into docs/emitters-envelopes-policy-data
…ata' into docs/emitters-policy-layers
…o docs/emitters-enum-value-types
…into docs/emitters-kotlin-reserved-sets
# Conflicts: # docs/emitters/testing/mandatory-tests.md
Plan.Examples was keyed by the IDs of types, properties, parameters and operations, and said nothing of what it does where those are not the holder. A content, a variant and a message carry no ID, and a content is where a document most often declares a body's examples: on the golden spec (spec @57b5196b, morphic @ab953d45), 435 request, response and error contents declare 651 of them. So a content's, variant's and message's own examples are read from the IR, an entity declaring none that adds no constraint takes its type's set, and a synthesized model holds each member's own first value, so a property shows one value alone and inside its model. The bounds and validity could not both hold: at least 250 golden-spec models require members nested four or more models deep, and one list requires four items, past the default depth of 3 and 2 items. The bounds now cut only what validity leaves optional, and a value past a fixed size limit or a required cycle is reported and left out. A negative bound is policy-invalid. An entity no correlation key identifies is seeded from its ID, and the stability statement names what moves a value: the schema, the key, the policy, CorrelationVersion or the synthesizer's code, not a step version the plan does not have. The mock reads these examples rather than synthesizing its own, and Example marks a synthesized value Inferred, the polarity Provenance and PagePlan use.
The docs design had the engine compute every version snapshot's route table, but the catalog and URL templates are the docs target's, and the Emitter interface had no call that returns them. A target whose Capabilities.CrossVersion is set now implements Router, which the engine calls on each snapshot before any Generate, and the registry refuses one that sets the flag without it. A run plans its own routes by the same pure function and is the one place their diagnostics are reported. Samples and routes are other runs' output, so the determinism tuple names them and the manifest's Inputs gains Peers: without it, a docs regeneration after an SDK target's options changed, or another snapshot did, matched every input and wrote nothing, leaving the docs on calls the SDK no longer has. The engine still runs an SDK target for its samples when that target's generation writes nothing, and Capabilities.ReadsSamples tells it which targets run last, since nothing else said so. A Multiplatform target's callers write Kotlin and Swift, so a target returns a sample per language rather than one per operation, and the agent documents are split by target and language. Routes are keyed by the label Document.Versions gives a snapshot.
A type was rendered once per direction it travels, which merges the create and patch shapes TypePlan.Shapes keeps apart: a patch body's members are optional, and one visible only at creation is absent from it. The docs now render a table for each shape the plan lists. The route plan was total over "the types that warrant a page", which nothing defined, and anchored a property on its type's page, which an anonymous type does not have. A declared model, union or enum, and one more than one reference names, gets a page, since on the golden spec (spec @57b5196b, morphic @ab953d45) 28 anonymous models are bodies 2 to 38 operations share; any other type renders where it is referenced. A slug carries no Inferred mark, which nothing could hold. A destructive warning and a CLI confirmation came from "a declared fact, a hint or a promoted extension", none of which exists: no IR field states it and no hint did. ShapeHint gains Destructive, which a document's extension reaches through the embedded reader. The docs emitter read documentation extensions itself, while embedded-policy.md holds that an emitter never reads one. Those keys now go through the one embedded reader into a hint's Extra members, which the docs target decodes strictly, and a tag's keys wait on #1221 as every tree's do. architecture.md's filter pass now reads the internal marker the docs design sent it, which it did not say.
The page rule counted direct references, so an anonymous model one reference names, but held by a list several models hold, was rendered on each of their pages: 184 on the golden spec (spec @57b5196b, morphic @ab953d45), such as the items of a declared list. A type the document declares now has a page whatever its kind, and an anonymous model, union or enum has one when more than one reference reaches it through the anonymous containers holding it, so no type is rendered in two places. Three pairs of the golden spec's declared types share their canonical words, so slugs colliding in one template's scope are told apart by ID, as identifiers are, and only a collision the templates make is refused. A type no shape reaches had no table, where the plan declares it once, unprojected; it now has one. An example is shown through the shape it is shown in, so a response's holds no write-only member. A value Secret or Sensitive marks, which the IR says to redact in docs, is a placeholder on pages, in agent documents and in samples. A synthesized model held each member's own set whole, which a depth bound cannot allow and an optional recursive member never ends: a synthesized member value is now cut to the depth left, a declared one kept whole. A message has an ID and is keyed by it; a content's and a variant's sets are what samples and the mock read a body's examples from. A map's entries are bounded by minProps and maxProps, the size limit is named, the CLI redacts a secret parameter too, and T-41 pins the rules it had left out.
The docs rendered every Response and ErrorCase but not an operation's Default, which the IR keeps apart from both: a spec's `default` response, with its content and example, lowers only there. It is now rendered beside them. A table per shape repeated itself wherever two shapes project a model alike, and 17,305 of the golden spec's 17,872 models (spec @57b5196b, morphic @ab953d45) mark no member's visibility. Shapes whose projections coincide now share one table, as they share one declaration. A member is anchored only where the model declaring it is rendered, and an anonymous model composed in place has no table of its own, so no member is anchored twice on one page. A sample's values are its operation's first scenario's Input: one the operation declares, as Smithy and TypeSpec can, else one assembled from its inputs' sets. Messages leave Plan.Examples: a message ID has no kind prefix to keep it apart from the other classes in one map (#1527), so its examples are read from the IR as a content's are. The policy's "closed vocabulary" now admits the counts its example bounds are, and the CLI's presentation hints name the Extra members they live in.
Every consumer reads a reference through TypePlan.Alias, and the docs rules did not. A decorator, the anonymous Model or Scalar a 3.0 allOf over a sole $ref or an annotated position becomes, would have been given a table, or a page when shared, and left without a route, which the route plan reports; and its examples, which describe the position it decorates, had no stated precedence. By ir.Decorated, 1,610 of the golden spec's types are decorators (spec @57b5196b, morphic @ab953d45), 645 of them models, and 758 carry examples. A decorator now has no page, table or anchor: a reference to it, a DocRef among them, renders and links as one to its Alias, and its annotations fold onto the position as CollapseAliases folds them, so the 3.0 and 3.1 spellings render alike, as T-16 requires of every artifact. A position's examples are its own, else the nearest decorator's, else its Alias's set, and the page rule counts references through decorators, which keeps every non-decorator type in one place on the golden spec. An agent document describes a call by its binding rather than an HTTP method and path alone, a message reads all its payload's contents, and two paragraphs are rewrapped.
The version-slice pass hands each run one snapshot's document, and nothing in that document says which version it is, while a docs run had to find its own entry in EmitterInput.Routes by label and mark the current version in its switcher. EmitterInput gains Snapshot, the label Document.Versions gives the snapshot, "" for a document stating none. It joins the determinism tuple, since two snapshots of one document differ only by it, and the manifest's field for what the engine hands a run beside its document, renamed Handed from Peers, hashes it with the samples and routes. A snapshot's docs run takes the samples of the SDK targets run on that snapshot, and one no SDK target runs on has none, which is no diagnostic. T-42 pins that Synthesize off leaves no example Inferred, as policy.md requires of every heuristic, and the package map lists the contract and plan types this design adds.
Only the agent documents said what becomes of raw HTML in a description; the pages, where passing it through would publish whatever markup or script a document holds, said nothing. Pages now render it as text too, save a bare <br>, which renders as a line break. On the golden spec (spec @57b5196b, morphic @ab953d45), descriptions hold 45 raw HTML tags by CommonMark's grammar outside code: 11 a <br/>, one an <img>, and 33 placeholders such as <key> and <uuid>, which a browser would parse as elements and drop from the page. A sample's redacted value is the SDK target's to write, since the call is its code and the docs never edit it; T-41 pins both rules.
Rendering a description's raw HTML as text left its links open: a Markdown link or an ExternalDocs URL can still carry a javascript: destination. Pages and agent documents now write a destination only under http, https or mailto, or as a relative path resolved against a base the docs options name; anything else renders as its text with a warning. The golden spec has 14 relative description links, such as #requests, that would break on a generated site. Nothing put the version into a route, so two snapshots routed one entity to one path, the switcher linked to itself and one run could prune the other's pages. Each snapshot's run is now its own generation under its label's directory, Router is handed the input its snapshot's Generate takes, and one run writes what spans versions. Also: slug and route collisions fold letter case as artifact paths do; a sample takes the first scenario ending in an Output; two ReadsSamples targets no longer wait on each other; an assembled scenario is Inferred where a value it takes is; the mock reports a content with no example; and the filter pass warns when it removes a required property.
Declared examples were used everywhere unchecked. On the golden spec a declared string example fails its own type in at least 294 types, such as json against an enum of JSON and CSV, so an SDK's test would send a value the mock refuses. A declared example its position refuses is now reported and left out, so every consumer falls to the next set alike. What a docs run showed depended on which other targets shared the run: the docs target run alone after go and docs dropped every sample. The docs options now name the targets whose samples they show, and the engine runs those for it. An assembled operation scenario pairs values the document never paired, so it is always Inferred, and ExamplePolicy.Assemble turns it off. It takes required inputs and the optional ones with declared values, and a success scenario is one ending in no Error, so a bodiless call counts. Also: a model rendered as several tables anchors a member in the first of them; channels, messages and auth schemes get pages; the files that span versions keep one owner in a manifest at the root, and a dropped version's directory is pruned; images and every document URL follow the link rule, a terms-of-service that is text stays text; the decorator example fallback credits CollapseAliases for what it folds; MaxItems is worded as policy.md has it; and the direct-reference page count is 1, not 184, since decorators are read through.
The last label's run had to return a second manifest the contract has no field for, nothing could prune a dropped version, and an unversioned document had no "last label" to write its site scope. A cross-version target's snapshot runs are now one generation: its manifest holds each run's under its label, each run's Prior is its own label's, and the writer reads a path across them, so a dropped label's files are pruned and a file moving between runs is not taken for an edit. The "" snapshot, any document no pass slices, writes its site scope itself. An assembled scenario's Input had no shape, and keying it by name is ambiguous: nine golden-spec operations have a body member named like a parameter. It now keys parameters by ParamID and the body under "body"; a declared scenario's shape is left to #1528. Contents, variants and messages have no key in Plan.Examples, so each consumer would have re-run the check; the plan now exports one function per kind that every consumer reads them through. Also: sample targets run with their own prior manifest and Scope and write nothing the run did not ask for; a page no entity owns is routed by its kind and scope; an open enum admits an unlisted example; the failing-example count is 295 (one integer enum was missed); the depth count is 245 under a stated rule; the decorator counts name their revision; and T-41 pins the documentation-extension rules.
The route plan promised an anchor to every type rendered on another's page and to every PropID where its model renders. A spelled type, such as an anonymous list, is spelled at each use, and a header has no model: the golden spec's 132 header PropIDs would each have drawn a diagnostic. Anchors now go to tables and to model members; a spelled type and a header take none. The cross-version manifest had no declared shape, and the pages spanning versions were told to sit both under the last label's directory and at the root. write-integrate.md now declares Generations, and those pages sit at the output root. Content, variant and message examples refused by the check had no channel to be reported in; Build reports each once and the plan's three readers report nothing. A recursive request body gave the CLI no last flag (17 golden-spec operations send one); past a type's first recurrence a field is one JSON or @file flag. Also: plan.MaxExampleValues is declared as 10,000, several times the roughly 1,300 values the largest default-bounded golden example holds; a named sample target with no configuration is refused; an operation with no success scenario has no sample and no diagnostic; the visibility count is 15,694, models marking nothing they hold either; the link tally counts the autolink; and the persistence paragraph states the churn a key change costs instead of denying it.
Only cross-version targets had a rule for a document split into versions, so an SDK target or a docs target without the capability would run once per snapshot into one directory and prune its own output. Every target now writes each snapshot's run under the directory its label names, the label as one path segment, and its runs make one generation; a label that is no segment, or two differing only by case, is refused. A scoped SDK run returned samples for part of the SDK, so the docs lost the rest. Scope now gates artifacts alone: a target returns a sample for every operation. A declared example was checked against its type but shown through its shape, which refused a valid patch body leaving out a required member; it is now checked through the shape it is shown in. The new reports named neither code nor, in places, severity, so no test could pin them: link-unwritten, route-missing, route-collision, extra-invalid, sample-unrendered, example-refused, example-unsynthesized and the mock's example-missing now have codes. Also: services and groups are routed, an unresolved auth scheme has a page, only anonymous models rendered as tables are anchored, agent document routes join the collision check, an enum member's set is its own value, a patch shape's table stands apart only where the model requires a member, 24 of the 28 shared bodies get pages, an assembled scenario needs every required input's value, the Assemble-off and missing-input cases no longer overlap, agent documents size code fences by the backticks inside, MaxExampleValues bounds synthesized examples, and plan.Example's field is Example, not Value.
The bounds cut a list only past the larger of MaxItems and its minimum, at any depth, so a required list of its own model never ended: the golden spec's cloudforce-one_TreeNode requires children, a list of TreeNode, and had no example though an empty list is valid. Deeper than MaxDepth a list or map now holds its minimum alone. The 1,300-value estimate rested on the old rule and is dropped; MaxExampleValues is justified by what it bounds. The previous commit ran every target once per snapshot. An SDK target would then write one package per version, all sharing one module path, which a docs design should not settle. Per-label runs are a cross-version target's again; any other target is run on a sliced document for its samples alone until #1531 decides how. A run made for samples alone reports as any run of its target does. Also: a cross-version manifest is a Generations even unsliced, and the last label takes the prior's "" entry when a document gains versions; version-slice keeps Document.Versions whole in every snapshot, which the "last label" rule reads; a type's example is admitted where one of its shapes admits it; . and .. are refused as labels; properties no model declares, a resource's among them, take no anchor; the case-collision example cites the Webhooks and webhooks groups, which share a scope; and T-41 and T-42 now pin the assembled sample scenario and a message's fallback.
Rendering all raw HTML as text kept script out but would show every tag of a model that documents in HTML, as Smithy models do, on every page. Descriptions now render through an allowlist of formatting elements, a checked href and src aside, and everything else, a placeholder such as <key> and a comment among them, stays visible as text. The golden spec's pages are unchanged by it. A relative URL in a field of its own, such as an ExternalDocs link's, resolves against the document's servers or location, not the docs site, so it now renders as text. A seeded choice repeats at every recurrence, so a required recursive union could pick its recursive variant forever: two golden-spec unions, cloudforce-one_EmailRuleCondition among them, recurse through a variant requiring a list of themselves. On a recurrence or past MaxDepth a choice now takes the first variant from which a value ends. A write-only member of a model only responses reach, or a read-only one of a model only requests reach, is in no table, so the anchor rule would have failed a docs run on the golden spec, which has both, with route-missing; such a member now takes no anchor. Also: the spanning pages follow the last label Routes keys, not the last Document.Versions lists, and are checked against every label's directory; a switcher lists only labels with a snapshot; the remaining refusals have codes (engine/target-unversioned, engine/sample-target-invalid, pass/filter-required-removed); a patch table stands apart where any held model requires a member; ContentExamples and VariantExamples take the shape the caller shows them through; a shape admitting none of its type's examples reports as an empty set; and decorators.md no longer says no SDK renders a schema example.
A type's declared example was admitted per shape, but the plan had no field to say which shapes admitted it, and a shape admitting none was left with no example though synthesis could fill it. An example is now checked with visibility set aside, and shown through a shape by one plan function, Shown: it drops the members that shape does not hold and fills each one it requires and lacks from that member's set, marking the result Inferred. A response-form example then serves a create shape, and no shape is left without one. The allowlist decided only an element a description leaves open, so a stray </td> would close the page's own table cell. A description is now parsed as an HTML fragment where it is rendered and written back out, an end tag it never opened rendering as text. Example and link descriptions, which OpenAPI also writes in CommonMark, are read like Docs.Description. Also: an operation the API starts, a webhook, a callback or a received message, has no sample and draws no diagnostic; dl, dt, dd, h1 to h6 and hr join the allowlist, and a custom element such as <note> renders as text; an img's alt is optional; the collision check covers a run's routes and its agent documents together; and the reason a relative URL in its own field renders as text no longer says the IR lacks servers.
The HTML rule named an HTML parser's fragment mode, which cannot give its stated result: it drops a stray </td>, invents elements, and turns a placeholder such as <key> into one holding the rest of the text. A description is now read token by token against a stack of its own, a listed tag opening or closing an element and every other token written from its source bytes. Agent documents also write each description back out within its item, so an unclosed fence or a link reference definition stays inside it. A link to a model shown as several tables reached only the first, so an operation's patch body landed on a table marking required what the patch relaxes. Each table now has an anchor, and a link names the shape it shows. Shown had no way to fail, though a required member with an empty set cannot be filled; it now returns false so the consumer takes the next example. A synthesized list repeated one value in every item, which a uniqueItems list refuses: items now take the set's values in turn, then synthesized ones seeded by index, and a map's keys are distinct values. Also: a named sample target must set the new Capabilities.WritesSamples; the engine checks the context after Router returns, as after Generate; a run whose label the prior lacks, as when versions are lost or renamed, takes the prior entry holding the spanning files; T-41's samples clause says "operation its callers call"; T-42 scopes "no example is Inferred" to Plan.Examples; and a sentence binding "which" to the wrong noun is split.
The tokenizer stopped stray end tags only: a balanced <td> or <li> in a description rendered in a schema table's cell or a list still adds a cell, row or item to the page's own. A tag that opens only inside a parent now opens only with that parent on top of the description's stack, else it renders as text. An assembled scenario left out an optional body with no declared example, so the samples of 231 golden-spec operations would have shown calls sending no body. The body is now always assembled; optional parameters still appear only with a declared value. A member no table of its declaring model holds, as one of a base no shape reaches while a subtype's table holds it (six on the golden spec, under bill-subs-api_subscription), had no anchor; it is now anchored in the first table of a model whose flattening holds it. Also: the engine reads a docs target's sample targets through a new SampleReader on its options; a message's fallback examples pair its payload's with its headers' set; plan.Example's Inferred is an ir.Inferred naming the step that made it; a union value several variants admit is shown as the first; a required Nullable member ends a cycle with null; a shape's table anchor is named for its shape, so a switcher reaches it across versions; and a value seeded from an ID moves with it, as the Stable list now says.
A description opening with <p>, as an HTML-documented model's does, is a CommonMark HTML block, whose lines the token rule never reached, and "written from its source bytes" would show <key> literally. Inline raw HTML and HTML blocks now go through one HTML tokenizer, which splits tags from text without building a tree: text is decoded and written escaped, and unlisted tokens are written escaped from their raw bytes. A p, li, dt, dd, tr, td or th on top closes when a sibling opens, as HTML closes it, so <ul><li>a<li>b</ul> renders as a list. A URL was checked before its character references were decoded, so javascript: passed the scheme check and ran in a browser. A destination is now decoded by HTML's rules in an attribute and CommonMark's in a link before it is checked, and written escaped again. Naming a table's anchor for its shape alone collided on any page holding two anonymous models' tables in one shape, as an operation's page does for a response and an error body; an anchor now joins its model's anchor to the shape's name, and the collision check reads anchors within a page. TypePlan.Undeclared names, required members only a catch-all governs, had no table row, no synthesized value and no CLI flag; they now have all three. A union value several variants admit is refused where the union is Strict, as the mock refuses it. Also: refused a and img render as their text and alt; "catalog order" is the byte order of TypeIDs; a version label is percent-encoded in links; an example Shown filled is labelled generated; a declared code sample's language is kept only as one word; a prior manifest is read as Generations where it holds snapshots; and the recursive-union example names the one golden-spec union it fits.
The HTML tokenizer switches to raw-text mode after a title, textarea, script or style tag, so an unlisted <title> inside a <p> block turned the rest of the description into one text token. None of those elements is listed, so the tokenizer never enters that mode (NextIsNotRawText after each start tag, which a probe confirms keeps the later tags). A self-closing listed tag, which HTML opens as a start tag, now opens an element, and every listed tag is written from its name, never its raw bytes, so a <table/> cannot open a table the stack never closes. A table's anchor was "its model's anchor joined to its shape's name", but a shape has no name (#1405 has not settled one). The suffix is now derived from the shape: its Filter lifecycles in order and "patch" for a patch shape, a shared table taking the first shape's and the unprojected one none. Also: Build, not Shown, refuses a value several variants of a Strict union admit; an Undeclared name is optional in a patch table, as any required member is; the example of an inherited member with no anchor names a base reached only in shapes leaving it out; Example and Link descriptions are read as CommonMark by the docs, not said to be so by the IR; an image's source may not be mailto:; and the prior entry a new last label takes is the one listing a path outside its own label's directory.
A balanced <table> in a description rendered inside a schema table's cell still broke the page: a browser closes the page's table around a nested one in ways no stack of the description's can follow, and drops the page's later cells. Table tags are now off the allowlist and render as text. The rule's contract is now its outcome, which T-41 checks by parsing each page as a browser does against a fixed hostile corpus: the page's own elements stand around every description. The raw-text guard ran after start tags only, but a self-closing <title/> enters raw-text mode too; it now runs after both, as a probe of x/net/html confirms. Shape-table anchors were built from Filter lifecycles after slugs were resolved, so a member named create beside a create table took the same anchor, a route-collision on a valid document, and the lifecycle spelling had no escaping. Table anchors now go through the page's ID-keyed resolution like identifiers, one per shape a table shows, and how a shape is named is left to #1405. Shown's fill and a message's payload-header pairing are pairings the document never made: both are now Inferred and turned off by Assemble, as INV6 asks of an inference. Also: a Docs.Refs span inside raw HTML or a link renders as its text; and the Snapshots-only rule says Routes is a cross-version run's.
A table's anchor is resolved within its own page, so the same table can carry another anchor in another version, and a switcher computing it locally could land on a member's row. The switcher now links a model's page in another version, never one of its tables. Build judges a Strict union's admission with visibility set aside, but a shape can drop the member that kept one variant out, so a value one variant admits at declaration can be admitted by two once shown. Shown now rechecks admission in the shape it shows and is false there. A patch carries no default onto the wire, yet its table could share one with a shape that shows defaults; a patch table now stands apart where a held model gives a member a default too, and shows none. Also: a Docs.Refs span inside a code span, or naming a type no route reaches, renders as its text and draws no route-missing, as the IR asks; a CLI's help writes a description as plain text with control characters, terminal escapes among them, removed; Generations declares its JSON member names; #1531 now covers every target without CrossVersion; and T-41 and T-42 pin the img with no alt, the switcher's order, every llms.txt scope and a uniqueItems list below its minimum.
Shown and MessageExamples were to obey ExamplePolicy.Assemble, but both take only a *Plan, which held no policy, so neither could, least of all from a cached plan. The plan now carries the Policy Build ran under, plain data its Version already hashes. Shown returns false rather than "reporting", and an example none of a set completes leaves the docs with none there, a sample with sample-unrendered and the mock with example-missing. CLI help dropped all markup, which lost the golden spec's placeholders, the <!--sse--> tags a description tells readers to type, and every link URL. It now reads a description through the pages' tokenizer and link rule, writing listed elements as their text, every other token as written and a link as its text and URL, with control characters still removed. An example holding only an ExternalURL had no rule: it is shown as a link and held by no set, since no value of it can be checked, filled or sent. Also: a listed end tag matching below the top of the stack closes through, as HTML does; a Docs.Refs span inside an <a> the description opened renders as text; "no shape is left without one" is scoped to Assemble on and non-empty required sets; Generations drops the JSON tags no other manifest struct carries; and T-41 pins the prior read on gaining or losing CrossVersion and a done context after Router.
A recursion through no model, such as the golden spec's cws_JSONValue, whose map and list variants hold the union again, never ended: lists were cut to their minimum only deeper than MaxDepth, which counts models. A list or map now holds its minimum on any recurrence of its type along the value too. Shown dropped every member its shape does not hold, which would also drop the keys an open model's catch-all admits, which golden-spec models' declared examples hold. Shown now keeps them, checked against the catch-all's type. An inclusive union's value, which may be several variants at once, is shown through all the variants admitting it rather than the first. A listed tag with an unlisted attribute was described both as the element and as text; it now renders as the element with the attribute dropped, so a Smithy-style <a href target="_blank"> is a link. Also: the emitters README now says the engine also runs the targets a docs target names, and what of other runs' output an emitter is handed; a GitHub-style table, which 12 golden-spec descriptions write and CommonMark does not read, is kept as preformatted text; an assembled body is shown through the request's shape, a sample taking the next scenario where it cannot be; with Assemble off a missing sample draws no diagnostic whatever its inputs hold; a switcher links the page holding an anonymous model's table where a version gives it no page; a sample target's prior manifest comes from its own output; and "no shape is left without one" also excludes a Strict union the shape leaves ambiguous.
A synthesized value was checked against the IR's type and constraints alone, never against the validation-only not, oneOf and anyOf that Unmodeled keeps, though the API holds a value to them: the golden spec's browser-rendering accessibilityTree body takes url or html, never both, and a synthesized body held both. Values are now checked against those constructs too, evaluated as the JSON Schema they are; a synthesized one failing them is left out and reported, a declared one refused. 34 golden-spec types carry one. CLI help stripped every control character, line feed and tab among them, which ran a description's lines and a code block's together. It now keeps line feeds and tabs and strips the rest. Shown took a ModelShape, which cannot say "unprojected", the form a type no shape reaches renders in; it now takes a *ModelShape, nil being unprojected, and that table is anchored by its model alone. Also: the docs render the answers the plan lists, Errors holding the service's CommonErrors, and the Default once wherever the plan places it; a p on top closes before an li, dt or dd is checked against its list, and "as HTML closes them" is dropped where the rule differs from HTML; the depth count is 250, counting inherited members as the synthesizer fills them; and a nested group's route joins its groups' slugs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The docs emitter was one bullet: it "reuses the shared plan verbatim (primary response/content, …)" and lowers to a
docast. Every docs target hits the questions below on its first build, and so do the samples, tests and mocks that share its values. They are one contract: what the non-SDK artifacts render, and from which shared facts.Return,OthersandErrors, the last holding the service'sCommonErrorstoo, theDefault(which the IR keeps apart from both) once wherever the plan places it, and everyContent. A type is rendered once for each projection itsTypePlan.Shapesgives it, shapes whose projections coincide sharing one table as they share one declaration, so a write-only property is in no read shape's table, and a model one operation creates and another patches gets a table for each. A direction alone would merge those two, since a patch body's members are optional; a table per shape would repeat itself for any model two shapes but a patch reach while nothing it holds is marked, and 15,694 of the golden spec's 17,872 models mark no visibility on their own members or on any model they hold. A member is anchored once, where the model declaring it renders and in the first of its tables that holds it, and each table has an anchor of its own, one for each shape it shows, named through the page's ID-keyed resolution like an identifier, so a member namedcreatebeside a create table keeps its own (how a shape is named is docs/emitters: name the declaration each visibility shape takes #1405's), so a patch body links to its patch table; a member no table of its declaring model holds, such as six read-only members ofbill-subs-api_subscription-v2that only a subtype's read table shows, is anchored in the first table of a model whose flattening holds it. Every reference is read throughTypePlan.Alias, as every consumer reads one: a decorator (1,610 golden-spec types byir.Decorated, 645 of them models) has no page, table or anchor, renders as itsAliasand folds its annotations onto the position, so a 3.0allOfdecorator and the 3.1$refrender alike, as T-16 requires. A type no shape reaches is rendered once, unprojected, as the plan declares it. An example is shown through the shape it is shown in, and a valueSecretorSensitivemarks, which the IR says to redact in docs, is a placeholder on pages, in agent documents and in samples, a sample's written by its SDK target, whose code the docs never edit.TypePlan.Undeclarednames required rows like any member, and so does an anonymous model, union or enum that more than one reference reaches through the anonymous containers holding it. On the golden spec 28 anonymous models are bodies that 2 to 38 operations share, 24 of them no decorator and so pages, and counting direct references alone would leave one more rendered on several pages, the enum a shared parameter's list holds. Channels, messages and auth schemes, which several operations name, get pages too, as services and groups do. Any other type renders where it is referenced, and everyPropIDa model declares is anchored where its model renders, with a diagnostic for anything left; a spelled type, such as an anonymous list, a header, which no model declares, and a member no table holds, such as a write-only one of a model only responses reach, take no anchor, since the golden spec's 132 headerPropIDs would otherwise each draw that diagnostic;Webhooksandwebhooks. Two entities on one URL after that are an error, checked after the templates apply and across every route kind, agent documents included. Each new report is named (emitters/docs/route-missing,route-collision,link-unwritten,extra-invalid,emitters/<target>/sample-unrendered,emitters/plan/example-refused,example-unsynthesized,emitters/mock/example-missing,engine/snapshot-label-invalid,engine/target-unversioned,engine/sample-target-invalid, and the filter pass'spass/filter-required-removed, which that pass's own tests pin when it is built), so T-41 and T-42 can pin them. That avoids the failure the issue quotes, where tied pages vanish with only a printed warning;Capabilities.CrossVersionimplements a newRouterinterface. The engine calls it on every snapshot before anyGenerate, handing it the input that snapshot'sGeneratetakes but for samples and routes, and passes all the tables inEmitterInput.Routes, keyed by each snapshot'sDocument.Versionslabel. A run finds its own entry, and the version its switcher marks current, under the newEmitterInput.Snapshot, since nothing in a snapshot's document says which version it is. Each snapshot's run writes under its label's directory, which the route plan puts before every route, and the runs make one generation: without it, two snapshots routed one entity to one path, so the switcher linked to itself and one run could prune the other's pages. A cross-version target writes each snapshot's run under the directory its label names, the label as one path segment other than.and..(engine/snapshot-label-invalidotherwise, as for two labels differing only by case), so no two runs share a path, and its snapshot runs make one generation: its manifest, a newGenerationsinwrite-integrate.md, holds each run's under its label, each run'sPrioris its own label's, and the writer reads a path across them, so a dropped label's files are pruned and a file moving between runs is no edit (write-integrate.md). The last label's run,Document.Versionsbeing kept whole in each snapshot, writes what spans versions, such as the site-scopellms.txt, at the output root; the last label takes a prior's""entry when a document gains versions. A target withoutCrossVersionis run on a sliced document for its samples alone, which write nothing, until docs/emitters: a target without CrossVersion has no rule for a document sliced into versions #1531 decides what such a target does with versions; the""snapshot of a document no pass slices writes its site scope itself. A page no entity owns, an index or anllms.txt, is routed by its kind and scope. A switcher links one entity across versions, a model by its page rather than one of its tables, whose anchors resolve within each page, and an absent entity resolves to a "not in this version" page.llms.txtat site, service, version and target scope go through the same route plan. "Safe" comes fromOpPlan.Effects. "Destructive" comes only from a newShapeHint.Destructive, which a document's extension can set through the embedded reader, since no IR fact or method states it (the issue's comment). Spec text is escaped, and raw HTML in a description renders through an allowlist on pages and in agent documents alike: formatting elements, tables aside, with no attribute,awith a checkedhrefandimgwith a checkedsrcand an optionalalt, render as themselves, so a model documented in common HTML formatting, as Smithy models are, reads as written; the description's inline raw HTML and HTML blocks alike are read by an HTML tokenizer, which splits tags from text without building a tree, against a stack of the description's own, never in raw-text mode, so an unlisted<title>cannot swallow what follows, since a tree-building parser's fragment mode drops a stray end tag and turns<key>into an element; a self-closing listed tag opens as HTML opens it, and a listed tag is written from its name; text is decoded and written escaped, every unlisted token is written escaped from its raw bytes, and a URL is checked only once its character references are decoded, sojavascript:is refused. A stray</td>renders as text rather than closing the page's own table, an<li>opens only under its list on that stack, and no table tag is listed, since a browser closes a page's table around a description's own in ways no stack can follow; every other tag or comment, a custom<note>among them, renders as text, an unlisted attribute is dropped from a listed tag, and a GitHub-style table, which 12 golden-spec descriptions write and CommonMark does not read, is kept as preformatted text. T-41 checks the outcome itself, parsing each page as a browser does against a hostile corpus.Example.DescriptionandLink.Description, which OpenAPI writes in CommonMark, are read the same way. On the golden spec only the 11<br/>render as markup: of the 45 raw HTML tags the golden spec's descriptions hold outside code, 33 are placeholders such as<key>, which no allowlist names and a browser would otherwise parse as elements and drop. Links are a channel of their own, since a Markdown link or anExternalDocsURL can carry ajavascript:destination, so a link or an image, and every URL the document gives, is written only underhttp,httpsormailto, or, in a description, as a relative path resolved against a base the docs options name, since a relative URL in a field of its own resolves against the document's servers or location, which a docs target does not hold; any other destination, a fragment alone among them, renders as its text with a warning, and a terms-of-service that is text stays text. The golden spec's descriptions hold 14 relative links, such as#requestsand/api/resources/filters/, which point into the site the author wrote for and would break on a generated one.EmitterOutput.Samples, one per operation and caller language, so a Multiplatform target returns Kotlin and Swift. A target withCapabilities.ReadsSamplesnames in its options the targets whose samples it shows, and the engine runs those on every snapshot before it, whether or not the run names them, with the configuration and prior manifest each would run with and writing none of their artifacts the run does not name, a named target with no configuration refused, and hands it their samples inEmitterInput.Samples, so what the docs show never depends on which targets share a run; a named target that is unregistered, does not set the newCapabilities.WritesSamplesor reads samples itself is an error, and the engine reads the list through a newSampleReaderon the docs target's options. A target returns a sample for every operation its callers call whatever itsScope, which gates artifacts alone, so a scoped SDK run's docs keep every sample, and none, with no diagnostic, for a webhook, a callback or a received message. It still runs a named target whose generation writes nothing, for its samples. Samples, routes and the run'sEmitterInput.Snapshotlabel join the determinism tuple, and the manifest'sInputsgainsHanded, which hashes all three. Without it, a docs regeneration after an SDK target's options changed matched every input and wrote nothing, leaving the docs on calls the SDK no longer has. A sample that fails to render is a diagnostic. The templates table now says so.Plan.Examplesholds, per type, property, parameter, enum member and operation, the declared examples, else the type's where the entity adds no constraint, else one synthesized underplan.Policy.Examples; an operation declaring no success scenario, one ending in noError, takes one assembled from its required parameters' sets, its optional parameters' declared ones and its body's, required or not, since 231 golden-spec operations send an optional body with no example on it, alwaysInferredsince the document never paired those values, which the newExamplePolicy.Assembleturns off. ItsInputkeys parameters byParamIDand the body underbody, since nine golden-spec operations have a body member named like a parameter; the shape of a declared scenario'sInput, which the IR leaves open, is ir: an operation scenario's Example.Input has no defined shape #1528; a sample's values are theInputof its operation's first success scenario, since anErrorscenario's input is chosen to fail; an operation with none, asAssembleoff leaves one declaring none, has no sample and draws no diagnostic. A position whose type is a decorator takes its own examples, else the nearest decorator's, asCollapseAliasesfolds them, else, where it adds no constraint, itsAlias's set (758 golden-spec decorators carry examples).Shownprojection, it drops what the shape does not hold and fills each member the shape requires and it lacks from that member's set, markedInferred, so a response-form example serves a create shape and a patch body may leave out a required member, and one it cannot complete, or aStrictunion's value two variants admit once shown in the shape, gives way to the set's next,ShownreadingAssemblefrom the policy the plan now carries (Plan.Policy), keeping a key an open model's catch-all admits and showing an inclusive union's value through every variant admitting it; an example holding only anExternalURLis shown as a link and held by no set; so every consumer falls to the next set alike rather than an SDK's test sending a value the mock refuses: on the golden spec a declared example fails its own type in at least 295 types, 292 of them outside their closed enum, such asjsonagainstJSONandCSV, and 3 overmaxLength; an enum the document leaves open admits an unlisted value.Plan.Examples, so the plan exportsContentExamples,VariantExamplesandMessageExamples, the one reading each consumer goes through;Buildreports each refused example once, so they report nothing.Inferred. It is seeded from the correlation key, or the ID where no key identifies the entity.MaxDeptha choice takes the first variant from which a value ends, so a required recursive union such ascloudforce-one_EmailRuleConditionends, and deeper thanMaxDepth, or on any recurrence of its type, a list or map holds only its minimum, so a recursion through no model, such ascws_JSONValue's, ends too, so a required list of its own model, such as the golden spec'scloudforce-one_TreeNode.children, ends. Validity wins where they disagree: 250 golden-spec models reach a model four deep through their own or inherited required, non-null members, deeper than the default bound, and one list requires 4 items against a default of 2. A value the synthesizer cannot make valid, or pastplan.MaxExampleValues(10,000, which bounds what a hostile schema can make one example cost), is reported, never emitted invalid. A negative bound ispolicy-invalid.Unmodeledkeeps for their type (not,oneOf,anyOf), evaluated as JSON Schema, since the API holds a value to them: the golden spec'sbrowser-renderingaccessibilityTree body takesurlorhtml, never both, which a synthesized body broke.Synthesizeoff can leave one, is a diagnostic of the mock's.irfacts.CorrelationVersiondoes, as on a rename, but a persisted one could be found again only by that key or by an ID the rename moves too, so persistence would add state without saving that churn.embedded-policy.md's rule. The one embedded reader maps documentation keys into a hint'sExtramembers, by its rules and layers, with an empty default mapping. The docs target decodes the members it defines strictly. A tag's keys wait on openapi: a promoted key kept under a scope reaches no typed field #1221, as every tree's do. An internal marker is the filter pass's to read, andarchitecture.md§2.2 now says so, so the docs never describe an SDK other than the one generated; the pass warns when it removes a required property, since the SDK may then send a request the API refuses, and pass: a filter pass must prune examples of what it removes #723 prunes what it removes from examples. A search index is an optional artifact.milestones.mdandarchitecture.md§2.3 name it. A table maps each CLI behavior to the IR or plan fact it reads. Flag constraints come from union branches byPropID, and flag names from ID-keyed collision resolution. That designs out the failure modes the issue reproduced: leaf-name keys,foo_baragainstfoo.bar, dropped deep fields, unions beside properties. Past a type's first recurrence a body field is one JSON or@fileflag, since 17 golden-spec operations send a recursive body. Help text goes through the pages' tokenizer and link rule and is written as plain text, keeping placeholders, comments a description shows and link URLs, with control characters, terminal escapes among them, removed, line feeds and tabs aside.T-41 pins the docs rules and T-42 the shared examples. Golden-spec counts are from spec @57b5196b compiled by morphic @ab953d45.
Closes #720
Closes #585
Closes #586
Closes #699
Closes #722
Closes #703
Closes #587
Test plan
Router,EmitterInput,EmitterOutput,Sample,RouteTable,Plan,Example,ContentExamplesand its two siblings,Policy,ExamplePolicy,ShapeHint,Capabilities,manifest.ManifestandInputs, the refineCtx), extracted verbatim from the final text, passgo vetagainst stubs of the types they name.ModelShape,TypePlan.Shapes,TypeCommon.Anonymous,Document.Versions,Docs.Refs,ResourceInfo,ir.Example's holders, correlation keys, the filter and version-slice passes, the embedded reader) checked against its document orgo doc ./ir.PropID); anonymous models with more than one holder (28, each held by 2 to 38 operations, all shared bodies), and those a direct-reference page rule would render in several places (1, an enum five operations reach through a shared parameter's list; none under the rule adopted); declared types sharing their canonical words (3 pairs); models with no visibility marker on their own members (17,305 of 17,872), and on any model they hold or inherit too (15,694); headerPropIDs (132 distinct over 176 uses, none a model member); request bodies reaching a cycle (17 operations); a model requiring a list of itself with nominItems(cloudforce-one_TreeNode), which the depth rule for lists now ends; operations declaring adefaultresponse (none; a probe spec shows one lowers toOperation.Defaultalone); decorators byir.Decorated(1,610: 645 models, 965 scalars, 758 with examples), and, with references read through them, no anonymous model rendered in two places; models reaching a model four or more deep through their own or inherited required, non-null members, through decorators (250; 245 through own members alone); types carrying a validation-only construct (34: 13anyOf, 12oneOf, 9not);minItemsabove 2 (one list, 4); raw HTML inDocsdescriptions by goldmark's CommonMark parser (45 tags outside code, plus 4 HTML comments, which render as text too, and one more placeholder inside an extension value); link destinations inDocsdescriptions by the same parser (988https, 7http, onehttpsautolink, 14 relative: 8 fragments and 6 paths; no image and none under another scheme); declared examples failing their own type (at least 295 types: 292 outside their closed enum, one of them an integer enum, 3 overmaxLength); operations whose body has a member named like one of their parameters (9); operations declaring a scenario (none of 3,767); declared types and operations whose source names differ only by case (one operation pair).go test ./internal/archtest/... ./internal/keydoc/...passes; a planted broken anchor in a new link failsTestDocuments_LinksResolve.