Skip to content

codegen: add a table-driven codec strategy - #469

Draft
iainmcgin wants to merge 4 commits into
table-runtimefrom
table-codegen
Draft

iainmcgin wants to merge 4 commits into
table-runtimefrom
table-codegen

Conversation

@iainmcgin

@iainmcgin iainmcgin commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Adds CodecStrategy::Table, the generator for the table codec proposed in #463. Unrolled stays the default. Stacked on #468 (the buffa::table runtime), which is stacked on #467.

buffa_build::Config::new()
    .codec_strategy(CodecStrategy::Table)
    .codec_strategy_in(CodecStrategy::Unrolled, &[".wa.Message"])

The plugin takes codec_strategy=table and repeatable codec_strategy_in=<path>=<strategy>. Rules match like preserve_unknown_fields_in: prefix, last match wins, rules over the global setting. A message with a oneof, map or group field, a custom string, bytes or collection type, MessageSet, or extension ranges with JSON stays unrolled, and so does every message that holds one. table_plan.rs computes that closure and reports it in one TableCodecFallbackSummary warning, silent when the user's own Unrolled rule is the cause. A rule that names such a message by its exact path is an error. compile() errors on rustc older than 1.77, and the MSRV job now also tests the table code on 1.77, so its timeout goes from 10 to 20 minutes.

The 1,644 KB to 817 KB figure in #463 was measured with oneofs and maps flattened, so a real schema saves less until the follow-up that lets table messages hold unrolled, extern and well-known-type children.

The conformance suite does not run under Table: TestAllTypesProto3 has oneofs and maps, so its messages fall back. Parity rests on buffa-test, which compiles each schema twice under renamed packages and compares bytes, sizes, decoded values and errors, including on truncated, bit-flipped and noise input. Under an experiment that set all 66 buffa-test protos to Table, 602 tests pass with 119 table messages.

A table message differs from an unrolled one in three ways, documented in the guide: a length past the end of its enclosing message fails at once with UnexpectedEof; merge_field cannot gather a non-contiguous buffer, so a type that another crate or run uses as a group or DELIMITED field must stay Unrolled; and clear() releases capacity. The generated code is tied to buffa::table, so regenerate it whenever buffa updates.

About 1,300 lines are outside test files, well over the 250-line guideline.

Followed by #475 (message fields that hold messages without a table), #476 (oneof) and #477 (map), stacked in that order.

`CodecStrategy::Table` generates a message's binary `Message` impl from a
static `buffa::table::Table` and the shared interpreters, instead of code
specialised to its fields. `CodecStrategy::Unrolled` stays the default.

The strategy is set globally with `Config::codec_strategy` (plugin option
`codec_strategy=`) and per message with `Config::codec_strategy_in` (plugin
option `codec_strategy_in=<path>=<strategy>`, repeatable). Paths use the
same prefix matching as the other `_in` rules, and the last matching rule
wins.

A table refers to the tables of its children, so a planner selects the
largest set of requested messages that can use it: oneofs, maps, groups,
custom representations, and messages that hold a message outside the set
stay unrolled. A rule that selects such a message warns, the global setting
produces one summary warning, and a rule that names the message by its exact
path is an error.

buffa-test compiles the same schemas with both strategies and checks that
they encode to the same bytes and decode to the same values, including on
truncated, bit-flipped, and random input.
@github-actions

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

Generated tables passed `::buffa::table::ABI` to `Table::new`, which
compares its argument with the same constant, so the check could never
fail. The generator now emits its own number, and a test keeps that number
equal to the runtime's.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant