Skip to content

table: use the table for messages with a oneof - #476

Draft
iainmcgin wants to merge 26 commits into
table-bridgefrom
table-oneof
Draft

iainmcgin wants to merge 26 commits into
table-bridgefrom
table-oneof

Conversation

@iainmcgin

Copy link
Copy Markdown
Collaborator

Messages with a oneof use the table under CodecStrategy::Table. On the WhatsApp schema (752 messages) with the table requested, 43 messages fall back on the bridge PR (#475) and 3 here, all with map fields. Text section, fat LTO, bridge PR → this one (fully unrolled in brackets):

opt-level bytes
z 1,009,611 → 961,027 (1,669,499)
s 1,085,504 → 1,008,384 (2,512,992)
3 1,264,660 → 1,099,420 (2,885,828)

A oneof is an Option<Enum> of unspecified layout, so a member has no offset. Every member gets a table entry at the offset of the Option, and generated code implements the new buffa::table::OneofEnum: the number of the member that is set, a pointer to its value, and a way to set a member by number. The impl is safe code that names each value's type, so a mismatch does not compile and generated crates keep forbid(unsafe_code). The lowest-numbered member sizes and writes whichever member is set, at its own position, so the bytes equal unrolled code's; the others only decode. Decoding follows unrolled code, including the message a failed merge leaves behind. A message member whose message has no table is reached through its Message impl, and #475's bytes rule covers oneof members. Only a oneof with a message member gets in-place decoding (1.2 KB at z).

EnumShape, an unsafe trait, gains a required method accepts that must agree with set; if it does not, a rejected closed-enum value leaves a default member where another member was set. The new unsafe (table/oneof.rs, table/shape.rs, the interpreters) runs under Miri through hand-written enums (CI: Stacked Borrows; Tree Borrows run locally). Generated OneofEnum code runs natively only.

Table ÷ unrolled for a message of two oneofs and two fields, by instruction count: size 4.9×, encode 2.9×, decode 1.4×.

Stacked on #475.

A oneof is an Option of an enum whose layout is unspecified, so a table
entry cannot address a member by offset. Add the OneofMember kind, a
OneofEnum trait that generated code implements for the enum, and the
descriptors the interpreters use to read the member that is set and to
replace it. Members decode by their payload kind through the arms that
ordinary fields use, and the member with the lowest number writes and
sizes the whole oneof, so the bytes match unrolled code.
The planner no longer rejects a oneof. Each member gets a table entry of
its own, all at the offset of the field that holds the oneof, and the
oneof's enum implements the accessor trait the interpreters read it
through. The accessors are safe code, so generated crates that forbid
unsafe code still compile, and they name the type of each member's value
so a mismatch is a compile error.
Members of every type, two oneofs interleaved with ordinary fields,
recursion through a oneof, sparse and extreme field numbers, nested and
keyword-named members, closed enums that reject a number, merging into a
oneof that is set, and a build with a type prefix, no unknown fields and
inline message members.
The member with the lowest number sizes and writes the whole oneof, and
the others used to enter the same function to find that out. A separate
kind for the leader lets the followers fall through the dispatch, which
takes the size pass of a message with two oneofs from 981 to 603
instructions.
A message member that is not the one that is set is decoded into a new
default member, which becomes the one that is set only if the decoding
succeeds, so a failure leaves the oneof as it was. Before, the default
member was installed first and a failure left it in place.

The oneof accessors also check that the default member they build is
the one that was asked for, and a payload lookup checks that the entry
is a member of the calling oneof. A message member's descriptor must be
one that reaches the message directly.
The generated payload accessors turned a reference into a raw pointer
with an `as` cast, which `trivial_casts` flags in user crates. They now
coerce it, which does not, and the same reference type still rejects a
variant of another type. The planner no longer sets a placeholder kind
on oneof members, the error for too many descriptors is shared, and the
codegen test fixtures are named for what they hold.
… closed enums

The differential tests now compare the message left after a failed merge
between the table and unrolled codecs, for a member of every kind, for
message members, and for the recursion and size limits. New schemas cover
a custom pointer for message members, oneofs named like Rust keywords with
members out of number order and idiomatic field names, a closed enum in a
oneof with no unknown fields, and a oneof whose members are messages of
another package in each layout.
Decoding a oneof now follows unrolled code exactly, so the documented
difference goes. The size claims quote the measured schema, the oneof
mechanism is described in one place, and the changelog fragment for the
oneof work is folded into the table codec's.
place_with swaps the new member into the oneof on success and drops
whichever member is left in the local afterwards, so the drop glue of the
oneof enum is instantiated once in it, not twice.
Cover the round trip into every kind of sink, merging into the member that
is set, and a failed decode, for a member whose message has no table here.
The Table::new safety docs and the check_member message name
MsgVt::direct_via_message next to MsgVt::direct.
A member whose message is set to unrolled, and one whose message another crate generates, are reached through the message's impl.
A oneof member that holds a message set to unrolled, a well-known type, a
message from another crate, and a message with a bytes field of a non-default
type now goes through the table in a differential test. The holder of the last
stays unrolled, and decodes its Bytes fields without a copy.
3 of 752 messages stay unrolled, for their maps, and the text section at opt-level z is 42% below unrolled code.
The generated MsgVt::direct names the payload type, so a table of another
message is a type error, with a compile_fail doctest. Table::new rejects a
direct descriptor on an ordinary message field.
The test read the slot between calls through one pointer, which Tree Borrows
rejects. The table tests now pass under both Stacked and Tree Borrows, and the
CI comment names the oneof cases Miri covers.
Also correct the comment on the swap in place_with_impl: the member that is
dropped there is the one the new member replaced, and a new member that
failed to decode is dropped where fresh goes out of scope.
…ract

The module documentation of the private oneof module was not rendered, so the
mechanism moves to the trait, and it now says that decoding uses arms of its
own. EnumShape::accepts must agree with set, the __table_entry! docs give the
leader argument, and a SAFETY comment names the field it means.
… and wide oneofs

A failed merge into a message member stored inline agrees with unrolled code.
Members that share a message or enum type share their descriptor, checked in
the generated table and by a differential test. A oneof in a message of 258
entries, which has no dense lookup, agrees with unrolled code.
The buffa-test one covers a oneof as well as a message with a map, and the
codegen one is about a holder that is a table.
…a message member

OneofVt::new no longer instantiates the in-place decoding for its enum, and
OneofVt::with_messages does. Generated code picks the constructor from the
oneof's members, and Table::new rejects a message member of a oneof built
with new.
@github-actions

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant