Conversation
A map field is one table entry of kind Map whose descriptor holds the entry's key and value kinds and two functions instantiated for the collection type: one to iterate it, one to insert a decoded entry. The key and value are read and written through the interpreters that already exist, so a map adds one interpreter arm per pass and no code per key or value type. Decoding mirrors the unrolled codec: each entry counts against the element-memory limit, a message value counts against the recursion limit, an entry with an unknown closed-enum number is kept whole as an unknown field, and a repeated key or value in one entry keeps the last.
The planner accepts a map with the default collection and default string and bytes types, and follows the value of a map of messages as a child. A custom collection or element type still falls back.
Every key type, every value type, entries of unusual shape, closed and open enum values, the element-memory, unknown-field and recursion limits, and HashMap and BTreeMap collections.
The guide, DESIGN.md, the CodecStrategy documentation and the table module documentation listed maps among the fields that keep a message unrolled.
|
All contributors have signed the CLA ✍️ ✅ |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Messages with
mapfields use the table underCodecStrategy::Table. Amapfield is one table entry of kindMap. ItsMapVtholds the key and value kinds and two functions instantiated for the collection type (iterate, insert a decoded entry); keys and values go through the existing interpreters. A map value whose message has no table is reached through itsMessageimpl, as in #475. Stacked on #476.On
whatsapp.proto(sizetext bytes, fat LTO,panic=abort, owned types only), all 752 messages use the table (749 on #476; the three left had maps). Moving them adds 6,824 bytes atz, the map interpreters. Per map field, on a synthetic schema of 600 maps over 96 key/value type pairs, table against unrolled, in text bytes: 152 vs 284 atz, 425 vs 434 ats, 536 vs 465 at 3.Time, bare metal (c7i.metal-24xl),
opt-level=3, table over unrolled at 8, 64 and 512 entries: decode 1.51x, 1.30x, 1.34x; encode 2.22x, 2.00x, 2.03x;compute_size1.48x, 1.49x, 1.45x. One run, ±5%.Decoding mirrors unrolled code: element-memory and recursion limits, unknown closed-enum entries kept as unknown fields, and a failed merge leaves the same state. One difference:
clear()on a table message releases map capacity, where unrolled code keeps it.The loops test for a map before dispatching. Naming the map interpreters in the dispatch arms makes
opt-level=3builds 2.6% larger on a schema without maps and 4% larger onwhatsapp.proto.buffa::tablegains publicMapVt,DirectMsgVtandKindMarker(andKindSlotnow requiresKindMarker), documented as unstable support code. A custom collection, string or bytes type in a map still falls back, with the reason "has a field with a custom string, bytes or collection type".