Skip to content

json: read every object key as data when buffering a value - #483

Merged
iainmcgin merged 3 commits into
mainfrom
iain/json-raw-value-key
Sep 26, 2026
Merged

iainmcgin merged 3 commits into
mainfrom
iain/json-raw-value-key

Conversation

@iainmcgin

Copy link
Copy Markdown
Collaborator

When any crate in a build enables serde_json's raw_value feature, serde_json::Value's Deserialize impl reads an object whose first key is $serde_json::private::RawValue as the JSON text in the string under that key, and parses that text with a new recursion limit. buffa buffered untrusted JSON through that impl, so a sender could nest such strings past the 128-level limit, and could supply a value that differs from the one in the request text. A 2.1 MB body of 19 levels overflowed a 384 KiB thread stack. Builds without the feature are not affected.

The affected shapes were repeated, map-valued and explicit-presence enum fields, the value of a "[...]" key on a message with an extension range, and google.protobuf.Any, in generated messages and in DynamicMessage. They now buffer through BufferedValue and BufferedObject in buffa::json_helpers::buffered, whose visitors build the same Value and read every key as data. The module is public, with value, opt_value and object for #[serde(deserialize_with)], so that any type that keeps untrusted JSON as a serde_json::Value can use it.

For review:

  • The "[...]" arm is in generated code. Code generated before this change keeps the old arm until it is regenerated, and code generated after it needs this runtime. The BSR plugin must be republished with the release that carries this.
  • serde_json/raw_value is a dev-dependency feature of buffa, buffa-types, buffa-descriptor and buffa-test, so the workspace tests run only with the feature on. One test per crate fails if it is off. The conformance crate still builds without it.
  • In a build with serde_json/arbitrary_precision, a number with a fraction or an exponent, or outside the i64 and u64 ranges, decoded inside an Any payload and an extension value, and is rejected after this change, as it already was in a double field. Where the Any payload is a google.protobuf.Value, the number is read as a struct. Reading the private Number key would let an object decode as a number in a build without the feature. json: a build with serde_json/arbitrary_precision rejects every number outside the i64 and u64 integer ranges #482 tracks support.
  • main carries breaking changes, so a 0.9.x release needs this cherry-picked to a maintenance branch.

With serde_json's `raw_value` feature enabled anywhere in a build,
`serde_json::Value`'s `Deserialize` impl reads an object whose first key
is `$serde_json::private::RawValue` as the JSON text in the string under
that key, and parses it with a new recursion limit. The enum helpers,
the extension-key arm in generated code, `Any`, and the reflective WKT
decoder buffered untrusted input through that impl.

They now buffer through `BufferedValue` and `BufferedObject`, which
build the same `Value` and read every key as data, so the depth of a
decode is bounded by the deserializer that feeds it.
Removes `#[doc(hidden)]` from the module, so that a type outside buffa
that keeps untrusted JSON as a `serde_json::Value` can buffer it the
same way. `BufferedValue` and `BufferedObject` derive `Clone`, `Debug`
and `PartialEq` and convert into what they hold. `value`, `opt_value`
and `object` are functions for `#[serde(deserialize_with)]`.

The guide states that `serde_json/arbitrary_precision` is unsupported
(#482).
@github-actions

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@iainmcgin
iainmcgin added this pull request to the merge queue Sep 26, 2026
Merged via the queue into main with commit bdcf161 Sep 26, 2026
11 checks passed
@iainmcgin
iainmcgin deleted the iain/json-raw-value-key branch September 26, 2026 20:00
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants