fix(traverse_util): validate keys in flatten_dict when separator is specified (#5554) - #5564
Open
Ultron09 wants to merge 1 commit into
Open
fix(traverse_util): validate keys in flatten_dict when separator is specified (#5554)#5564Ultron09 wants to merge 1 commit into
Ultron09 wants to merge 1 commit into
Conversation
…pecified (google#5554) When `flatten_dict` is called with a separator (`sep is not None`): 1. Non-string keys raised an opaque `TypeError` from deep inside string joining. 2. Keys containing the separator (e.g. `{"a/b": 1, "c": {"d": 2}}`) flattened to paths that `unflatten_dict` split, silently corrupting the round-trip structure without error. This fix: - Validates each key in `_flatten` when `sep is not None` to ensure it is a string and does not contain `sep`. - Raises a descriptive `ValueError` showing the invalid key and its full path. - Documents the validation and `ValueError` in `flatten_dict` docstrings. - Adds regression unit tests in `traverse_util_test.py` covering both cases and nested paths. Fixes google#5554.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #5554
When
flax.traverse_util.flatten_dictis called with a separator (sep is not None):{"a/b": 1, "c": {"d": 2}}withsep="/") flattens to{"a/b": 1, "c/d": 2}. Whenunflatten_dict(..., sep="/")is called,"a/b"is split into a nested path{"a": {"b": 1}}, silently corrupting the round-trip structure without error.{1: {2: 3}}withsep="/") surfaces an internalTypeError: sequence item 0: expected str instance, int foundfrom deep inside string joining.Solution
_flattenwhensep is not Noneto ensure it is a string and does not containsep.ValueError:flatten_dictdocstrings to document the string/separator requirement andValueError.traverse_util_test.pycovering non-string keys, separator-containing keys, nested dictionary paths, and verifyingsep=Nonebehavior remains unaffected.cc @IvyZX @cgarciae @jheek @Amey-Thakur