Skip to content

fix(traverse_util): validate keys in flatten_dict when separator is specified (#5554) - #5564

Open
Ultron09 wants to merge 1 commit into
google:mainfrom
Ultron09:fix/flatten-dict-separator-validation
Open

fix(traverse_util): validate keys in flatten_dict when separator is specified (#5554)#5564
Ultron09 wants to merge 1 commit into
google:mainfrom
Ultron09:fix/flatten-dict-separator-validation

Conversation

@Ultron09

Copy link
Copy Markdown

Summary

Fixes #5554

When flax.traverse_util.flatten_dict is called with a separator (sep is not None):

  1. Case 1 (Structural corruption): A string key containing the separator (e.g. {"a/b": 1, "c": {"d": 2}} with sep="/") flattens to {"a/b": 1, "c/d": 2}. When unflatten_dict(..., sep="/") is called, "a/b" is split into a nested path {"a": {"b": 1}}, silently corrupting the round-trip structure without error.
  2. Case 2 (Opaque TypeError): A non-string key (e.g. {1: {2: 3}} with sep="/") surfaces an internal TypeError: sequence item 0: expected str instance, int found from deep inside string joining.

Solution

  • Validates each key in _flatten when sep is not None to ensure it is a string and does not contain sep.
  • Raises a descriptive ValueError:
    ValueError: flatten_dict with sep='/' requires string keys that do not contain the separator; got key 'a/b' at path ('a/b',)
    
  • Updates flatten_dict docstrings to document the string/separator requirement and ValueError.
  • Added unit tests in traverse_util_test.py covering non-string keys, separator-containing keys, nested dictionary paths, and verifying sep=None behavior remains unaffected.

cc @IvyZX @cgarciae @jheek @Amey-Thakur

…pecified (google#5554)

When `flatten_dict` is called with a separator (`sep is not None`):
1. Non-string keys raised an opaque `TypeError` from deep inside string joining.
2. Keys containing the separator (e.g. `{"a/b": 1, "c": {"d": 2}}`) flattened to paths that `unflatten_dict` split, silently corrupting the round-trip structure without error.

This fix:
- Validates each key in `_flatten` when `sep is not None` to ensure it is a string and does not contain `sep`.
- Raises a descriptive `ValueError` showing the invalid key and its full path.
- Documents the validation and `ValueError` in `flatten_dict` docstrings.
- Adds regression unit tests in `traverse_util_test.py` covering both cases and nested paths.

Fixes google#5554.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

flatten_dict with a separator cannot round-trip keys containing the separator, and raises an opaque error on non-string keys

2 participants