Make JSON depth limits and experiment verification deterministic - #9
Merged
Conversation
There was a problem hiding this comment.
🟢 Approval recommended
The changes are cohesive, low-risk, and include targeted regression tests that enforce the intended deterministic behavior and failure propagation.
Pull request overview
This PR makes JSON depth rejection deterministic across Python versions by enforcing a fixed container-nesting limit before calling json.loads, and ensures Make-based verification fails fast so deterministic artifact comparisons can’t be accidentally masked.
Changes:
- Add a string-aware, pre-decode JSON nesting validator with a fixed limit (
MAX_JSON_NESTING = 64) and ensure excessive depth is rejected before decoding. - Add regression tests covering the exact depth boundary, string/escape handling (quoted braces), and “reject-before-decode” behavior.
- Update verification/evidence artifacts and add engineering notes documenting the rationale and reproducibility implications.
File summaries
| File | Description |
|---|---|
| tests/test_json_depth_contract.py | Adds contract tests for exact nesting boundary, quoted brace handling, and pre-decode rejection. |
| src/memoryflow/io.py | Introduces MAX_JSON_NESTING and a pre-decode nesting scan to make depth limits deterministic. |
| site/results.json | Updates recorded source hash for regenerated evidence inputs. |
| README.md | Adds quick links to engineering notes, core model, and reproduction section. |
| Makefile | Sets shell flags to fail fast (-eu) so verification comparisons can’t be masked. |
| evidence/benchmark-summary.md | Updates model-source hash to match regenerated evidence. |
| docs/engineering-notes.md | Documents the new deterministic JSON boundary and verification/evidence expectations. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JSON depth rejection depended on Python recursion behavior, and chained Makefile commands could hide comparison failures. Enforce a string-aware depth limit before decoding and propagate verification failures.
Add boundary regression tests and engineering notes; regenerate synthetic simulation evidence from the changed source. Existing MPS measurements were checked, not re-measured.
Validation: make verify — 189 tests, 96.58% coverage, lint/type checks, deterministic reproduction, measurement artifact checks, site checks, and wheel/sdist validation.