Skip to content

[FlatPostgresCollection] Sort keys before preparedStatement.addBatch() to prevent deadlocks#320

Merged
suddendust merged 3 commits into
mainfrom
sortKeysToPreventDeadlocks
Jul 21, 2026
Merged

[FlatPostgresCollection] Sort keys before preparedStatement.addBatch() to prevent deadlocks#320
suddendust merged 3 commits into
mainfrom
sortKeysToPreventDeadlocks

Conversation

@suddendust

@suddendust suddendust commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Sort keys to prevent deadlocks in batched writes on FlatPostgresCollection

Summary

Enforces a canonical, string-sorted key order across all JDBC-batched, key-keyed write operations in FlatPostgresCollection. Postgres acquires per-row locks in batch-entry order for INSERT ... ON CONFLICT DO UPDATE and UPDATE ... WHERE key = ?, so two concurrent batches that touch overlapping key sets in opposite orders can form a wait-for cycle and one side gets aborted with ERROR: deadlock detected (SQLState 40P01).

Sorting inside the store — the last hop before addBatch(...) — makes the invariant local to the collection and independent of caller iteration order, so no downstream service has to remember to sort.

Symptom this fixes

Callers were seeing production log lines like:

Failed to execute batch update. SQL: UPDATE "myTable" SET "attributes.col1" = ?, "col2" = ? WHERE "pkCol" = ?
Error: Batch entry 8 UPDATE "myTable" ... was aborted:
ERROR: deadlock detected.

The offending SQL is produced only by the Map-based bulkUpdate on this class, and the deadlock reproduces trivially with two concurrent callers that iterate overlapping keys in opposite orders. The same class of bug also exists on bulkUpsert / bulkCreateOrReplace, which batch INSERT ... ON CONFLICT DO UPDATE — the DO UPDATE branch takes a row-exclusive lock and follows the same batch-entry ordering.

Testing

Didn't add any additional tests - The existing integration tests should be sufficient.

Checklist:

  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • Any dependent changes have been merged and published in downstream modules

List<Key> keys = keyGroup.getKeys();
List<List<Object>> allKeyParams = keyGroup.getKeyParams();

Integer[] order = new Integer[keys.size()];

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This array contains the order of the keys - We add to the batch according to this order.

@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.90%. Comparing base (b381c1b) to head (4b37f48).

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #320      +/-   ##
============================================
+ Coverage     80.89%   80.90%   +0.01%     
  Complexity     1580     1580              
============================================
  Files           242      242              
  Lines          7628     7632       +4     
  Branches        751      752       +1     
============================================
+ Hits           6171     6175       +4     
  Misses          968      968              
  Partials        489      489              
Flag Coverage Δ
integration 80.90% <100.00%> (+0.01%) ⬆️
unit 55.31% <14.28%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown

Test Results

  124 files  ±0    124 suites  ±0   38s ⏱️ ±0s
  846 tests ±0    845 ✅ ±0  1 💤 ±0  0 ❌ ±0 
1 183 runs  ±0  1 182 ✅ ±0  1 💤 ±0  0 ❌ ±0 

Results for commit 4b37f48. ± Comparison against base commit b381c1b.

♻️ This comment has been updated with latest results.

@suddendust suddendust changed the title Sort keys to prevent deadlocks in updates [FlatPostgresCollection] Sort keys before preparedStatement.addBatch() to prevent deadlocks Jul 21, 2026
@suddendust
suddendust merged commit d5c2963 into main Jul 21, 2026
7 checks passed
@suddendust
suddendust deleted the sortKeysToPreventDeadlocks branch July 21, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants