Skip to content

[FlatPostgresCollection] Sort keys before preparedStatement.addBatch() to prevent deadlocks #320

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
suddendust merged 3 commits into from
Jul 21, 2026
Merged

[FlatPostgresCollection] Sort keys before preparedStatement.addBatch() to prevent deadlocks #320

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
58db9bd
Sort keys to prevent deadlocks in updates
suddendust Jul 21, 2026
2819e9f
Use TreeMap instead of LinkedHashMap to avoid explicit sorting
suddendust Jul 21, 2026
4b37f48
Added comment
suddendust Jul 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 18 additions & 5 deletions ...tore/src/main/java/org/hypertrace/core/documentstore/postgres/FlatPostgresCollection.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Comparator;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
Expand All @@ -37,6 +38,7 @@
import java.util.Map.Entry;
import java.util.Optional;
import java.util.Set;
import java.util.TreeMap;
import java.util.stream.Collectors;
import org.hypertrace.core.documentstore.BulkArrayValueUpdateRequest;
import org.hypertrace.core.documentstore.BulkDeleteResult;
Expand Down Expand Up @@ -360,8 +362,10 @@ public boolean bulkUpsert(Map<Key, Document> documents) {
PostgresDataType pkType = getPrimaryKeyType(tableName, pkColumn);

try {
// Parse all documents
Map<Key, TypedDocument> parsedDocuments = new LinkedHashMap<>();
// TreeMap keyed by Key.toString() so the downstream JDBC batch iterates rows in a canonical
// order. Postgres acquires row locks in batch-entry order, so overlapping concurrent batches
// must share an ordering to avoid deadlocks.
Map<Key, TypedDocument> parsedDocuments = new TreeMap<>(Comparator.comparing(Key::toString));

List<Key> ignoredDocuments = new ArrayList<>();
for (Map.Entry<Key, Document> entry : documents.entrySet()) {
Expand Down Expand Up @@ -454,8 +458,10 @@ public boolean bulkCreateOrReplace(Map<Key, Document> documents) {
PostgresDataType pkType = getPrimaryKeyType(tableName, pkColumn);

try {
// Parse all documents
Map<Key, TypedDocument> parsedDocuments = new LinkedHashMap<>();
// TreeMap keyed by Key.toString() so the downstream JDBC batch iterates rows in a canonical
// order. Postgres acquires row locks in batch-entry order, so overlapping concurrent batches
// must share an ordering to avoid deadlocks.
Map<Key, TypedDocument> parsedDocuments = new TreeMap<>(Comparator.comparing(Key::toString));

List<Key> ignoredDocuments = new ArrayList<>();
for (Map.Entry<Key, Document> entry : documents.entrySet()) {
Expand Down Expand Up @@ -1022,6 +1028,13 @@ private int executeBatchUpdate(
List<Key> keys = keyGroup.getKeys();
List<List<Object>> allKeyParams = keyGroup.getKeyParams();

Integer[] order = new Integer[keys.size()];

@suddendust suddendust Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This array contains the order of the keys - We add to the batch according to this order.

for (int i = 0; i < order.length; i++) {
order[i] = i;
}
// Sort keys to ensure consistent order to avoid deadlocks
Arrays.sort(order, Comparator.comparing(i -> keys.get(i).toString()));

List<String> setFragments = new ArrayList<>(keyGroup.getSetFragments());
List<Object> timestampParam = new ArrayList<>();
appendLastUpdatedTimestamp(setFragments, timestampParam, tableName, epochMillis);
Expand All @@ -1034,7 +1047,7 @@ private int executeBatchUpdate(
LOGGER.debug("Executing batch update SQL: {} for {} keys", sql, keys.size());

try (PreparedStatement ps = connection.prepareStatement(sql)) {
for (int i = 0; i < keys.size(); i++) {
for (int i : order) {
int idx = 1;
for (Object param : allKeyParams.get(i)) {
ps.setObject(idx++, param);
Expand Down
Loading