Skip to content

DEV-1310: deepClone() no longer copies the connection's queues - #338

Merged
JongChern merged 2 commits into
masterfrom
DEV-1310_deepclone_live_lists
Oct 23, 2026
Merged

JongChern merged 2 commits into
masterfrom
DEV-1310_deepclone_live_lists

Conversation

@JongChern

Copy link
Copy Markdown
Contributor

DEV-1310. ShimmerPC.deepClone() copies a device by serialising it. Consensys's Bluetooth manager configures a connected device from such a copy, and so does the Java HIL. Serialisation takes no lock, so the copy walked lists that the IO thread and the timers were still changing:

  • ConcurrentModificationException out of deepClone():
    • the instruction list, which the timers add to and the IO thread retires from;
    • while streaming, the PC timestamp list, which the IO thread changes for every packet.
  • deepClone() returning null: the processing thread's packet queue holds RawBytePacketWithPCTimeStamp, which isn't Serializable, so the copy failed whenever a packet was waiting.

The fix

A clone is a copy of the configuration, not of the connection. In ShimmerBluetooth, mListofInstructions, mListofPCTimeStamps and mABQPacketByeArray are now transient, and a new readObject() gives the copy empty ones. A deserialised object runs no field initialisers, so without it they'd be null.

The HIL's HilShimmerPC gets the same change to its own log and counters (test code). The HIL's 1-in-30 configure failure that led to this ticket was its own log: the driver adds a line to it on every message, and the copy serialised thousands of lines.

Tested

Unit tests: every new test fails on master and passes here.

  • API_00034_DeepCloneWhileBusyTest:
    • clone while a thread queues and retires instructions;
    • clone while the IO thread timestamps packets;
    • clone with a packet waiting for the processing thread (null on master);
    • clone a simulator-connected device, status timer on, while an app reads battery and status (on master the clone carried the device's pending instructions).
  • HilShimmerPCTest: clone while the driver logs.

ShimmerDriverPC (96 tests) and ShimmerDriver (177) pass.

Hardware: a probe copied a connected Shimmer3 (LogAndStream v1.1.7, Classic) the way deepClone() does, over and over, and named the field being written whenever a copy threw:

Phase master this branch
Straight after connect, 10 connects 0 of 50 0 of 50
Idle, timers on, 60 s 6 of ~10,000 (HIL log) 0 of 13,035
App reading battery and status, 60 s 300 of ~7,000 (HIL log and battery times) 0 of 12,697
Streaming, 30 s 35 of ~3,000 (driver's mListofPCTimeStamps) 0 of 6,107

Not tested: the same probe on a Shimmer3R. The bench unit wasn't answering over Classic. The race is in host-side serialisation of fields the two types share, so it isn't expected to differ.

Other APIs

None of the other host APIs (C#, web SDK, pyshimmer, Swift, shimmer-logandstream) copies a device by serialisation. ShimmerAndroidAPI's Shimmer and Shimmer3BLEAndroid deepClone() the same way, but both extend ShimmerBluetooth and add no live lists of their own, so Android gets this fix through the driver pin (republish and bump).

🤖 Generated with Claude Code

JongChern and others added 2 commits October 23, 2026 10:43
ShimmerPC.deepClone() copies a device by serialising it, which takes no
lock. On a connected device the timers and the IO thread kept changing the
instruction list (and, while streaming, the PC timestamp list and packet
queue) during the copy, so deepClone() could throw
ConcurrentModificationException and the configure built on it failed.

A clone is a copy of the configuration, not of the connection: those three
queues are now transient and readObject() gives the copy empty ones.

The Java HIL's HilShimmerPC had the same problem with its own log, which
the driver's threads append to on every message; its counters are now
transient too.

Tests: API_00034 clones while a thread queues and retires instructions,
and a simulator-connected device while an app reads battery and status;
HilShimmerPCTest clones while the driver logs. All three fail on master.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
On a Shimmer3 streaming at 256 Hz, about one copy in a hundred threw
ConcurrentModificationException in mListofPCTimeStamps, which the IO
thread changes for every packet. And the processing thread's packet queue
holds packets that can't be serialised, so deepClone() returned null
whenever one was waiting. Both tests fail on master.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@JongChern
JongChern merged commit ee43fbd into master Oct 23, 2026
2 checks passed
@JongChern
JongChern deleted the DEV-1310_deepclone_live_lists branch October 23, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant