Skip to content

DEV-1269: Java HIL v1, the driver on real Shimmer3/3R over Classic, and a release gate - #331

Merged
JongChern merged 7 commits into
masterfrom
DEV-1269_java_hil
Oct 22, 2026
Merged

JongChern merged 7 commits into
masterfrom
DEV-1269_java_hil

Conversation

@JongChern

@JongChern JongChern commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What this adds

Hardware-in-the-loop tests for the Java driver. A PC connects to real Shimmer3 and Shimmer3R units over their paired Bluetooth Classic serial ports, through ShimmerPC with the driver's timers on, as Consensys and MATLAB run it. One command builds, runs every scenario on every unit, and prints pass/fail:

.\tools\hil\run-java-hil.ps1

Setup, scenarios and output are in tools/hil/README.md.

  • JavaHilScenariosTest, run by a new hilTest task. Without -Phil.port it is skipped, and the normal test task leaves it out, so CI is unaffected.

    • streamRest: stream at rest; check loss, timestamps, rate, silence and accel.
    • midStreamSet: SD logging started the way Consensys does (stop streaming, start logging on the quiet link, stream again; DEV-436), stopped mid-stream, and battery reads while streaming.
    • reconnectLoop: five connects on one ShimmerPC.
    • inStream: each in-stream reply (status, directory, battery) on its own. Then the DEV-1262 case: with the unit streaming and logging (started the Consensys way), stop SD logging mid-stream and request status and battery at once.
    • queueStress: while streaming, four app threads queue reads and check the queue length in tight loops for 60 s, under a hang watchdog. That is DEV-1266's deadlock on demand, with no SD logging (DEV-1266: Take the instruction list's two locks in one order #334).
  • Stream checks per stretch between the deliberate restarts. Loss within 1 s of an SD logging stop is reported but doesn't fail, because the unit pauses while it closes its file. Loss near a battery read fails, and loss anywhere else is held to the limit.

  • No connect retries, and a failed connect says why. "Port busy" means another program on the PC has the COM port open: another HIL, Consensys, a serial terminal. That fails the scenario at once, naming the cause, because results from a unit shared with another host mean nothing. A CONNECTION_FAILED for any other reason, or a unit that never initialises, is reported as such. The README says nothing else may use the units during a run, and explains why a BLE host is the dangerous case: a Shimmer3R accepts it during a Classic stream.

  • Watchdog: every API call runs under one. If a call hangs, the report gets every thread's stack, with the locks each holds and any deadlock the JVM detects (DEV-1266).

  • HilShimmerPC, a test-only subclass, does the counting without changing what the driver does:

    • in-stream replies by type, decoded statuses and battery readings;
    • every driver log line, verbose or not;
    • which timers are running.

    Its configure step does not write the calibration dump back (configureFromClone does by default, AA-246), so runs leave the bench units' calibration alone.

  • StreamAnalysis is the Android HIL's class, so both HILs judge a stream by the same rules. StreamAnalysisTest runs in CI.

  • tools/hil/run-java-hil.ps1:

    • finds each unit's COM port from its MAC and builds once;
    • runs each scenario in its own Gradle run, with a time limit;
    • checks that every report came back.
    • -Report posts the commit status java-hil, for the full default run on a clean, pushed commit only.
  • Release Java API now stops unless java-hil passed on the commit, or hil_waiver gives a Jira key. The release notes record which, and dry runs only warn.

Heads-up: the gate needs the three driver fixes this PR found

On master as it is, the gate fails:

With all three on top of this branch, a full run passed on both units. Until they merge, a release needs hil_waiver.

DEV-1245 no longer trips the HIL. That fault came from writing the RTC mid-stream, and Consensys never does that. The scenarios now follow Consensys's order, so they don't either.

Tested on the bench (2026-10-21)

A full run of all four scenarios, on the PR's head, whose driver is master's (dffb582c) unchanged. Bench PC, units paired over Classic, the driver's default CRC (1 byte), the Shimmer3 undocked, and nothing else using either unit.

Shimmer3 3E36, v1.1.7 Shimmer3R 86F8, v1.1.21
streamRest 7655 packets, 0 lost, 256.00 Hz; accel NaN in every sample (see below); passed only because the check skipped NaN, fixed since passes: 7649 packets, 0 lost, 256.00 Hz, accel 9.66 m/s²
midStreamSet (Consensys order) passes; 13 packets missing, all at SD logging stops passes; 62 missing, all at SD logging stops; no timestamp faults
reconnectLoop passes, 5 connects passes, 5 connects
inStream fails in 1 of 5 overlaps on the lost reply below; 3 directory, 9 status and 8 battery replies otherwise fails in 4 of 5 overlaps on the lost reply below

Every connect succeeded first time, and no packet was lost near a battery read or anywhere away from an SD logging stop.

Earlier runs today overlapped two other HILs (C# and shimmer-logandstream) on the same units. Every "Port busy" this PR saw came from that clash, and so did DEV-1276's evidence. Those runs aren't counted here.

Checked after removing the retry (2026-10-22, Shimmer3 3E36):

  • With COM43 held open by another process: streamRest fails at once with "COM43 is open in another program (another HIL, Consensys, a serial terminal?): jssc reports "Port busy"".
  • With the port free: reconnectLoop connected all 5 cycles. Two other faults showed up on the way, neither from this change. Both runs (11:48 and 11:50) also overlapped a C# HIL streaming another unit through the same PC Bluetooth adapter, which it measured at about 240 ms radio silences, so treat them as seen under a busy radio:
    • A connect that hung. The driver's throwaway GET_SAMPLING_RATE got its reply 273 ms late, which was read as GET_SHIMMER_VERSION's ACK. The version was never read, and the connect timer gave up after 20 s ("Lost connection").
    • The Shimmer3's own sample loss over Classic: 49 packets in short bursts, with no parser errors and the unit undocked (the same loss as DEV-1248).

What it found:

  • DEV-1297, firmware: a Shimmer3 (v1.1.7) sometimes skips one sample to answer a battery read mid-stream. That was 4 skips in 33 reads, each exactly at the battery reply, with no parser errors, and at the same rate with and without the driver fixes. The check reports a single sample skipped within 25 ms of a decoded battery reply rather than failing on it. Any other loss near a battery read still fails. Before this, midStreamSet failed on the Shimmer3 about one run in three. Afterwards 4 of 4 runs passed, two of them reporting one skip.
  • DEV-1266 reproduced on hardware, with no SD logging. queueStress on a Shimmer3R deadlocked within 0.3 s in both runs without DEV-1266: Take the instruction list's two locks in one order #334: the IO thread held the instruction list in getInstruction() and waited for the device, while a stress thread held the device and waited for the list. With DEV-1266: Take the instruction list's two locks in one order #334, two 60-second runs were clean, with about 670 calls and 0 timeouts each.
  • DEV-1291: a reply that arrives during the stream's pause after an SD logging stop is thrown away. Both units do it, the Shimmer3 less often. After STOP_LOGGING_ONLY the Shimmer3R stops sending data packets for about 100 ms while it closes its file. A status request answered in that pause sends FF 8A 71 .. straight after the previous ACK, with no data packet before it. processPacket() only looks for an ACK after a data packet (there is a TODO: ACK in bufferTemp[0] not handled), so it throws the reply away. The command then times out after 2 s, and the timeout's clearAllInstructions() silently drops the battery request queued behind it. Android's Classic parser (Shimmer.java) has the same TODO.
  • DEV-1262's fix (DEV-1262: while streaming, complete the command in flight only on its own in-stream reply #330) holds on hardware: no in-stream reply matched to the wrong command on either unit.
  • DEV-1245, narrowed down: on the Shimmer3R, writing the RTC while streaming is enough to cause the out-of-order timestamps. There were 6 faults from 13 writes, each 130–292 ms after one, with no SD logging involved. In Consensys's order there were none.
  • DEV-1275, new: a status reply during the connect sequence reports CONNECTED and initialised while about 9 connect commands are still queued. Streaming started then can take a connect reply into the stream and die.
  • The Shimmer3's accel came out NaN in streamRest, a second symptom of DEV-1275. The driver reported the unit ready while its connect sequence was still reading the calibration dump. The HIL's configure then took over the queue mid-transfer, and the dump's second chunk was cleared as junk. The driver kept 128 of the dump's 536 bytes, and all 7655 samples were NaN. The dump itself is fine: its accel block holds the standard defaults. In the other scenarios the configure came early enough that no dump was read, and accel read about 10.9 m/s² at rest from this unit's stored calibration (11.1 on v1.1.5 this morning), so that calibration is off too. None of this code has changed since 0.11.9_beta.
  • The HIL's own timeout count was broken in the first commit: its pattern never matched the driver's [0x72] GET_STATUS_COMMAND form, so commandTimeouts was always 0. Fixed; it now finds all 14 timeouts in the bench logs.

Also checked:

  • gradlew :test: StreamAnalysisTest 8 of 8; the hardware scenarios are left out.
  • gradlew :hilTest without a port: all four skipped.
  • The release workflow's HIL check follows the Android one, which was tested against real and mocked statuses. This PR dry-runs the workflow, since it changes it.

Notes

  • The checkout's folder must be named Shimmer-Java-Android-API, because ShimmerDriverPC's settings.gradle finds its siblings that way. In a worktree, use a junction; the README explains how.
  • hilTest should be run with --no-watch-fs (the runner does). Gradle's file watcher misses edits made through a junction's real path, and would run stale classes.

🤖 Generated with Claude Code

…Classic, and a release gate

Hardware-in-the-loop tests of ShimmerDriver/ShimmerDriverPC on a PC with paired units, through
ShimmerPC with its timers on, as Consensys and MATLAB run it.

- JavaHilScenariosTest (hilTest task, skipped without -Phil.port, left out of the normal test
  task): streamRest, midStreamSet, reconnectLoop and inStream. inStream requests each in-stream
  reply (status, directory, battery) and then overlaps SET_RWC with status and battery requests,
  the DEV-1262 case. Every API call runs under a watchdog that dumps every thread with its locks.
- HilShimmerPC, a test-only subclass, counts in-stream replies by type, decoded statuses and
  battery readings, and captures every driver log line, without changing what the driver does.
  The configure step does not write the calibration dump back, so runs leave the units' stored
  calibration alone.
- StreamAnalysis is the Android HIL's class (same rules on both); its tests run in CI.
- tools/hil/run-java-hil.ps1 finds each unit's COM port from its MAC, builds once, runs each
  scenario in its own Gradle run under a time limit, checks every report came back, and with
  -Report posts the commit status java-hil (full default run, clean pushed commit only).
- Release Java API stops unless java-hil passed on the commit, or hil_waiver gives a Jira key;
  the release notes record which.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI lite review requested due to automatic review settings October 21, 2026 08:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

…ink Windows can't open

midStreamSet and inStream now start SD logging as Consensys does (DEV-436):
stop streaming, start logging on the quiet link, then stream again. SD
logging is still stopped mid-stream. Consensys never writes the RTC
mid-stream, so the scenarios no longer do either; that is what caused
DEV-1245's out-of-order timestamps on the Shimmer3R.

The stream is judged per stretch between the deliberate restarts. Loss
within 1 s of an SD logging stop is reported but does not fail, because
the unit pauses while it closes its file. Loss near a battery read fails,
and loss anywhere else is held to the limit.

A connect whose Bluetooth link can't be opened (CONNECTION_FAILED; jssc
reports "Port busy") is tried once more, and connectRetries in the report
counts those retries.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI lite review requested due to automatic review settings October 21, 2026 08:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

The pattern expected the command as one word, but the driver logs it as
"[0x72] GET_STATUS_COMMAND". So every commandTimeouts count was 0, and the
"no command timeouts" checks could never fail. Checked against the bench
logs so far: it now finds all 14 timeouts in them.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI lite review requested due to automatic review settings October 21, 2026 08:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

The accel check was skipped when the mean was NaN, so a stream with no
calibrated accel passed. A Shimmer3 (3E36, LogAndStream v1.1.7) did exactly
that: all 7655 samples NaN after the driver read its calibration dump.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copilot AI lite review requested due to automatic review settings October 21, 2026 12:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

…trying it

Every "Port busy" the HIL ever met was another program holding the COM
port, never a flaky first connect. Two HILs shared a unit, and that sharing
produced the misleading DEV-1276. A retry only lets the run go on while the
unit is shared, so its results mean nothing.

So a connect is no longer retried. When it fails, the report says why:
- the port is open in another program (jssc "Port busy"), which the HIL
  now sees because HilShimmerPC records the messages the driver prints
  with consolePrintException;
- the link could not be opened for another reason;
- the unit did not initialise in time.

The README now says that nothing else may use the units during a run, and
why a BLE host is the dangerous case.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
JongChern and others added 2 commits October 22, 2026 14:40
While a unit streams, four app threads call readBattery() and
readStatusLogAndStream() and check the queue length, in tight loops for 60 s,
keeping the queue under 8. Each of those calls locks the device and then the
instruction list, so one is nearly always under way while the IO thread,
holding the list, fetches the next instruction. That is the window DEV-1266's
lock inversion needs. No SD logging is involved.

The scenario fails if the calls and replies stop for 10 s, or if the JVM
finds a deadlock. Either way the events get every thread's stack, and the
unit is left alone.

On a Shimmer3R (v1.1.21) without the DEV-1266 fix (#334), it deadlocked
within 0.3 s in both runs. With the fix, two 60-second runs were clean, with
about 670 calls and 0 timeouts each.

It runs by default, so the release gate includes it. Until #334 merges, the
gate fails on a deadlock.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A Shimmer3 (LogAndStream v1.1.7) sometimes gives up one sample slot to
answer GET_VBATT mid-stream: packet n, the battery reply, then packet n+2.
That was 4 skips in 33 battery reads. It's the unit, not the host: there
were no parser errors, and builds with and without the driver fixes skip at
the same rate. Filed for the firmware as DEV-1297. The check charged it as
packet loss near a battery read, so midStreamSet failed on the Shimmer3
about one run in three.

A gap near a battery read is now excused only when it is exactly one sample
and the packet after it arrived within 25 ms of a decoded battery reply.
Such a gap is reported, as skippedAtBatteryReplies plus an event naming
DEV-1297. Anything else near a battery read still fails. A packet the driver
dropped would leave stray bytes, which checkDriverLog() fails on anyway.
HilShimmerPC now records when each battery reply is decoded.

Checked on 3E36: 4 midStreamSet runs all passed. Two of them reported one
skip at a battery reply, and there was no other loss near any read.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@JongChern
JongChern merged commit 3847ae5 into master Oct 22, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants