Skip to content

DEV-976: switch the link CRC at the SET_CRC ACK, which comes in the new mode - #212

Merged
JongChern merged 1 commit into
masterfrom
DEV-976_crc_switch_ack_in_new_mode
Oct 20, 2026
Merged

JongChern merged 1 commit into
masterfrom
DEV-976_crc_switch_ack_in_new_mode

Conversation

@marknolan

@marknolan marknolan commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

DEV-976. The C# side of ShimmerResearch/shimmer-web-sdk#176 ("turning the link CRC off no longer loses its ACK"). In C# the switch that breaks is the opposite one: turning a two-byte CRC on.

Stacked on #211. This branch is based on DEV-976_refuse_crc_on_firmware_that_drops_it, because both PRs change WriteCRCMode and the new test builds on #211's CrcFirmwareDevice. It targets master so that CI runs. Until #211 merges, the diff also shows #211's two commits, and only 59bec31 is this PR's. Merge #211 first.

The bug

  • The firmware switches its CRC before it builds the ACK to SET_CRC_COMMAND, so the ACK is framed in the new mode. In shimmer3r-firmware at LogAndStream_Shimmer3R_v1.00.010, BtUart_processCmd calls setBtCrcMode (shimmer_bt_comms.c:1262), and BtUart_sendRsp reads the mode when it builds the reply (:2344). From v1.00.019 the code is in log-and-stream-common (Comms/shimmer_bt_uart.c:944 and :2422 at v1.01.017). I checked every tagged Shimmer3R and Shimmer3 LogAndStream release that has the command, and all of them do this.
  • WriteCRCMode recorded the new mode only after its 300 ms sleep, so ReadData parsed that ACK in the old mode.
  • The connected-state loop never drops a bare ACK's CRC. It drops CRC bytes only after a byte that is not 0xFF. The CRC of a bare ACK is a constant: F4 65 by the API's own ShimmerUartCrcCalc.

Off → two bytes broke. The ACK is FF F4 65. F4 is no response code, but 0x65 is DETECT_EXPANSION_BOARD_RESPONSE. It took the next reply's ACK (0xFF) for a length and swallowed the 255 bytes after it, ending early only when a read timed out after 1 s of silence.

  • A reply to the command after WriteCRCMode(TWO_BYTE) is lost.
  • A stream started straight afterwards is never seen. The start command's ACK goes into the swallow, so the API stays Connected and delivers no packet. When the 255 bytes run out, the expansion board read at connect becomes "Unknown" (ID 244, which is 0xF4). With the test double: 98 packets sent, 0 delivered, state Connected after 3 s.
  • ShimmerConsoleAppExample connects this way (TWO_BYTE in its constructor). Reading the code, at connect the swallow takes the configuration writes' ACKs and the status reply. It ends during the 1.5 s waitTilTimeOut that follows, because the status reply never arrives. So that connect loses its status read. This was not run.

The other switches only worked by luck:

Switch ACK on the wire Before this PR Why
off → 1 FF F4 ok F4 is no response code
off → 2 FF F4 65 next reply lost 0x65 read as an expansion board response
1 → off, 2 → off FF ok nothing to drop
1 → 2 FF F4 65 ok F4's one-byte drop takes the 65
2 → 1 FF F4 ok, by timing F4's drop loop rereads the mode for each byte. WriteCRCMode has set one byte before the next reply arrives, so the loop drops only that reply's ACK

ShimmerCapture's CRC checkbox only switches between off and one byte, so it was never affected.

The fix

WriteCRCMode sets mWaitingForCRCModeACK and mCRCModeAfterACK before it writes. When the ACK arrives, the connected-state ACK handler switches to that mode and drops that many CRC bytes, as the firmware framed it.

  • The flag is cleared when the wait ends, however it ends (in a finally), so a later reply's ACK is never taken for this one.
  • If no ACK arrives within the 300 ms, the mode is applied at the end of the wait, as before. A late ACK is then read in the new mode, and the next reply still arrives intact (tested).
  • A switch made while streaming is unchanged. ShimmerCapture disables its checkbox while streaming. The firmware can also send packets in the new mode ahead of the ACK, so mid-stream there is no exact point to switch at.

Testing

ShimmerUnitTests/ShimmerCrcSwitchTest runs the real ReadData against CrcLinkDevice, a CrcFirmwareDevice that also answers the way the firmware does:

  • it applies SET_CRC before it ACKs,
  • it frames every reply in its own mode with ShimmerUartCrcCalc,
  • it streams data packets after START_STREAMING,
  • its reads time out after ReadTimeout, as a serial port's do.

The tests cover:

  • Every switch (6 tests), each followed by two sampling-rate reads that must arrive intact.
  • Every switch in turn on one link: off → 1 → 2 → 1 → off → 2 → off, the sequence DEV-976: turning the link CRC off no longer loses its ACK shimmer-web-sdk#176 ran on hardware.
  • A stream started straight after turning two bytes on.
  • An ACK that arrives after the wait, and one that never arrives.
  • The CRC of a bare ACK is F4 65, and 0x65 is DETECT_EXPANSION_BOARD_RESPONSE.
Library ShimmerCrcSwitchTest
without the fix 3 of 11 fail: off → 2, every switch in turn (at its off → 2 step), and the stream
the fix without dropping the ACK's CRC 1 fails: the stream
the fix with the flag left set after the wait 1 fails: the ACK that never arrives
this PR 11 of 11 in each of 3 runs. ShimmerBluetoothTests: 46 of 46

Built with Visual Studio 2022's MSBuild (a full rebuild of ShimmerAPITest.sln) and run under vstest.console, as CI does.

Hardware: not run. This rests on the firmware source and the test double. To check it on a Shimmer3R on LogAndStream 1.0.11 or later, or a Shimmer3, call WriteCRCMode(TWO_BYTE) and then StartStreaming(): data should arrive. Before this PR, none did.

Found, not fixed here

With a CRC on, the connected-state loop drops no bare ACK's CRC in steady state either. Its F4 then costs the next reply its first byte. That byte is usually the reply's own ACK, which does no harm. The exception is the START_STREAMING ACK:

  • A SET command followed within 1 s by StartStreaming() leaves the API Connected with no data.
  • With the test double, WriteAccelRange then StartStreaming(): one byte delivered 0 of 66 packets, two bytes 0 of 65, and off streamed.
  • StartStreamingandLog is not affected, because it reads the status first.

This is older than DEV-976 and has nothing to do with switching, so it is left for a separate change. Fixing it changes how every ACK is parsed while a CRC is on.

Other copies in this repo

  • ShimmerCaptureXamarin/ShimmerBluetooth.cs has its own ReadData but no CRC support, so it cannot turn one on.
  • ShimmerAPI/TestSerialPort/Program.cs has no CRC code.
  • The transport subclasses use ShimmerBluetooth.ReadData, so they get the fix: ShimmerLogAndStreamSystemSerialPort, the 32feet classes, and ShimmerLogAndStreamBLE.

Sibling APIs

Neither Java nor Swift needs this. Both were traced on paper; neither was run.

  • web SDK: fixed in DEV-976: turning the link CRC off no longer loses its ACK shimmer-web-sdk#176 (merged). It broke in the other direction, turning the CRC off, because its framer waits for the old mode's trailer. That PR judged C# unaffected because C# "never waits for the ACK". That is true for turning the CRC off, but it missed the 0x65.
  • Java (Shimmer-Java-Android-API master, c86a5748): does not need it.
    • It switches its mode on the IO thread, at the moment it reads the SET_CRC ACK (ShimmerBluetooth.java:2287-2289), so the next reply is read in the new mode.
    • It never reads a bare ACK's CRC either. But it clears the serial buffer before every command while not streaming (:687-689), and drops non-FF bytes while it waits for an ACK. So a stray F4 65 never reaches the response parser, and the steady-state problem above does not arise.
    • A normal connect is off → one byte. DEV-976: stop turning the link CRC on for Shimmer3R firmware that drops it at a stop Shimmer-Java-Android-API#316 changes only the gating.
  • SwiftAPI (main, c977149): does not need it.
    • sendCRCCommand sets CRCMode right after the write and before it awaits the reply (Shimmer3Protocol.swift:1458-1460). The SET_CRC handler then reads the ACK at that width (:695-713).
    • Every SET handler consumes its bare ACK's CRC, so the steady-state problem does not arise here either.
    • On main the only switch is off → two bytes, in connect(). add crc and PRR UI SwiftAPI#22 makes every switch reachable, and all of them are read at the new width.
    • The mirror case is a NACK. A refused SET_CRC comes back in the old mode, but Swift has already switched. Main has no NACK handling at all. DEV-982: handle NACK from a Shimmer3 instead of leaving the caller suspended SwiftAPI#18 drops a NACK as 1 + CRCMode bytes at the new width and keeps the new mode. That fix belongs in SwiftAPI's NACK PR: read a NACK to SET_CRC at the old width, and put the old mode back.
  • pyshimmer has no link CRC.

Wrappers and consumers

  • Shimmer-Labview-API pins the ShimmerAPI NuGet package at 1.3.10 and uses the two-argument ShimmerLogAndStreamSystemSerialPort constructor. It never calls WriteCRCMode, so it never turns a CRC on and is not exposed. It picks this up only when it is rebuilt against a later release.
  • ASM_BaseStation never turns a Shimmer3/3R link CRC on.

Overlap with open PRs

Checked with git merge-tree against this branch:

#204 (NACK handling) composes with this one. A NACKed SET_CRC skips the firmware's switch and arrives in the old mode. This PR switches only on an ACK, so #204 drops the NACK's CRC in the right mode. Two things are left for after both merge:

🤖 Generated with Claude Code

…ew mode

The firmware switches its Bluetooth CRC while it processes SET_CRC_COMMAND,
before it builds the ACK, so the ACK is framed in the new mode
(shimmer3r-firmware shimmer_bt_comms.c:1262 and :2344 at
LogAndStream_Shimmer3R_v1.00.010, log-and-stream-common
Comms/shimmer_bt_uart.c:944 and :2422 at v1.01.017, and the same in every
tagged Shimmer3 release with the command). WriteCRCMode recorded the new
mode only after its 300 ms sleep, so ReadData parsed that ACK in the old one.

Off to two bytes broke. The ACK is FF F4 65, where F4 65 is the CRC of a
bare ACK, and the connected-state loop never drops a bare ACK's CRC. F4 is
no response code, but 0x65 is DETECT_EXPANSION_BOARD_RESPONSE. It took the
next reply's ACK for a length of 255 and swallowed everything after it until
a read timed out, after 1 s of silence:

- A reply to a command sent after WriteCRCMode(TWO_BYTE) was lost.
- A stream started straight after never reached SHIMMER_STATE_STREAMING.
  The start command's ACK went into the swallow, no packet was delivered,
  and the expansion board read at connect became "Unknown".
- ShimmerConsoleAppExample connects this way. There the swallow ends in the
  1.5 s waitTilTimeOut leaves when the status reply it swallowed never comes.

The other switches worked only by luck. Off to one byte: F4 is no response
code. One byte to two: F4's one-byte drop takes the 65. Two bytes to one:
the drop loop rereads the mode for each byte, and WriteCRCMode has set one
byte before the next reply arrives. To off: a bare FF needs nothing dropped.

WriteCRCMode now sets mWaitingForCRCModeACK and mCRCModeAfterACK before it
writes, and ReadData's connected-state ACK handler switches to that mode and
drops its CRC. The flag is cleared when the wait ends, however it ends, so a
later reply's ACK is never taken for it. If no ACK arrived within the wait,
the mode is applied at its end, as before. A switch made while streaming is
unchanged.

ShimmerCrcSwitchTest runs ReadData against CrcLinkDevice, a CrcFirmwareDevice
(ShimmerCrcFirmwareTest) that answers as the firmware does: it applies
SET_CRC before it ACKs, frames each reply in its own mode with
ShimmerUartCrcCalc, streams after START_STREAMING, and times reads out after
ReadTimeout. Every switch is followed by two sampling-rate reads. Without
the fix 3 of its 11 tests fail: off to two bytes, every switch in turn (at
its off-to-two step), and the stream. Removing the drop of the ACK's CRC
fails the stream test alone, and leaving the flag set fails the
never-arriving ACK test alone. All 46 ShimmerBluetoothTests pass (Visual
Studio 2022 MSBuild, vstest.console).

Derived from the firmware source and a test double. Not run on hardware.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

@JongChern JongChern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against shimmer3r-firmware LogAndStream_Shimmer3R_v1.00.010: the SET_CRC case calls setBtCrcMode(args[0]) before the reply is built, and BtUart_sendRsp frames every reply in the current mode (shimmer_bt_comms.c:2344), so the ACK comes in the new mode. A bare ACK's CRC is F4 65 with this API's ShimmerUartCrcCalc (odd-length zero pad), and 0x65 is DETECT_EXPANSION_BOARD_RESPONSE, which explains the 255-byte swallow after off -> two-byte. The fix switches at the ACK, clears the flag in a finally, and leaves mid-stream switches alone. CI re-run against today's master: all green (unit tests 58/58, PublishNuGet pack).

@JongChern
JongChern merged commit 8ae0c2c into master Oct 20, 2026
6 checks passed
JongChern added a commit that referenced this pull request Oct 20, 2026
Resolves the conflict with #212 (DEV-976, switch the link CRC at the
SET_CRC ACK) as this PR's description sets out:
- WriteCRCMode: the 2-byte refusal runs before mCRCModeAfterACK and the
  ACK wait flag are set, so a refused call never leaves the flag set.
- Its docblock keeps both PRs' additions.
- ShimmerUnitTests.csproj compiles both ShimmerCrcStatusPushTest.cs and
  ShimmerCrcSwitchTest.cs.

Built ShimmerAPITest.sln with VS 18 MSBuild and ran ShimmerBluetoothTests
under vstest.console, as CI does: 65 of 65 pass (master's 58 plus this
PR's 7). ShimmerConsoleAppExample, which CI does not build, also builds.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants