Skip to content

DEV-1451 Shimmer3: give the alternative accel and mag their packet bytes - #347

Open
marknolan wants to merge 2 commits into
masterfrom
DEV-1451_shimmer3_alt_accel_mag
Open

marknolan wants to merge 2 commits into
masterfrom
DEV-1451_shimmer3_alt_accel_mag

Conversation

@marknolan

@marknolan marknolan commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

DEV-1451. A Shimmer3 stream with the alternative accelerometer (channel IDs 0x14-0x16) or the alternative magnetometer (0x17-0x19) on was mis-sized: those IDs took no bytes on a Shimmer3.

The defect

  • Packet size. interpretDataPacketFormat gave 0x14-0x19 a name, a type and 2 bytes only on a Shimmer3R. On a Shimmer3 the expected packet was 6 bytes short per sensor, so the stream was mis-framed from the first packet.
  • Decoding. Even with the size right, buildMsg would fail next:
    • its alt-accel and alt-mag blocks run on a Shimmer3 too, because updateEnabledSensorsFromChannels sets SENSOR_ALT_ACCEL/SENSOR_ALT_MAG there;
    • they look up the Shimmer3R names, which are not in a Shimmer3's signal list (index -1);
    • they calibrate with getCurrentCalibDetailsAccelAlt()/getCurrentCalibDetailsMagAlt(), which are null on a Shimmer3.
  • SD files. A Shimmer3 SD file with these sensors goes through the same buildMsg blocks, so it is affected the same way.

What the firmware sends

shimmer3-firmware LogAndStream_Shimmer3/i2c.c, I2C_configureChannels, and log-and-stream-common docs/SHIMMER3_STREAMING_DATA_FORMAT.md §4.1 rows 14-15 and §5.2:

IDs Sensor Bytes Encoding
0x14-0x16 MPU-9150/9250 or ICM-20948 accelerometer, read from ACCEL_XOUT_H 2 each int16, big-endian
0x17-0x19 its AK8975/AK8963/AK09916 magnetometer, read from HXL 2 each int16, little-endian

The firmware's HIL rig streamed a Shimmer3 with both on (ICM-20948 board, v1.01.009): 1016 packets, 21 channels each.

The fix

  • interpretDataPacketFormat: on a Shimmer3, Accel_MPU_X/Y/Z typed i16r, and Mag_MPU_X/Y/Z typed i16. These are the names and types a Shimmer3 SD file already gives them.
  • buildMsg:
    • the ADXL371 and LIS3MDL blocks run on a Shimmer3R only;
    • a Shimmer3's two sensors are decoded uncalibrated, wherever the channel list puts them, from Bluetooth or an SD file.
    • The driver holds no Shimmer3 calibration for these sensors, and the firmware answers zeros to the calibration query.
  • Shimmer3R: unchanged.

Tests

API_00042_Shimmer3AltAccelMagChannelsTest, 9 tests, no hardware. The packets are laid out as the firmware packs them on an ICM-20948 board. There the alternative accel is the same chip reading as the wide-range accel, in the other byte order with Y negated, and the alternative mag the same reading as the mag. So each pair checks the other's byte order. The tests cover:

  • the packet size from an inquiry: 41 bytes, not 29;
  • each sensor alone;
  • the names and types;
  • a channel after them (the BMP280 pressure) still in place;
  • uncalibrated values, and no calibrated ones;
  • agreement with the wide-range copy;
  • a stream of 4 packets framing at the packet size;
  • the Shimmer3R unchanged.

Results:

  • On the unpatched driver: 8 of the 9 fail (packet 29 bytes, not 41; ArrayIndexOutOfBoundsException: Index -1 in the high-g block). The Shimmer3R case passes on both.
  • ShimmerDriver: 187/187 (178 + 9).
  • ShimmerDriverPC: 103/103.

Where else this applies

  • Android: ShimmerAndroidAPI sizes packets with this method, so it gets the fix only after a driver republish and a pin bump.
  • Our internal host library had the same rule. It is fixed alongside, and its reference data is regenerated from this branch.
  • C# API and pyshimmer: the same defect, fixed separately.
  • Swift and the JavaScript SDK: already handle these channels on a Shimmer3.

Bench check, 16 Nov 2026

Two Shimmer3s on LogAndStream v1.01.009, both boards where the ICM-20948 stands in for the LSM303 (SR49-3-1 and SR31-9-1, RN4678 radios). On these boards the firmware fills the wide-range accel and the mag from the same ICM-20948 reading the alternative accel and mag carry, in the other byte order, so every sample can be checked exactly.

  • Capture: LN accel, battery, gyro, wide-range accel, mag, alternative accel, alternative mag and pressure at 51.2 Hz. 21 channels, 47-byte packets, 10 s over Bluetooth. 503 packets from each unit, no gaps, stop ACKed.
  • The firmware's layout, checked on the raw bytes: wide-range X = alt accel X, wide-range Y = -alt accel Y, wide-range Z = alt accel Z, mag X/Y = alt mag X/Y, mag Z = -alt mag Z: 503/503 on each unit, for each of the six. The alt accel reads about 1 g at +/-2 g (16,279 and 16,376 counts on average).
  • This branch: the capture framed at the driver's own packet size (46 bytes after the header, from interpretInqResponse) and each packet through buildMsg, as Accel_MPU and Mag_MPU, with no framing errors. Every raw value matched that layout, on every channel it decodes, in all 503 packets from each unit.

Not covered

  • No calibration. A Shimmer3's alternative accel and mag stay uncalibrated, as before.

🤖 Generated with Claude Code

On a Shimmer3, channel IDs 0x14-0x16 (the MPU9x50/ICM-20948 accel,
big-endian) and 0x17-0x19 (its AK8975/AK8963/AK09916, little-endian) take
2 bytes each, as shimmer3-firmware's I2C_configureChannels packs them.
interpretDataPacketFormat gave them bytes only on a Shimmer3R, so a
Shimmer3 stream with either sensor on was 6 bytes short per sensor and
mis-framed from the first packet; and buildMsg's alt blocks, which
updateEnabledSensorsFromChannels turns on for a Shimmer3 too, looked up
the Shimmer3R names (index -1) and calibrated with null details.

- interpretDataPacketFormat: on a Shimmer3, Accel_MPU_X/Y/Z as i16r and
  Mag_MPU_X/Y/Z as i16, the names and types a Shimmer3 SD file already
  gives them.
- buildMsg: the ADXL371 and LIS3MDL blocks run on a Shimmer3R only; a
  Shimmer3's two sensors are decoded uncalibrated wherever the channel
  list puts them, from Bluetooth or an SD file. The driver holds no
  Shimmer3 calibration for them.

API_00042: 9 tests on packets laid out as the firmware packs them on an
ICM-20948 board, where each alt sensor repeats a wide-range channel's
reading in the other byte order. On the unpatched driver 8 fail (packet
29 bytes, not 41; index -1 in the high-g block); the Shimmer3R case
passes on both.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@marknolan marknolan self-assigned this Nov 12, 2026
@marknolan
marknolan marked this pull request as ready for review November 16, 2026 11:53
@marknolan
marknolan requested a review from JongChern November 16, 2026 11:54
Master gained API_00042_EventRecordFileTest (DEV-971) after this branch was
cut, and its tests now run to API_00048. No test code changed.

Co-Authored-By: Claude Fable 5.2 <[email protected]>

@JongChern JongChern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. I reviewed the change against master and pushed one small commit to this branch, 7077d6d3.

What I changed: the test is renamed from API_00042_Shimmer3AltAccelMagChannelsTest to API_00049_Shimmer3AltAccelMagChannelsTest. No test code changed.

Why: master gained API_00042_EventRecordFileTest after this branch was cut, and its tests now run to API_00048. Two tests with one number would make the numbers useless for finding a test.

What I checked:

  • Merged with today's master: ShimmerDriver 228 of 228.
  • The new test against master's driver without the fix: 8 of 9 fail, as the description says.
  • On this branch after the rename: ShimmerDriver 187 of 187.
  • The orientation step does not count these sensors on a Shimmer3, so the uncalibrated values cannot feed it.

Still open, not blocking: ShimmerAndroidAPI gets this only after a driver release and a pin bump. Nothing here ran on a unit beyond the bench check in the description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants