Skip to content

DEV-1452: decode a Shimmer3's alternative accel and mag (0x14-0x19) - #233

Merged
JongChern merged 1 commit into
masterfrom
DEV-1452_shimmer3_alt_accel_mag
Nov 18, 2026
Merged

JongChern merged 1 commit into
masterfrom
DEV-1452_shimmer3_alt_accel_mag

Conversation

@marknolan

@marknolan marknolan commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

DEV-1452. A Shimmer3 streaming with the alternative accelerometer or the alternative magnetometer on was mis-framed from the first packet, and a Shimmer3 SD file with either on was mis-read.

The defect

InterpretDataPacketFormat gave channels 0x14-0x19 a name, a data type, 2 packet bytes and an enabled-sensor bit only under ShimmerVersion.SHIMMER3R. On a SHIMMER3 the arms matched but did nothing, so each channel took 0 bytes: the packet size was 6 bytes short per sensor, every channel after them was read from the wrong offset, and the read loop lost framing.

ShimmerSDLog has the same gap for Shimmer3 files. They carry no channel list, so the reader builds the record layout from the header's enabled-sensor bits, and it had no entries for bit 22 (alt accel) or bit 21 (alt mag). A file with either on was read 6 bytes short per record per sensor. Worse, those header bits are the same as SENSOR_ACCEL_ALT / SENSOR_MAG_ALT, so BuildMsg went into the Shimmer3R blocks, could not find the High G or Alt Magnetometer channels, and getSignalIndex fell back to index 0: it reported "High G Accelerometer X/Y/Z" and "Alt Magnetometer X/Y/Z" as the timestamp, calibrated with the Shimmer3R ADXL371 and LIS3MDL defaults. Checked on the old code with a three-record file: packet size 14 instead of 26, and every one of those six channels read RAW 4096, the first timestamp.

What a Shimmer3 sends

From shimmer3-firmware LogAndStream_Shimmer3/i2c.c (I2C_configureChannels, and the copy into the data buffer in the same file) and log-and-stream-common docs/SHIMMER3_STREAMING_DATA_FORMAT.md (the channel table, section 4.1 rows 14-15, section 5.2):

Channels Sensor Encoding
0x14-0x16 X/Y/Z_ALT_ACCEL MPU-9150/9250 or ICM-20948 accelerometer, read from ACCEL_XOUT_H int16 big-endian, 2 bytes each
0x17-0x19 X/Y/Z_ALT_MAG AK8975/AK8963 or AK09916 magnetometer, read from HXL int16 little-endian, 2 bytes each

Packet order: LN accel 0x00-0x02, VBATT 0x03, ADCs, gyro 0x0A-0x0C, WR accel 0x04-0x06, mag 0x07-0x09, alt accel 0x14-0x16, alt mag 0x17-0x19, BMP 0x1A/0x1B, ExG. The SD record is the same bytes as the streamed packet, without the packet type. Enabled-sensor bits are the Shimmer3R's: SENSOR_ACCEL_ALT 0x400000, SENSOR_MAG_ALT 0x200000.

The fix

  • InterpretDataPacketFormat: each of the six arms gets a SHIMMER3 branch. 2 bytes, i16r for 0x14-0x16 and i16 for 0x17-0x19, and the enabled-sensor bit. The accelerometer gets new names, Shimmer3Configuration.SignalNames.ALTERNATIVE_ACCELEROMETER_X/Y/Z ("Alternative Accelerometer X/Y/Z"), because the existing High G names are the Shimmer3R's ADXL371. The magnetometer reuses ALT_MAGNETOMETER_X/Y/Z.
  • BuildMsg: the existing alt accel and alt mag blocks, which calibrate with Shimmer3R matrices, now run only on a Shimmer3R. A Shimmer3 gets RAW only, no CAL, since the API holds no Shimmer3 calibration for these sensors. On a Shimmer3 they don't write accelerometer[] / magnetometer[] and don't count toward the 3D-orientation condition, since they are uncalibrated there.
  • ShimmerSDLog: a Shimmer3 file's layout gets the alt accel (bit 22, i16r) and alt mag (bit 21, i16) after the magnetometer and before the pressure sensor, the firmware's order.
  • Shimmer3R: unchanged. 0x14-0x16 are still i12*> High G, 0x17-0x19 i16 Alt Magnetometer, both calibrated, and both still count toward the orientation.

Tests

ShimmerUnitTests/ShimmerAltAccelMagDecodeTest.cs, 12 tests through InterpretDataPacketFormat + BuildMsg, or ShimmerSDLog for the file. The main fixture is the one the Java driver's fix (DEV-1451) uses: an ICM-20948 board with no LSM303 fitted, so the alt accel carries the same chip reading as the WR accel (which the firmware byte-swaps to little-endian and negates in Y) and the alt mag the same reading as the mag (negated in Z). 19 channels, 41 bytes after the packet type.

  • Packet size 41 with every IMU channel on. Alt accel RAW 1234 / -1234 / 16384, alt mag RAW 291 / -240 / 512, no CAL, no High G channels. Every channel before them still reads its own bytes. WR X equals alt X, WR Y equals minus alt Y, and the mags agree the same way.
  • Each sensor alone adds 6 bytes, and both together add 12.
  • The pressure sensor after them (0x1A u16r, 0x1B u24r) is read at the right offset.
  • Enabled-sensor bits and data types.
  • On a Shimmer3 the alt sensors don't feed the orientation. On a Shimmer3R they still do.
  • Shimmer3R decodes as before: i12*> / i16, High G and Alt Magnetometer names, CAL present.
  • A Shimmer3 SD file with LN accel, alt accel, alt mag and BMP.

Run against the old ShimmerBluetooth.cs and ShimmerSDLog.cs, with only the three new name constants added so that the tests compile, 10 of the 12 fail. Packet sizes come out 29 / 9 / 9 / 9 / 8 where 41 / 15 / 15 / 21 / 20 are expected, the enabled bits come out 0, and the alt channels are missing. The SD test fails too. The two Shimmer3R tests pass on both. Removing only the orientation gate makes the Shimmer3 orientation test fail.

Whole VSTest project (ShimmerBluetoothTests.dll): 124/124 on master before, 136/136 after.

The same logic elsewhere in this repo

  • ShimmerAPI/ShimmerAPI/ShimmerSDLog.cs: same defect, fixed and tested here (above).
  • Shimmer32FeetAPI: no parser of its own. Shimmer32Feet, ShimmerLogAndStream32Feet and ShimmerLogAndStream32FeetBLE extend ShimmerBluetooth / ShimmerLogAndStream, so they get this fix.
  • ShimmerCaptureXamarin/ShimmerCaptureXamarin/ShimmerBluetooth.cs: not the same defect. It has no arms for 0x14-0x19 at all (it goes from 0x13 at line 2401 to 0x1A at line 2411), so they fall to the unknown-ID else at line 2559, which gives each one 2 unnamed bytes. Framing is right and the values are dropped. Not changed: that copy is abandoned and has no Shimmer3R support.
  • ShimmerAPI/TestSerialPort/Program.cs: no channel-list parser.

Bench check, 16 Nov 2026

Two Shimmer3s on LogAndStream v1.01.009, both boards where the ICM-20948 stands in for the LSM303 (SR49-3-1 and SR31-9-1, RN4678 radios). On these boards the firmware fills the wide-range accel and the mag from the same ICM-20948 reading the alternative accel and mag carry, in the other byte order, so every sample can be checked exactly.

  • Capture: LN accel, battery, gyro, wide-range accel, mag, alternative accel, alternative mag and pressure at 51.2 Hz. 21 channels, 47-byte packets, 10 s over Bluetooth. 503 packets from each unit, no gaps, stop ACKed.
  • The firmware's layout, checked on the raw bytes: wide-range X = alt accel X, wide-range Y = -alt accel Y, wide-range Z = alt accel Z, mag X/Y = alt mag X/Y, mag Z = -alt mag Z: 503/503 on each unit, for each of the six. The alt accel reads about 1 g at +/-2 g (16,279 and 16,376 counts on average).
  • This branch: the capture through the tests' AltSensorDecoder (InterpretDataPacketFormat, then BuildMsg per packet), framed at its PacketSize (46 after the header), with no framing errors. That covers the pressure and temperature after the alternative mag too. Every raw value matched that layout, on every channel it decodes, in all 503 packets from each unit.

🤖 Generated with Claude Code

InterpretDataPacketFormat gave channels 0x14-0x19 a name, data type,
size and enabled-sensor bit only on a Shimmer3R. On a Shimmer3 they took
no bytes, so a stream with the alternative accelerometer or magnetometer
on was framed 6 bytes short per sensor from the first packet.

On a Shimmer3, 0x14-0x16 are the MPU-9150/9250 or ICM-20948
accelerometer (int16 big-endian) and 0x17-0x19 its AK8975/AK8963/AK09916
magnetometer (int16 little-endian), packed after the magnetometer and
before the pressure sensor. They now decode as "Alternative
Accelerometer X/Y/Z" and "Alt Magnetometer X/Y/Z", RAW only, as the API
holds no Shimmer3 calibration for them. On a Shimmer3 they no longer
count toward the 3D orientation, since they are uncalibrated there. The
Shimmer3R's high-g accel and alt mag decode as before.

ShimmerSDLog had the same gap for Shimmer3 files, which carry no channel
list: the record layout it built from the header's sensor bits left both
sensors out, so a file with either on was mis-framed, and BuildMsg
reported "High G Accelerometer" and "Alt Magnetometer" values read from
the timestamp. It now adds them in the firmware's order.

12 new tests, 10 of which fail on the old code. VSTest 136/136.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@marknolan marknolan self-assigned this Nov 12, 2026
@marknolan
marknolan marked this pull request as ready for review November 16, 2026 11:53
@marknolan
marknolan requested a review from JongChern November 16, 2026 11:54

@JongChern JongChern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. I made no changes.

What I checked:

  • The rule matches the Java driver's fix (ShimmerResearch/Shimmer-Java-Android-API#347): on a Shimmer3, 0x14-0x16 are int16 big-endian and 0x17-0x19 are int16 little-endian, two bytes each, raw only.
  • The SD file change sits inside the Shimmer3-only layout, so a Shimmer3R file is read as before.
  • The orientation gate: a Shimmer3's uncalibrated alternative sensors no longer count toward it.
  • CI runs the unit test project and passes. I did not build the solution locally.

One thing to know, not blocking: the channel names differ from the Java driver's. This API uses "Alternative Accelerometer X" and reuses the Shimmer3R's "Alt Magnetometer X". The Java driver uses Accel_MPU_X and Mag_MPU_X. The two APIs already name channels differently, so I am leaving it.

@JongChern
JongChern merged commit 0f52236 into master Nov 18, 2026
5 checks passed
@JongChern
JongChern deleted the DEV-1452_shimmer3_alt_accel_mag branch November 18, 2026 05:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants