Repository navigation
DEV-971: keep configuration byte 232 on a full write, expose its fields, probe for the event channel - #343
Conversation
Byte 232 holds the button mode, the log triggers and the TTL input enable on LogAndStream firmware with the event channel. The full configuration write built its image from zeros, so every configure from Consensys or the Android config dialog put 0 there and turned the button back to legacy. - ButtonLogMode: the byte's fields and the firmware's correction rules (log-and-stream-common shimmer_button_log_mode.c, via the web SDK's buttonLogMode.ts), including press recording and the TTL input taken out on SR48-6-0. - ShimmerObject: the parse keeps the byte as read, 0xFF included, and the generator writes it back (corrected where it can be trusted). Before packing bytes 217 and 218 it holds userButtonEnable at the button mode's nearest equivalent and clears singleTouchStart where no press can start logging, as the firmware does. Getters and setters for the three fields, and config labels for them. - The byte is trusted only when the Shimmer ACKed the probe, or the value read is one the firmware keeps: firmware without it leaves 0xFF, and the dock cannot probe. - ShimmerVerObject.isSupportedSetFeature(): SET_FEATURE arrived with log-and-stream-common, in LogAndStream_Shimmer3R v1.0.52 and LogAndStream_Shimmer3 v1.0.20. Older firmware discards 0xB7 and runs the two bytes after it as commands (03 is GET_SAMPLING_RATE, which is ACKed), so it is never sent there. The layout places byte 232 from those releases on; the event channel's builds report the release before them, so the version cannot say more. - ShimmerBluetooth.readEventChannelSupport(): SET_FEATURE with FEATURE_EVENT_STREAM and 0, sent on connect. ACK means the event channel is there; the NACK is handled without dropping the link, and still counts towards the operation's progress. - writeConfigBytes(): when the byte was never read nor set, as when a configuration is written on connect before the read, the chunk at 128 is sent either side of byte 232, so the Shimmer keeps its own. Not benched. Co-Authored-By: Claude Opus 5.5 <[email protected]>
LogAndStream_Shimmer3 v1.00.017 is the first release whose log-and-stream-common pin has SET_FEATURE (df58877, with the two-byte argument fix e5ea88c). From v1.00.017 to v1.00.019 the module sat at LogAndStream/log-and-stream-common, and v1.00.017 builds its Bluetooth handler from there. The earlier check looked only at the path v1.00.020 moved it to. The gate moves to v1.0.17, the same release isSupportedRn4678ErrorDetection() already uses. Those releases NACK FEATURE_EVENT_STREAM, so they read as having no event channel, and their SET_INFOMEM handler is the same one that takes the split write around byte 232. Co-Authored-By: Claude Opus 5.5 <[email protected]>
applyButtonLogModeRules() kept singleTouchStart wherever byte 232 left a press that can start logging. That rule exists only in a firmware build with IS_SUPPORTED_SINGLE_TOUCH, and no LogAndStream release has it: every one clears the bit on each configuration it takes, whatever byte 232 holds. So a full write sent a bit the Shimmer drops, and the driver showed single touch on where it never acts. - The rules now clear singleTouchStart on firmware that has byte 232 (LogAndStream from the release that brought SET_FEATURE, the same gate the layout uses), and keep userButtonEnable at the button mode's nearest equivalent there as before. - On other firmware they change nothing. A byte 232 setter called for an SDLog Shimmer, or LogAndStream before the byte, used to move userButtonEnable and clear single touch on firmware that has neither rule, and SDLog is the one firmware the driver offers Single Touch Start for. - API_00037: single touch is cleared in every mode, 0xFF included, and firmware without the byte keeps bytes 217 and 218. All three tests fail on the old rule. Not benched. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Brings in DEV-1408 (the callback consumer thread survives a throwing handler). No conflicts. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…hannel readEventChannelSupport()'s javadoc: no Shimmer3 release has it yet, and the builds before a release report the release before them, so the driver still asks rather than going by the version. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Resolves the conflict #343 predicted with #293 (DEV-982) in processNotStreamingWaitForAck(): the event channel probe's NACK is taken first, then any other NACK as a refusal. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…nnel status The Java driver now does the host side of the event channel that the web SDK does (log-and-stream-common docs/SHIMMER3_EVENT_CHANNEL.md and SHIMMER3_BT_COMMUNICATION_PROTOCOL.md section 7.15): - EventFile, EventRecord: the SD card's events/ files, a 64-byte header and 16-byte records, with pairing to the data file by its initial timestamp. - writeEventStream(boolean): SET_FEATURE / FEATURE_EVENT_STREAM. Each record frame, [0x8A][0xBB][0x10][record][CRC-16], reaches EventChannelListener. Frames are taken between data packets (any link CRC mode), straight after the opt-in's ACK, at the start of the buffer, and when idle, including before a command's ACK or reply. They are only parsed after an opt-in, and only once their own CRC checks out. - readEventChannelStatus(): GET_EVENT_CHANNEL_STATUS_COMMAND (0xB9). Not streaming only: its reply is not wrapped in 0x8A, so the streaming parser could not tell it from a packet. Tests port the web SDK's tests/events/record.test.ts inputs and results, and run the IO thread over an in-memory radio for the streaming and idle paths. Android's Shimmer.java overrides the streaming parser and does not know the frame, so it must not opt in until ported. Co-Authored-By: Claude Opus 5.5 <[email protected]>
… event channel Tools > Event markers (Shimmer3R), or the EVENT MARKERS button, opens a window that does with the Java driver what Shimmer Capture web does with the web SDK: - whether the unit has the event channel, and asking again; - configuration byte 232 (button mode, what starts logging, TTL input), written with the whole configuration and read back; - opt in / opt out, the live records with sequence gaps flagged, and the frame and bad-CRC counters; - the event channel status (not while streaming); - an event file from a card: its header, completeness, the data file it pairs with, and its records timed from the file open. Each live edge is also marked on the main plot as a red vertical bar, placed by the latest data packet's Shimmer clock against the plot's PC-time axis. CI does not build ShimmerPCBasicExamples; built locally. Co-Authored-By: Claude Opus 5.5 <[email protected]>
DEV-971: Java event markers - event files, live records, status, and a test window
Each was first reproduced by a test that failed on the unchanged code (API_00044_EventStreamReviewReproTest), then fixed: 1. Idle: event records right behind an in-stream reply were lost. After an unsolicited status push ([ACK][0x8A][0x71]), sent on a dock, an undock or a press that starts logging, clearSerialBuffer() wiped the records queued behind it (0 of 2 arrived). With the event channel on, the bytes after the reply are now left for the next pass. A record frame after a stray ACK no longer has link-CRC bytes cleared after it either: it carries its own CRC. 2. Streaming: an ACK nobody waited for, before a record frame, re-ran the ACK handling of the SET that went last. It is now taken only while an ACK is awaited. 3. Streaming with no link CRC: the packet before a 0x8A was taken as a packet even when the 0x8A was a data byte after a lost byte, so misframed bytes reached the app as a sample. It is now taken only once the frame after it passes its own CRC; otherwise the bytes are discarded as before. 4. readEventChannelStatus() could be queued behind a start of streaming not yet acknowledged, and its unwrapped reply, [ACK][0xBA][9], was then not taken between data packets and the command timed out. It is now refused while a start of streaming is queued, and a reply that still arrives between packets is taken. ShimmerDriver 219/219 and ShimmerDriverPC 154/154 pass; the Java 8 MATLAB jar builds. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ehind a bare ACK's CRC From the review of #349: - The idle tests' radios now take each burst whole and let reads wait for their bytes, as a serial port does (WaitingTestRadio), so the IOThread can no longer race the test's writes. API_00043's idle tests use it too. - The status-push, frame-after-ACK and status-reply tests now run with no link CRC, a 1-byte and a 2-byte one. - The misframing test checks that the good packets after the damaged one still arrive; the status-reply test that the command is retired, the lock released, the timer cancelled and the packets either side parsed. The CRC runs reproduced one more defect, which is fixed here: with a link CRC on (the driver turns it on at connect where the firmware has it), the records sent in the same burst as the opt-in's ACK were lost, 0 of 2. The bare ACK's link CRC is left after the ACK is taken; the idle path read it as an unknown byte and cleared the whole serial buffer, records included. With the event channel on, the idle path now drops only the bytes it read. All 7 tests in API_00044 fail on #348's code and pass here. ShimmerDriver 219/219, ShimmerDriverPC 155/155; the Java 8 MATLAB jar builds. Co-Authored-By: Claude Opus 5.5 <[email protected]>
DEV-971: fix five event-channel parser defects found in #348's review
|
Reviewed at 24d985e by reading; I did not run the suites, and CI was still running. I read the probe and its NACK handling, the firmware gate, the configuration write around byte 232, and (in #349's review) the streaming and idle parser. I did not go line by line through Checks out
Needs attention
Smaller
|
… tests From the review of #343: - A Bluetooth configuration write before any read sent a Shimmer3R's byte 233 (DEV-99 audio mode and gain, DEV-1413 sound level) as 0, which turned the unit's audio mode off. It is now left out as byte 232 is, unless it was read or the sound level is turned on, on the same firmware. writeMem() takes a list of bytes to leave out. 3 of the 5 tests in API_00047_AudioConfigWriteTest fail without it. - The event channel probe's NACK now goes through processNackFromCommand(), which marks the event channel unsupported and does not report the NACK as a refused command. processEventChannelProbeNack() is gone. - API_00036_ButtonLogModeTest and API_00037_ButtonLogModeConfigWriteTest are now API_00045 and API_00046: master's log-start-refused tests have those numbers. Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
Proposed description for this PR, once #351 is merged. The current description is out of date, as the review comment says. This version adds:
Mark, it's your PR, so take this in if it reads right. The source is below, ready to paste. Markdown sourceDEV-971 adds an event channel to Shimmer3 and Shimmer3R LogAndStream. It also adds configuration byte 232, which holds the button mode, the log triggers and the TTL input enable. The byte is specified in log-and-stream-common `docs/SHIMMER3_CONFIGURATION_INFOMEM.md` §8.3.
**The driver's full configuration write builds its image from zeros, so any configure from Consensys or the Android config dialog put 0 in byte 232.** That silently turned the button back to legacy, which undid a setting such as "button disabled" (`0x01`) or "button disabled, logging off" (`0x11`). This PR closes that gap and adds the byte to the config model. The driver can also tell whether the firmware has the byte.
It also carries the driver's host side of the event channel, merged in from #348 and #349: event files, live event records and the status (section 4).
Firmware and web support are in draft on the same branch name: ShimmerResearch/log-and-stream-common#216, ShimmerResearch/shimmer3-firmware#138, ShimmerResearch/shimmer3r-firmware#319, ShimmerResearch/shimmer-web-sdk#218 and ShimmerResearch/shimmer-capture-web#11.
## What changes
### 1. A full configuration write keeps byte 232
- **Parse.** `ShimmerObject.configBytesParse()` keeps the byte as read, `0xFF` included.
- **Generate.** `configBytesGenerate()` writes the byte back unless the caller set one. Where the byte can be trusted (see 3), it writes the value the firmware would correct it to.
- This covers every path that goes through the generator: the Bluetooth `writeConfigBytes()`, the dock write, the reload after flashing, and the clone-settings copy.
- **Before anything is read.** A configuration can be written on connect before the read, for example with `isSetupDeviceWhileConnecting()` or a fixed config. `writeConfigBytes()` then leaves out the bytes the driver does not have from this Shimmer, so the Shimmer keeps its own:
- byte 232, if it was neither read nor set;
- on a Shimmer3R, byte 233 (DEV-99 audio mode and gain, DEV-1413 sound level), if it was not read and the sound level is not turned on. The host never sets bits 0-3, so writing them unread put 0 there and turned the audio mode off. With the sound level turned on before a read, the byte is still written with bits 0-3 at 0, since one bit cannot be written alone.
- The chunk at 128 goes as the runs between the bytes left out, for example `[128..231] [234..255]`. No other chunk moves, which matters because the firmware treats the chunks at 0 and 128 specially (calibration dump, MAC address).
- This only happens on firmware that can have byte 232 (see 3). Older firmware is written exactly as before, byte 233 included.
### 2. The three fields in the config model
- **`ButtonLogMode`** ports the byte's fields and the firmware's correction rules (`shimmer_button_log_mode.c`, through the web SDK's `buttonLogMode.ts`), rule for rule. That includes the SR48-6-0 rule: press recording and the TTL input are taken out on that board.
- **Accessors on `ShimmerObject`:**
- `getButtonMode()` / `setButtonMode()`
- `getLogTriggers()` / `setLogTriggers()`
- `isTtlInputEnabled()` / `setTtlInputEnabled()`
- `setButtonLogMode(mode, triggers, ttl)`
- `getButtonLogMode()` (the raw byte) and `getButtonLogModeCorrected()`
- **Fields can be set in any order.** Log triggers set while the mode is still legacy are kept until the mode is set.
- **The firmware's side effects are applied to the model, in the setters and again before a write, on firmware that has the byte (see 3):**
- `userButtonEnable` (byte 217 bit 5) is held at the button mode's nearest equivalent.
- `singleTouchStart` (byte 218 bit 7) is cleared. No LogAndStream release is built with single touch (`IS_SUPPORTED_SINGLE_TOUCH` is 0 in log-and-stream-common), so each clears the bit on every configuration it takes, whatever byte 232 holds.
Because this happens in the model, `writeTrial()` (SET_TRIAL_CONFIG) sends the same bits. Other firmware is left alone, so a setter called for an SDLog Shimmer, the one firmware the driver offers Single Touch Start for, changes neither bit.
- **Config labels.** `Configuration.Shimmer3.GuiLabelConfig.BUTTON_MODE`, `LOG_TRIGGERS` and `TTL_INPUT` go through `set/getConfigValueUsingConfigLabel()`, so the fields can also be set from a `USER` fixed config map. They are not added to the Consensys option maps, so Consensys does not show them yet.
### 3. Detection
LogAndStream_Shimmer3R **v1.01.026** will be the first Shimmer3R release with the event channel, and no Shimmer3 release has it yet. The builds before a release report the version of the release before them, so the version cannot say whether a build has it. The driver asks instead:
- **The probe.** `ShimmerBluetooth.readEventChannelSupport()` sends `SET_FEATURE` (0xB7) with `FEATURE_EVENT_STREAM` (3) and 0. That changes nothing, since a connection starts opted out. It is sent on connect, and `isEventChannelSupported()` returns the answer.
- **ACK.** The firmware has the event channel.
- **NACK.** It does not. `processNackFromCommand()`, the NACK handling from #293, unwinds it like any refused command: the link stays up, the next command goes out, and it counts in the operation's progress. Because the NACK is an answer, it sets `isEventChannelSupported()` to false and is not passed to `eventCommandRefused()`.
- **Which firmware is asked.** `SET_FEATURE` arrived with log-and-stream-common (df58877 and e5ea88c, 2026-01-29), first released in LogAndStream_Shimmer3R **v1.00.052** and LogAndStream_Shimmer3 **v1.00.017**. That is `ShimmerVerObject.isSupportedSetFeature()`, and v1.00.017 is also the release `isSupportedRn4678ErrorDetection()` already gates on.
- The first version of this PR gated Shimmer3 at v1.00.020, because the check had looked only at the path where v1.00.020 moved the module. In v1.00.017-019 it sat at `LogAndStream/log-and-stream-common`. Fixed in 955084ca.
- v1.00.017-019 have `SET_FEATURE` but not the event channel, so they NACK the probe.
- Older firmware drops the 0xB7 byte unanswered and runs the two bytes after it as commands. `03` is `GET_SAMPLING_RATE_COMMAND`, which the firmware ACKs, so a probe there would read as "supported". Those versions are never asked, and are treated as having no event channel.
- The layout places byte 232 (`ConfigByteLayoutShimmer3.idxButtonLogMode`) only from those releases on.
- **When the byte is trusted** (`isButtonLogModeMeaningful()`): only when the probe was ACKed, or when the value read back is one the firmware keeps.
- Firmware without the byte leaves `0xFF`, which reads as legacy and is written back as read.
- The dock cannot probe, so there only the value read counts.
### 4. The event channel's host side (#348, #349)
Following log-and-stream-common `docs/SHIMMER3_EVENT_CHANNEL.md` and `docs/SHIMMER3_BT_COMMUNICATION_PROTOCOL.md` §7.15, as shimmer-web-sdk#218 does:
- **Event files.** `EventFile` and `EventRecord` read `events/<experiment>/<session>/<NNN>` ("SEVT", a 64-byte header, then 16-byte records). `belongsWithDataFile()` and `getCompanionFile()` pair a file with its data file.
- **Live records.** `writeEventStream(boolean)` opts the connection in or out. Each record frame, `[0x8A][0xBB][0x10][record][CRC-16]`, goes to every `EventChannelListener`, idle or streaming, with any link CRC mode, and only once its own CRC checks out.
- **Status.** `readEventChannelStatus()` sends `GET_EVENT_CHANNEL_STATUS_COMMAND` (0xB9). It is refused while streaming, or while a start of streaming is queued, since its reply has no 0x8A wrapper.
- **Test window.** `EventMarkersFrame` in ShimmerPCBasicExamples (SensorMapsExample, Tools › Event markers).
- **Android must not opt in yet.** `ShimmerAndroidAPI`'s `Shimmer.java` overrides the streaming parser and does not know the record frame. Reading event files and byte 232 reach Android through the driver pin as usual.
## The same full-write problem in the sibling APIs
| API | Full configuration write | Needs the fix? |
|---|---|---|
| C# (`Shimmer-C-API`) | `WriteShimmer3Infomem()` is `[Obsolete]` (unfinished) and writes the caller's 384 bytes unchanged. Its only callers in ShimmerCapture are commented out. SET_TRIAL_CONFIG sends back the bytes it last read. | **No** |
| Python (`pyshimmer`) | Writes no configuration at all: the dock API only reads InfoMem. | **No** |
| Swift (`SwiftAPI`) | `writeShimmer3InfoMem()` writes the caller's image unchanged, so the library keeps whatever it is given. The `ShimmerExamplePlot` app mostly writes the image it read, modified. One hard-coded 384-byte preset (accel) has 0 at byte 232 and would turn a Shimmer back to legacy. | **No** for the library. The example preset is a note only. |
| Web SDK | Has its own DEV-971 change, ShimmerResearch/shimmer-web-sdk#218. | Covered there |
The live event channel is in the web SDK (#218) and shimmer-host-core; C# still needs it.
## Who needs a republish
Nothing reaches a consumer until a driver release is cut after this merges (the last is `API_v0.11.12_beta`):
- **ShimmerAndroidAPI** pins `shimmerdriver`/`shimmerbluetoothmanager` `0.11.12_beta`. The release workflow opens the pin-bump PR. The probe's NACK is handled in the IO thread's not-streaming path, which `Shimmer.java` does not override, so it arrives with the pin. The live frame does not (section 4).
- **Shimmer-MATLAB-ID**: `ShimmerJavaClass.jar`, also re-vendored by a PR the release workflow opens.
- **ASM_PC** builds this repo from a pinned submodule commit. It needs a bump to a master commit after the merge.
- **Consensys** picks it up with its next driver update. Importing event files there is DEV-1457.
## Tests
- `API_00045_ButtonLogModeTest` (ShimmerDriver):
- the §8.3 table row by row, the 16 values with and without the TTL bit: what a press does, what starts logging, the `userButtonEnable` held and single touch;
- that those 32 are exactly the values the firmware keeps;
- all 256 values against an independent restatement of the correction;
- `encode`, setting fields in either order, and SR48-6-0.
- `API_00046_ButtonLogModeConfigWriteTest` (ShimmerDriverPC):
- every kept value survives a full write;
- `userButtonEnable` is held, and single touch is cleared in every mode;
- setters and config labels work;
- `0xFF` is kept as read (single touch still cleared), and a byte the firmware would correct is trusted only after the probe;
- SR48-6-0;
- older firmware is written as before, and keeps bytes 217 and 218 when a byte 232 setter is called (LogAndStream before the byte, and SDLog);
- the Bluetooth write's chunks leave out 232 only while the byte is unknown, and carry the right bytes.
- `API_00047_AudioConfigWriteTest` (ShimmerDriverPC): before a read, a Shimmer3R's byte 233 is left out with 232; once read it goes back as read; the sound level turned on before a read is written; 232 set before a read is written without 233.
- `API_00038_EventChannelProbeTest` (ShimmerDriverPC), driving the real IO thread over an in-memory radio:
- ACK → supported;
- NACK → not supported, the link stays up and the next command goes out, the NACK still counts in the operation's progress, and it is not reported as a refused command;
- firmware older than `SET_FEATURE`, and a streaming Shimmer, are not asked.
- `API_00042_EventRecordFileTest` (ShimmerDriver), `API_00043_EventStreamTest` and `API_00044_EventStreamReviewReproTest` (ShimmerDriverPC): the event channel, from #348 and #349.
- **Mutation check.** At `3606c951`: without the byte-232 write, 9 of the 232 tests fail; without the NACK branch, 2 fail; with the old single-touch rule, 3 fail. At `24d985e5`, before #351: 3 of `API_00047`'s 5 fail without the byte-233 skip.
- **Full suites** pass locally at `d7d7fd80` (#351): ShimmerDriver `test` 219/219, ShimmerDriverPC 161/161 (`clean build`, as CI runs it), the Java 8 jar (`shadowJar -PmatlabJava8`) builds, and ShimmerPCBasicExamples compiles.
## On a unit
Results posted on #348 and #349, from a Shimmer3R on LogAndStream v1.1.26 over Bluetooth Classic, with the 1-byte link CRC the driver sets at connect:
- the probe ACKed on connect;
- byte 232: button mode 4 written, and read back as `0x04`;
- live records idle and while streaming, between data packets: 21 in one run, 13 in each of two more, with no sequence gaps and no bad CRC;
- a double press started logging and another stopped it, and nothing behind the unit's status pushes was lost;
- the status answered idle and after a stream, and refused straight after `startStreaming()`;
- a 20 s stream at 51.2 Hz, opted in: 0 samples missing.
**Not yet on a unit:**
- **The probe on released firmware that NACKs it:** a Shimmer3R on v1.00.052 to v1.01.025, or a Shimmer3 on v1.00.017 or later. Only `API_00038`'s in-memory radio covers it.
- The byte-233 change, TTL edges, and an event file read from a card.
## Review fix (`3606c951`)
The rules kept `singleTouchStart` wherever byte 232 left a press that can start logging, following log-and-stream-common §8.3. That rule is only built with `IS_SUPPORTED_SINGLE_TOUCH`, which no LogAndStream release sets, so a full write sent a bit the Shimmer drops. The rules now clear it on firmware with the byte and change nothing on other firmware (section 2). §8.3 is corrected in ShimmerResearch/log-and-stream-common#216 (`a531896`), and the web SDK made the same fix in ShimmerResearch/shimmer-web-sdk#218.
## Review fixes (#351)
From the review comment of 2026-11-13: byte 233 is left out before a read as byte 232 is, the probe's NACK goes through the one NACK handler, and the button-mode tests are renumbered `API_00045` and `API_00046`, since `API_00036` and `API_00037` were taken on master by the log-start-refused tests.
🤖 Generated with [Claude Code](https://claude.com/claude-code)🤖 Generated with Claude Code |
|
Bench, 13 Nov 2026: the event channel probe on released firmware that NACKs it. A Shimmer3R on LogAndStream v1.1.25, the release without the event channel, over Bluetooth Classic with the 1-byte link CRC the driver sets at connect. Code:
The same check just before, with the unit still on v1.1.26: the probe ACKed, connect took 4.1 s, and a 5 s stream gave 247 packets with the link kept. Not covered:
🤖 Generated with Claude Code |
|
Bench, 13 Nov 2026: a configuration written on connect, before the read. A Shimmer3R on LogAndStream v1.1.25 over Bluetooth Classic, with the 1-byte link CRC. Code: The run:
Run three times. The first attempt never reached the write: COM32 was still busy 3 s after the previous disconnect. The second and third passed; the third added the Not covered:
🤖 Generated with Claude Code |
DEV-971: #343 review fixes - leave byte 233 out before a read, one NACK handler, renumber tests
JongChern
left a comment
There was a problem hiding this comment.
Approving at a7dde8d. With #349 and #351 merged in, the points from my review are fixed, the branch is up to date with master, and the bench runs on v1.1.25 (probe NACKed, a write before the read) and v1.1.26 (the live event channel) cover the risks I raised. I also read ButtonLogMode against the firmware's correction rule and EventFile's bounds checks this time. Reviewed by reading; CI on this head was still running.
Before merge: take up the proposed description (it's in a comment above), and let CI finish. After merge: the Java HIL on master's commit before the next release, and the Android opt-in guard with the driver pin bump. No Shimmer3 has been on the bench for this yet.
Brings in #343 as merged, with #351's event channel fixes and renumbered tests. Co-Authored-By: Claude Fable 5.2 <[email protected]>
Co-Authored-By: Claude Fable 5.2 <[email protected]>
DEV-1459: merge master (#343) into DEV-1459, and write an audio mode set before a read
DEV-971 adds an event channel to Shimmer3 and Shimmer3R LogAndStream. It also adds configuration byte 232, which holds the button mode, the log triggers and the TTL input enable. The byte is specified in log-and-stream-common
docs/SHIMMER3_CONFIGURATION_INFOMEM.md§8.3.The driver's full configuration write builds its image from zeros, so any configure from Consensys or the Android config dialog put 0 in byte 232. That silently turned the button back to legacy, which undid a setting such as "button disabled" (
0x01) or "button disabled, logging off" (0x11). This PR closes that gap and adds the byte to the config model. The driver can also tell whether the firmware has the byte.It also carries the driver's host side of the event channel, merged in from #348 and #349: event files, live event records and the status (section 4).
The firmware and web sides merged on 11 Nov 2026: ShimmerResearch/log-and-stream-common#216, ShimmerResearch/shimmer3-firmware#138, ShimmerResearch/shimmer3r-firmware#319, ShimmerResearch/shimmer-web-sdk#218 and ShimmerResearch/shimmer-capture-web#11. ShimmerResearch/shimmer-host-core#94 has it in the Rust library.
What changes
1. A full configuration write keeps byte 232
ShimmerObject.configBytesParse()keeps the byte as read,0xFFincluded.configBytesGenerate()writes the byte back unless the caller set one. Where the byte can be trusted (see 3), it writes the value the firmware would correct it to.writeConfigBytes(), the dock write, the reload after flashing, and the clone-settings copy.isSetupDeviceWhileConnecting()or a fixed config.writeConfigBytes()then leaves out the bytes the driver does not have from this Shimmer, so the Shimmer keeps its own:[128..231] [234..255]. No other chunk moves, which matters because the firmware treats the chunks at 0 and 128 specially (calibration dump, MAC address).2. The three fields in the config model
ButtonLogModeports the byte's fields and the firmware's correction rules (shimmer_button_log_mode.c, through the web SDK'sbuttonLogMode.ts), rule for rule. That includes the SR48-6-0 rule: press recording and the TTL input are taken out on that board.Accessors on
ShimmerObject:getButtonMode()/setButtonMode()getLogTriggers()/setLogTriggers()isTtlInputEnabled()/setTtlInputEnabled()setButtonLogMode(mode, triggers, ttl)getButtonLogMode()(the raw byte) andgetButtonLogModeCorrected()Fields can be set in any order. Log triggers set while the mode is still legacy are kept until the mode is set.
The firmware's side effects are applied to the model, in the setters and again before a write, on firmware that has the byte (see 3):
userButtonEnable(byte 217 bit 5) is held at the button mode's nearest equivalent.singleTouchStart(byte 218 bit 7) is cleared. No LogAndStream release is built with single touch (IS_SUPPORTED_SINGLE_TOUCHis 0 in log-and-stream-common), so each clears the bit on every configuration it takes, whatever byte 232 holds.Because this happens in the model,
writeTrial()(SET_TRIAL_CONFIG) sends the same bits. Other firmware is left alone, so a setter called for an SDLog Shimmer, the one firmware the driver offers Single Touch Start for, changes neither bit.Config labels.
Configuration.Shimmer3.GuiLabelConfig.BUTTON_MODE,LOG_TRIGGERSandTTL_INPUTgo throughset/getConfigValueUsingConfigLabel(), so the fields can also be set from aUSERfixed config map. They are not added to the Consensys option maps, so Consensys does not show them yet.3. Detection
LogAndStream_Shimmer3R v1.01.026, a pre-release of 11 Nov 2026, is the first Shimmer3R release with the event channel. No Shimmer3 release has it yet: the latest, v1.01.009, came out before it. The builds before a release report the version of the release before them, so the version cannot say whether a build has it. The driver asks instead:
ShimmerBluetooth.readEventChannelSupport()sendsSET_FEATURE(0xB7) withFEATURE_EVENT_STREAM(3) and 0. That changes nothing, since a connection starts opted out. It is sent on connect, andisEventChannelSupported()returns the answer.processNackFromCommand(), the NACK handling from Handle NACK from a Shimmer3/3R instead of dropping the connection #293, unwinds it like any refused command: the link stays up, the next command goes out, and it counts in the operation's progress. Because the NACK is an answer, it setsisEventChannelSupported()to false and is not passed toeventCommandRefused().SET_FEATUREarrived with log-and-stream-common (df58877 and e5ea88c, 2026-01-29), first released in LogAndStream_Shimmer3R v1.00.052 and LogAndStream_Shimmer3 v1.00.017. That isShimmerVerObject.isSupportedSetFeature(), and v1.00.017 is also the releaseisSupportedRn4678ErrorDetection()already gates on.LogAndStream/log-and-stream-common. Fixed in 955084c.SET_FEATUREbut not the event channel, so they NACK the probe.03isGET_SAMPLING_RATE_COMMAND, which the firmware ACKs, so a probe there would read as "supported". Those versions are never asked, and are treated as having no event channel.ConfigByteLayoutShimmer3.idxButtonLogMode) only from those releases on.isButtonLogModeMeaningful()): only when the probe was ACKed, or when the value read back is one the firmware keeps.0xFF, which reads as legacy and is written back as read.4. The event channel's host side (#348, #349)
Following log-and-stream-common
docs/SHIMMER3_EVENT_CHANNEL.mdanddocs/SHIMMER3_BT_COMMUNICATION_PROTOCOL.md§7.15, as shimmer-web-sdk#218 does:EventFileandEventRecordreadevents/<experiment>/<session>/<NNN>("SEVT", a 64-byte header, then 16-byte records).belongsWithDataFile()andgetCompanionFile()pair a file with its data file.writeEventStream(boolean)opts the connection in or out. Each record frame,[0x8A][0xBB][0x10][record][CRC-16], goes to everyEventChannelListener, idle or streaming, with any link CRC mode, and only once its own CRC checks out.readEventChannelStatus()sendsGET_EVENT_CHANNEL_STATUS_COMMAND(0xB9). It is refused while streaming, or while a start of streaming is queued, since its reply has no 0x8A wrapper.EventMarkersFramein ShimmerPCBasicExamples (SensorMapsExample, Tools › Event markers).ShimmerAndroidAPI'sShimmer.javaoverrides the streaming parser and does not know the record frame. Reading event files and byte 232 reach Android through the driver pin as usual.The same full-write problem in the sibling APIs
Shimmer-C-API)WriteShimmer3Infomem()is[Obsolete](unfinished) and writes the caller's 384 bytes unchanged. Its only callers in ShimmerCapture are commented out. SET_TRIAL_CONFIG sends back the bytes it last read.pyshimmer)SwiftAPI)writeShimmer3InfoMem()writes the caller's image unchanged, so the library keeps whatever it is given. TheShimmerExamplePlotapp mostly writes the image it read, modified. One hard-coded 384-byte preset (accel) has 0 at byte 232 and would turn a Shimmer back to legacy.The live event channel is in the web SDK (#218) and shimmer-host-core; C# still needs it.
Who needs a republish
Nothing reaches a consumer until a driver release is cut after this merges (the last is
API_v0.11.12_beta):shimmerdriver/shimmerbluetoothmanager0.11.12_beta. The release workflow opens the pin-bump PR. The probe's NACK is handled in the IO thread's not-streaming path, whichShimmer.javadoes not override, so it arrives with the pin. The live frame does not (section 4).ShimmerJavaClass.jar, also re-vendored by a PR the release workflow opens.Tests
API_00045_ButtonLogModeTest(ShimmerDriver):userButtonEnableheld and single touch;encode, setting fields in either order, and SR48-6-0.API_00046_ButtonLogModeConfigWriteTest(ShimmerDriverPC):userButtonEnableis held, and single touch is cleared in every mode;0xFFis kept as read (single touch still cleared), and a byte the firmware would correct is trusted only after the probe;API_00047_AudioConfigWriteTest(ShimmerDriverPC): before a read, a Shimmer3R's byte 233 is left out with 232; once read it goes back as read; the sound level turned on before a read is written; 232 set before a read is written without 233.API_00038_EventChannelProbeTest(ShimmerDriverPC), driving the real IO thread over an in-memory radio:SET_FEATURE, and a streaming Shimmer, are not asked.API_00042_EventRecordFileTest(ShimmerDriver),API_00043_EventStreamTestandAPI_00044_EventStreamReviewReproTest(ShimmerDriverPC): the event channel, from DEV-971: Java event markers - event files, live records, status, and a test window #348 and DEV-971: fix five event-channel parser defects found in #348's review #349.3606c951: without the byte-232 write, 9 of the 232 tests fail; without the NACK branch, 2 fail; with the old single-touch rule, 3 fail. At24d985e5, before DEV-971: #343 review fixes - leave byte 233 out before a read, one NACK handler, renumber tests #351: 3 ofAPI_00047's 5 fail without the byte-233 skip.a7dde8da, the merge of DEV-971: #343 review fixes - leave byte 233 out before a read, one NACK handler, renumber tests #351.d7d7fd80(DEV-971: #343 review fixes - leave byte 233 out before a read, one NACK handler, renumber tests #351, the same tree): ShimmerDrivertest219/219, ShimmerDriverPC 161/161 (clean build, as CI runs it), the Java 8 jar (shadowJar -PmatlabJava8) builds, and ShimmerPCBasicExamples compiles.On a unit
Results posted on #348, #349 and #351, over Bluetooth Classic with the 1-byte link CRC the driver sets at connect.
A Shimmer3R on v1.1.26, with the event channel:
0x04;startStreaming();A Shimmer3R on v1.1.25, the release before the event channel (at
d7d7fd80):isEventChannelSupported()was false, the opt-in and the status read were refused without being sent, the link stayed up, and a stream ran normally afterwards.[128, 104]and[234, 22], and the firmware ACKed both pieces. The unit kept its own bytes 232 and 233. Both pieces were stored, and the MAC read back as the unit's own.Not yet on a unit:
Review fix (
3606c951)The rules kept
singleTouchStartwherever byte 232 left a press that can start logging, following log-and-stream-common §8.3. That rule is only built withIS_SUPPORTED_SINGLE_TOUCH, which no LogAndStream release sets, so a full write sent a bit the Shimmer drops. The rules now clear it on firmware with the byte and change nothing on other firmware (section 2). §8.3 is corrected in ShimmerResearch/log-and-stream-common#216 (a531896), and the web SDK made the same fix in ShimmerResearch/shimmer-web-sdk#218.Review fixes (#351)
From the review comment of 2026-11-13: byte 233 is left out before a read as byte 232 is, the probe's NACK goes through the one NACK handler, and the button-mode tests are renumbered
API_00045andAPI_00046, sinceAPI_00036andAPI_00037were taken on master by the log-start-refused tests.🤖 Generated with Claude Code