Repository navigation
DEV-1095: anchor each SD file on its first packet's own counter time - #309
Merged
Merged
Conversation
The SD header's initial timestamp is the RTC when the file was created (sdFileSyncTs), not the first packet's time: the firmware captures that in firstTsForSdFile but never writes it back. Subtracting the first packet's raw timestamp pinned each file's first packet to the header time, erasing a creation-to-first-packet lead that differs per file. File 000 is created after sampling starts, so a Shimmer3R recording stepped back -158.8 ms at the 000 -> 001 split. SdTimestampAnchor rebuilds the first packet's full counter value from the header's high bits and the packet's 24-bit timestamp (signed, wrap- corrected), exact within +-256 s of the header whatever moment the firmware records. Both timestamp paths - ShimmerObject and SensorShimmerClock - use it. 2-byte counters, a zero initial timestamp and a lead over 10 s keep the previous behaviour. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Header and first-packet values from the recording's raw files: file 000's first packet predates its header by 5332 ticks (162.72 ms), file 001's by 129 (3.94 ms). The old rule reproduces the export's 158.78 ms step; the anchored split is exactly one sample period. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Leads of 214 and 131 ticks gave the export's other backwards step, -1.556 ms near hour 42. Both real splits now share one assertion helper. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Raw files from a 33-file Shimmer3 recording (LogAndStream v1.1.5, 1024 Hz) confirm the rule there. File 000's first record was sampled 1.831 ms after its header and every later file's 0.885 ms before, so the header's low 24 bits share the record counter, and all 32 splits close to exactly one sample period. The old rule's only artefact on that recording is +2.7 ms at the 000 to 001 split. The class doc said file 000's records predate its header by about 160 ms. That is the Shimmer3R; a Shimmer3 takes file 000's header first. Comment only. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes DEV-1095: SD-imported timestamps step at data-file splits. On a 66.8 h Shimmer3R recording, a permanent −158.8 ms step appears at the 000 → 001 split, and +15…+24 ms at every later split.
Cause
sdFileSyncTsinlog-and-stream-common), not the first packet's time. The firmware capturesfirstTsForSdFilebut never writes it back to the header.Fix
driverUtilities/SdTimestampAnchor, pure and static likeTimestampUnwrap. It rebuilds the first packet's full counter value from the header's high bits plus the packet's own 24-bit timestamp, as a signed, wrap-corrected distance.ShimmerObject.parseTimestampShimmer3andSensorShimmerClock.mFirstTsOffsetFromInitialTsTicksclaimed the firmware writes the first-packet time into the header. It doesn't, and the javadoc is corrected.Effect on output
Re-importing an existing recording shifts file 000 by roughly its SD start-up lead (about 160 ms on the recording above) and later files by a few ms. That's the correction, but exports made before and after this change will not match exactly.
Testing
API_00012_SdTimestampAnchorTest(11 tests), added toAPI_00005_Suite_ShimmerDriver. It covers:cd ShimmerDriver && ./gradlew test: 114 tests, 0 failures.ShimmerDriverPC. Here it fails before compiling, resolving:ShimmerBluetoothManager("No variants exist"), and an untouchedorigin/masterfails the same way, so it's environmental. This PR's CI is the check for that project.000/001files (Shimmer3R). Each file's first packet was sampled before its header timestamp: by 162.72 ms in 000 and 3.94 ms in 001. The old rule reproduces the Consensys export to the microsecond; with the fix, the gap across the 000 → 001 split is exactly one sample period. Those values are a test vector inAPI_00012(realShimmer3rRecordingSplitsByOnePeriod).Related
shimmer-web-sdk(sdlog/decode.tscopied this logic). DEV-1095: anchor each SD file on its first packet's own counter time shimmer-web-sdk#160.🤖 Generated with Claude Code