Repository navigation
mecka: fix the double-mirrored LEFT wrist frame, and correct existing zarrs - #655
Conversation
|
| fix_enabled = any( | ||
| isinstance(t, RotateLocalFrame) for t in (self.transform_list or []) | ||
| ) | ||
| for name, ds in list(datasets.items())[: self.WRIST_CONVENTION_SAMPLE]: |
There was a problem hiding this comment.
Later episodes escape validation If a dataset contains both older and reconverted episodes, this check examines only the first three in sorted path order. When those three are older, a later reconverted episode still receives the left-wrist rotation but is never checked. Its wrist frame is double-flipped, so training mixes incompatible frames.
|
|
||
|
|
||
| def _poses_from(kp, relabel): | ||
| from egomimic.scripts.mecka_process.mecka_to_zarr import _unrelabeled_hand_frame |
There was a problem hiding this comment.
Convention test shares implementation The synthetic stored rotations come from
_unrelabeled_hand_frame, which the converter and convention checker also use. A mistake in that helper could therefore leave these tests passing while both production paths produce the wrong orientation. An independent expected-frame check for both hands and the historical left-hand relabel would catch that regression.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1880cd7 to
fedb16c
Compare
1eab436 to
823b4dc
Compare
fedb16c to
3301a22
Compare
… zarrs compute_hand_pose_xyzquat built the palm normal as cross(thumb_dir, pinky_dir). Thumb and pinky are anatomically swapped between hands, so that cross product already mirrors chirality for free; applying rot_left on top of it flipped x/y a second time and landed the LEFT hand on the RIGHT hand's spatial convention. Both hands now use the same rot_right relabel. Reconverting every affected zarr would be the obvious fix, so this avoids it: rot_left == rot_right @ diag(-1, -1, 1), which makes right-multiplying the stored left pose by Rz(180 deg) exactly equivalent to reconverting with rot_right. RotateLocalFrame does that right-multiply (R_new = R_old @ R_fix), relabelling the pose's own axes without moving its origin, on (7,) poses and (T, 7) chunks alike, and leaves zero-norm quaternions alone because those mark padded frames. Human.get_transform_list(fix_left_wrist_convention=True) prepends it to the raw LEFT pose keys before any frame math: ee_pose in the cartesian modes, wrist_pose in the keypoints modes, since the converter wrote the same rotation into both. The flag is a property of the DATA, not of the vendor: measured against the production set (1833 of 1838 episodes probed, 2026-09-18), every mecka episode AND all 20 scale episodes still carry the double-mirrored left wrist, with the right-hand control clean, so data/mecka.yaml and data/scale.yaml both opt in. Data reconverted after this commit must not, or it double-flips. Because that correctness lives in the data, left_wrist_convention() re-derives the verdict from an episode's own keypoints -- recomputing both wrists under each relabel and comparing with the stored quaternions, with the RIGHT hand as the control -- and S3EpisodeResolver samples a few resolved episodes at startup. Only the double-flip raises: the flag on against post-fix data would correct what the converter already corrected, which is a fresh error from a stale setting. The flag off against pre-fix data is what every run did before this commit, and the generic human configs (aria, human, the cotrains) have never set it, so that warns and names the knob rather than turning their startup into a crash. Five legacy 2025 episodes match neither convention (a third converter wrote them); their control fails, so they warn too. data/mecka_scale_cotrain_pi.yaml and data/industry_eva_pi.yaml are deleted: both merge mecka and scale into ONE human_bimanual dataset under a single transform_list, and the correction is per-lab, so neither file has a correct value for the flag. model/pi0.5_cotrain_mecka_scale.yaml goes with them. test_wrist6d_roundtrip runs the real transform lists on synthetic world-frame poses, parametrized over the flag, and compares every stage against plain numpy/scipy SE(3) math that shares no code with the pipeline; the convention checker is tested the same way, on synthetic keypoints. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01VBmhpVy7ZcMtsBJWh38cJp
3301a22 to
8631d8e
Compare

compute_hand_pose_xyzquat built the palm normal as cross(thumb_dir, pinky_dir).
Thumb and pinky are anatomically swapped between hands, so that cross product
already mirrors chirality for free; applying rot_left on top of it flipped x/y
a second time and landed the LEFT hand on the RIGHT hand's spatial convention.
Both hands now use the same rot_right relabel.
Reconverting every mecka zarr would be the obvious fix, so this avoids it:
rot_left == rot_right @ diag(-1, -1, 1), which makes right-multiplying the
stored left pose by Rz(180 deg) exactly equivalent to reconverting with
rot_right. RotateLocalFrame does that right-multiply (R_new = R_old @ R_fix),
relabelling the pose's own axes without moving its origin, on (7,) poses and
(T, 7) chunks alike, and leaves zero-norm quaternions alone because those mark
padded frames.
Human.get_transform_list(fix_mecka_left_wrist=True) prepends it to the raw LEFT
pose keys before any frame math: ee_pose in the cartesian modes, wrist_pose in
the keypoints modes, since the converter wrote the same rotation into both.
data/mecka.yaml opts in; aria and scale must not, because they run different
converters, and neither must mecka data reconverted after this commit, which
would double-flip.
test_wrist6d_roundtrip runs the real transform lists on synthetic world-frame
poses, parametrized over the flag, and compares every stage against plain
numpy/scipy SE(3) math that shares no code with the pipeline.
Co-Authored-By: Claude Opus 5 (1M context) [email protected]
🤖 Generated with Claude Code