Skip to content

mecka: fix the double-mirrored LEFT wrist frame, and correct existing zarrs - #655

Merged
RyanPCo merged 1 commit into
mainfrom
ryanco/mecka-left-wrist-fix
Oct 19, 2026
Merged

RyanPCo merged 1 commit into
mainfrom
ryanco/mecka-left-wrist-fix

Conversation

@RyanPCo

@RyanPCo RyanPCo commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Branch 3 of 8 in the split of #640, which is the tip of the stack and carries the map of all eight.
Unit suite on this branch: 460 passed, 0 failed.

compute_hand_pose_xyzquat built the palm normal as cross(thumb_dir, pinky_dir).
Thumb and pinky are anatomically swapped between hands, so that cross product
already mirrors chirality for free; applying rot_left on top of it flipped x/y
a second time and landed the LEFT hand on the RIGHT hand's spatial convention.
Both hands now use the same rot_right relabel.

Reconverting every mecka zarr would be the obvious fix, so this avoids it:
rot_left == rot_right @ diag(-1, -1, 1), which makes right-multiplying the
stored left pose by Rz(180 deg) exactly equivalent to reconverting with
rot_right. RotateLocalFrame does that right-multiply (R_new = R_old @ R_fix),
relabelling the pose's own axes without moving its origin, on (7,) poses and
(T, 7) chunks alike, and leaves zero-norm quaternions alone because those mark
padded frames.

Human.get_transform_list(fix_mecka_left_wrist=True) prepends it to the raw LEFT
pose keys before any frame math: ee_pose in the cartesian modes, wrist_pose in
the keypoints modes, since the converter wrote the same rotation into both.
data/mecka.yaml opts in; aria and scale must not, because they run different
converters, and neither must mecka data reconverted after this commit, which
would double-flip.

test_wrist6d_roundtrip runs the real transform lists on synthetic world-frame
poses, parametrized over the flag, and compares every stage against plain
numpy/scipy SE(3) math that shares no code with the pipeline.

Co-Authored-By: Claude Opus 5 (1M context) [email protected]

🤖 Generated with Claude Code

RyanPCo commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@RyanPCo
RyanPCo marked this pull request as ready for review September 29, 2026 21:28
@RyanPCo

RyanPCo commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai

@greptile-apps

greptile-apps Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[Medium risk] Fixes hand pose convention in motion capture data processing.

The PR should not merge until mixed-convention datasets cannot silently receive the wrong left-wrist correction.

Findings

  1. P1 Later episodes escape validation ▶
  2. P2 Convention test shares implementation ▶

Summary

The PR corrects the left-hand pose relabel in the Mecka converter, adds a load-time rotation for existing Mecka and Scale zarrs, and adds convention checks and pipeline tests.

  • Mixed old and newly converted episodes can evade the three-episode convention check.
  • The new convention tests lack an independent assertion of the converter’s hand-frame orientation.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Stored episode poses] --> B{Dataset correction enabled?}
  B -->|Yes| C[Rotate left pose local axes]
  B -->|No| D[Keep stored pose]
  C --> E[Human frame transforms]
  D --> E
  E --> F[Training actions and proprioception]
  A --> G[Check first three episodes]
  G --> H[Warn or reject detected mismatch]
Loading

Reviews (1) · Last reviewed commit: "mecka: fix the double-mirrored LEFT wris..."

fix_enabled = any(
isinstance(t, RotateLocalFrame) for t in (self.transform_list or [])
)
for name, ds in list(datasets.items())[: self.WRIST_CONVENTION_SAMPLE]:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Later episodes escape validation If a dataset contains both older and reconverted episodes, this check examines only the first three in sorted path order. When those three are older, a later reconverted episode still receives the left-wrist rotation but is never checked. Its wrist frame is double-flipped, so training mixes incompatible frames.



def _poses_from(kp, relabel):
from egomimic.scripts.mecka_process.mecka_to_zarr import _unrelabeled_hand_frame

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Convention test shares implementation The synthetic stored rotations come from _unrelabeled_hand_frame, which the converter and convention checker also use. A mistake in that helper could therefore leave these tests passing while both production paths produce the wrong orientation. An independent expected-frame check for both hands and the historical left-hand relabel would catch that regression.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

RyanPCo commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Merge activity

  • Oct 19, 7:23 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Oct 19, 7:35 PM UTC: Graphite rebased this pull request as part of a merge.
  • Oct 19, 7:35 PM UTC: @RyanPCo merged this pull request with Graphite.

@RyanPCo
RyanPCo changed the base branch from ryanco/embodiment-6d-modes to graphite-base/655 October 19, 2026 19:32
@RyanPCo
RyanPCo changed the base branch from graphite-base/655 to main October 19, 2026 19:33
… zarrs

compute_hand_pose_xyzquat built the palm normal as cross(thumb_dir, pinky_dir).
Thumb and pinky are anatomically swapped between hands, so that cross product
already mirrors chirality for free; applying rot_left on top of it flipped x/y
a second time and landed the LEFT hand on the RIGHT hand's spatial convention.
Both hands now use the same rot_right relabel.

Reconverting every affected zarr would be the obvious fix, so this avoids it:
rot_left == rot_right @ diag(-1, -1, 1), which makes right-multiplying the
stored left pose by Rz(180 deg) exactly equivalent to reconverting with
rot_right. RotateLocalFrame does that right-multiply (R_new = R_old @ R_fix),
relabelling the pose's own axes without moving its origin, on (7,) poses and
(T, 7) chunks alike, and leaves zero-norm quaternions alone because those mark
padded frames.

Human.get_transform_list(fix_left_wrist_convention=True) prepends it to the raw
LEFT pose keys before any frame math: ee_pose in the cartesian modes,
wrist_pose in the keypoints modes, since the converter wrote the same rotation
into both. The flag is a property of the DATA, not of the vendor: measured
against the production set (1833 of 1838 episodes probed, 2026-09-18), every
mecka episode AND all 20 scale episodes still carry the double-mirrored left
wrist, with the right-hand control clean, so data/mecka.yaml and data/scale.yaml
both opt in. Data reconverted after this commit must not, or it double-flips.

Because that correctness lives in the data, left_wrist_convention() re-derives
the verdict from an episode's own keypoints -- recomputing both wrists under
each relabel and comparing with the stored quaternions, with the RIGHT hand as
the control -- and S3EpisodeResolver samples a few resolved episodes at startup.
Only the double-flip raises: the flag on against post-fix data would correct
what the converter already corrected, which is a fresh error from a stale
setting. The flag off against pre-fix data is what every run did before this
commit, and the generic human configs (aria, human, the cotrains) have never
set it, so that warns and names the knob rather than turning their startup
into a crash. Five legacy 2025 episodes match neither convention (a third
converter wrote them); their control fails, so they warn too.

data/mecka_scale_cotrain_pi.yaml and data/industry_eva_pi.yaml are deleted:
both merge mecka and scale into ONE human_bimanual dataset under a single
transform_list, and the correction is per-lab, so neither file has a correct
value for the flag. model/pi0.5_cotrain_mecka_scale.yaml goes with them.

test_wrist6d_roundtrip runs the real transform lists on synthetic world-frame
poses, parametrized over the flag, and compares every stage against plain
numpy/scipy SE(3) math that shares no code with the pipeline; the convention
checker is tested the same way, on synthetic keypoints.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VBmhpVy7ZcMtsBJWh38cJp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant