Skip to content

feat(manipulation): add linear Cartesian planning - #3240

Open
TomCC7 wants to merge 30 commits into
mainfrom
cc/feat/constrained-planning
Open

feat(manipulation): add linear Cartesian planning#3240
TomCC7 wants to merge 30 commits into
mainfrom
cc/feat/constrained-planning

Conversation

@TomCC7

@TomCC7 TomCC7 commented Jul 28, 2026

Copy link
Copy Markdown
Member

Contribution path

Problem

The manipulation planner contract can only request collision-free joint-space
paths. It cannot express a straight tool-center-point motion for approaches,
retreats, or interactive Cartesian moves, even though RoboPlan provides an
official Cartesian path planner.

The Viser manipulation panel also resolves its Cartesian gizmos to joint goals
and invokes free-space planning, so there is no convenient way to exercise or
preview a linear TCP path.

Solution

  • Add a backend-neutral internal plan_linear_cartesian_path planner operation
    with absolute world-frame poses, relative Cartesian deltas, and the existing
    target-plus-auxiliary-group convention.
  • Implement the operation using RoboPlan's official bounded-speed Cartesian
    planner, including synchronized multi-group timing and full-model start-state
    handling.
  • Return combined globally named joint states with RoboPlan timestamps and
    velocities, and atomically post-validate waypoints and interpolated edges
    against the DimOS collision world.
  • Return UNSUPPORTED from planner backends without linear Cartesian support;
    never substitute a free-space path for a failed linear request.
  • Add typed RoboPlan Cartesian-planner configuration while retaining the legacy
    planner-name compatibility path.
  • Add a Viser Planning mode selector with Free-space as the default and
    Linear Cartesian for absolute transform-control targets. Existing preview
    and synchronized execution reuse the planner-supplied trajectory directly.

The public RPC, skill, MCP, and CLI motion APIs remain unchanged.

How to Test

Run the interactive feature:

uv run dimos run xarm7-planner-coordinator -o manipulationmodule.visualization.backend=viser

Move the TCP gizmo, choose Linear Cartesian, click Plan, and then
Preview.

Automated validation performed:

uv run pytest dimos/manipulation -m 'not (self_hosted or mujoco or self_hosted_large)'
uv run pytest -m self_hosted dimos/manipulation/test_roboplan_world_integration.py
uv run mypy dimos/manipulation

Results: 360 manipulation tests passed, 2 real-RoboPlan integration tests
passed, and mypy passed across all manipulation source files.

AI assistance

OpenAI Codex with GPT-5 was extensively involved in API exploration,
implementation, tests, documentation, and validation. The design decisions and
scope were reviewed collaboratively with the author.

Checklist

  • I have read and approved the CLA.

@mintlify

mintlify Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
dimensional 🟢 Ready View Preview Jul 28, 2026, 5:21 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.85673% with 77 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...imos/manipulation/planning/world/roboplan_world.py 79.65% 18 Missing and 17 partials ⚠️
dimos/manipulation/manipulation_module.py 74.02% 12 Missing and 8 partials ⚠️
dimos/manipulation/visualization/viser/gui.py 72.50% 7 Missing and 4 partials ⚠️
dimos/manipulation/planning/factory.py 64.28% 2 Missing and 3 partials ⚠️
...os/manipulation/test_roboplan_world_integration.py 94.20% 2 Missing and 2 partials ⚠️
dimos/manipulation/visualization/operator.py 80.00% 1 Missing and 1 partial ⚠️
@@            Coverage Diff             @@
##             main    #3240      +/-   ##
==========================================
+ Coverage   74.96%   75.10%   +0.13%     
==========================================
  Files        1117     1120       +3     
  Lines      106875   107513     +638     
  Branches     9699     9754      +55     
==========================================
+ Hits        80123    80746     +623     
+ Misses      23945    23935      -10     
- Partials     2807     2832      +25     
Flag Coverage Δ
OS-ubuntu-24.04-arm 68.56% <72.65%> (+0.02%) ⬆️
OS-ubuntu-latest 70.70% <80.02%> (+0.06%) ⬆️
Py-3.10 70.69% <80.02%> (+0.05%) ⬆️
Py-3.11 70.69% <80.02%> (+0.07%) ⬆️
Py-3.12 70.69% <80.02%> (+0.06%) ⬆️
Py-3.13 70.69% <80.02%> (+0.05%) ⬆️
Py-3.14 70.70% <80.02%> (+0.06%) ⬆️
Py-3.14t 70.69% <80.02%> (+0.06%) ⬆️
SelfHosted-Large 29.13% <19.97%> (-0.11%) ⬇️
SelfHosted-Linux 36.07% <41.53%> (+0.25%) ⬆️
SelfHosted-macOS 35.11% <41.53%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/manipulation/planning/planners/config.py 100.00% <100.00%> (ø)
...imos/manipulation/planning/planners/rrt_planner.py 35.50% <100.00%> (+0.98%) ⬆️
...imos/manipulation/planning/planners/test_config.py 100.00% <100.00%> (ø)
...on/planning/planners/test_rrt_planner_selection.py 100.00% <100.00%> (ø)
dimos/manipulation/planning/spec/models.py 98.83% <100.00%> (+0.02%) ⬆️
dimos/manipulation/planning/spec/protocols.py 100.00% <100.00%> (ø)
...anipulation/test_generated_plan_materialization.py 100.00% <100.00%> (ø)
dimos/manipulation/test_manipulation_module.py 100.00% <100.00%> (ø)
dimos/manipulation/test_manipulation_unit.py 100.00% <ø> (ø)
dimos/manipulation/test_planning_factory.py 98.09% <100.00%> (+0.17%) ⬆️
... and 10 more

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread dimos/manipulation/manipulation_module.py
Comment thread dimos/manipulation/planning/world/roboplan_world.py
Comment thread dimos/manipulation/planning/world/roboplan_world.py
Comment thread docs/capabilities/manipulation/index.md Outdated
Comment thread dimos/manipulation/planning/spec/models.py Outdated
@mintlify

mintlify Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
dimensional 🟡 Building Jul 28, 2026, 5:20 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@TomCC7
TomCC7 marked this pull request as ready for review July 28, 2026 06:13
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Jul 29, 2026
"""Plan a collision-free path for an ordered planning-group selection."""
...

def plan_linear_cartesian_path(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method should plan to CartesianTarget

Whether the path is Linear or otherwise the planner config can decide.

The planning API remains simpler and we can have a Spline Cartesian path planner.

"""List of joint states forming a path (each waypoint has names + positions)"""


CartesianTarget: TypeAlias = "PoseStamped | Transform"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into RoboPlan cartesian planner, and it appears it can do cartesian planning for a sequence of poses.

We should let CartesianTarget accept a list of PoseStamped or Transform. And for linear cartesian planning we can assert list should be only size 2.

And if we wanted to do a zigzag pattern, we can break the CartesianTarget list into lists of 2points that we feed to the planner.

Comment on lines +84 to +85
FREE_SPACE = "free_space"
LINEAR_CARTESIAN = "linear_cartesian"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be

JOINT_SPACE
CARTESIAN_SPACE

@mustafab0

Copy link
Copy Markdown
Contributor

Only halfway through.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants