Repository navigation
a whole bunch of agent assisted updates to myrobotlab. - #1444
Conversation
|
💯 |
…ices to the containers
…umentation for agents to build new services.
… the inmoov jmonkey simulator to display the current positions of the hands, also to let you zoom around and change orientation of the viewport. this adds the VirtualInMoovIkDemo that is an attempt to get decent forward and inverse kinematics working.
…reating a common orientation for the simulator and the ik3d solver.
…imulator, also some point click and reach support in the simulator.
supertick
left a comment
There was a problem hiding this comment.
Code review: inline findings below, ranked roughly by severity. Jme3Util, InMoov2Arm (via WebXR) and the Runtime startup changes look most urgent.
One finding couldn't be attached inline because GitHub doesn't render the JMonkeyEngine.java diff (too large):
JMonkeyEngine.java~L1225:getAngleignores the axis and the node's real rotation. It now returns the last angle passed torotateTo. For a node never driven throughrotateTo, or moved by animation, physics or a loaded scene,currentAngleDegis 0, sogetAnglereturnsmapper.calcInput(0)(e.g. 10 for omoplate) whatever the mesh actually shows.getAngle(name, "z")also returns the same value asgetAngle(name, "x").
🤖 Generated with Claude Code
| // Rotate about the bone's own axis, starting from its authored bind pose. | ||
| // Decomposing to Euler angles and substituting one term (the old approach) | ||
| // rewrites the bind rotation and gimbal-locks the Z axis at ±90°. | ||
| Quaternion bind = o.captureBindRotation(); |
There was a problem hiding this comment.
Non-arm joints now render offset by their rest angle. rotateTo now applies the mapped angle on top of the captured bind rotation (bind * R(axis, deg)) instead of setting one absolute Euler component. Only the arm mappers in InMoov2Config were re-based so that rest maps to 0°. The head, torso, eye, hand and finger mappers still output absolute angles. Example: torso.midStom (0..180 → 50..130 on y) at rest 90 used to set Euler Y=90°. Now it rotates 90° past the authored bind pose. The same goes for eyeY, head.rothead (no mapper, 90 at rest) and every finger.
| * The IK demo overwrites these from measured VinMoov node distances when the | ||
| * simulator is running. | ||
| */ | ||
| public static final double DH_OMOPLATE_A = 0.040; |
There was a problem hiding this comment.
Unit change breaks WebXR IK. getDHRobotArm now builds the arm in meters (0.04, 0.28 …) instead of millimeters, but WebXR.java (around line 88) still sends millimeter targets: ik.moveTo("left", new Point(70 + x, -550 + y, z)). That target is about 550 m away for an arm that reaches about 0.6 m, so the solver never converges and the arm saturates at its joint limits on every pose update. WebXR needs to switch to meters, or the units need to stay consistent.
| Peer subPeer = subPeers.get(subPeerKey); | ||
| if (subPeer.autoStart) { | ||
| ServiceInterface peerSi = Runtime.create(sc.getPeerName(subPeerKey), subPeer.type); | ||
| if (peerSi != null && !peerSi.isRunning()) { |
There was a problem hiding this comment.
Nested autoStart peers are no longer started. Auto-start sub-peers are now created with Runtime.create, and only that peer is queued to start. The old recursive Runtime.start also started the peer's own autoStart peers. If A auto-starts B and B auto-starts C, Runtime.start("a", type) creates B (and maybe C through the plan) but only calls startService on B, so C is never started.
| return requestedService; | ||
| for (ServiceInterface service : startOrder) { | ||
| try { | ||
| if (service != null && !service.isRunning()) { |
There was a problem hiding this comment.
Race: startService now runs outside processLock. Thread 1 creates service X, releases processLock and is about to call startService. Thread 2 calls Runtime.release("x") under the lock and unregisters it. Thread 1 then starts the released, unregistered X: its threads run, and it sends started for a service that no longer exists. Also, a concurrent Runtime.start(name) for a service that exists but isn't running yet can return it before it has started.
| invoke("publishListening", true); | ||
| } | ||
|
|
||
| private synchronized void stopMicrophoneCapture() { |
There was a problem hiding this comment.
Possible native use-after-free. stopMicrophoneCapture only waits 2 s for the recognition thread, then stopService/loadModelFromPath call closeVosk(), which frees the native Recognizer and Model. If acceptWaveForm or a blocked microphone.read keeps the thread busy past the join, the thread later calls getFinalResult() on a closed recognizer and can crash the JVM. Join without a timeout (after interrupting or closing the mic), or check that the thread has exited before closing.
| // startService) binds VinMoov and applies node mappers — do not touch the | ||
| // live scene graph again here from this thread (JME is not thread-safe). | ||
| JMonkeyEngine jme = (JMonkeyEngine) Runtime.create(simName, "JMonkeyEngine"); | ||
| if (jme == null) { |
There was a problem hiding this comment.
startSimulator bypasses startPeer. It hardcodes Runtime.create(simName, "JMonkeyEngine") and ignores the simulator peer type in config. If a service of another type already exists under that name, the (Simulator) cast in the existing branch throws ClassCastException. It also no longer calls broadcastState as startPeer did, so the WebGui misses the peer-state update.
|
|
||
| @Override | ||
| public void startService() { | ||
| super.startService(); |
There was a problem hiding this comment.
startService runs pip install --upgrade on the calling thread every time. With py4jInstall=true, every OakD start (and InMoov2 startChestCamera) blocks on pip resolving and downloading depthai, numpy, opencv and blobconverter. That takes seconds online, and offline it stalls until timeout. Because of --upgrade, it hits the network even when everything is already installed. Consider installing only when the packages are missing, or doing it once at install time.
| // retrieve Maven runtime-scoped classified artifacts, and retrieve by | ||
| // [originalname] can overwrite lwjgl.jar with the last native jar. | ||
| String lwjglVersion = "3.3.2"; | ||
| addDependency("org.lwjgl", "lwjgl-opencl", lwjglVersion); |
There was a problem hiding this comment.
Old LWJGL jars stay on the classpath. LWJGL moves from 3.2.3 to 3.3.2 and the Ivy retrieve pattern changes, but existing libraries/jar/lwjgl-*-3.2.3.jar files aren't removed. Reinstalling JMonkeyEngine puts the 3.3.2 jars next to them, which mixes versions on the classpath. That can cause NoClassDefFoundError (e.g. CallbackI$V) or native version-mismatch errors when GLFW starts.
| int ret; | ||
| try { | ||
| ret = proc.waitFor(); | ||
| reader.join(8000); |
There was a problem hiding this comment.
Data race on pipOut. pipOut is a plain StringBuilder that the reader thread appends to. If reader.join(8000) times out (e.g. a lingering child process keeps stdout open), tailLines(pipOut.toString()) races with append. It can throw or return garbled text, which hides the real pip error. Use a StringBuffer or synchronize, and/or join without a timeout after the process exits.
|
From Grok: The ten review comments on pull request 1444 are fixed in the current code. The focused tests for the mapper rebase, the joint-angle math, and the old-jar cleanup passed. Simulator joints. getAngle now reads the bone's twist about the requested axis from its rotation relative to the bind pose, so X and Z are no longer the same cached value. Head, torso, eye, hand, and finger mappers in InMoov2Config are shifted so the servo rest angle maps to 0° from that bind pose, which is the same contract the arm mappers already used. WebXR and the simulator peer. The leftover 1,000-step IK loop is gone. The left-hand target uses meter offsets (0.07, -0.55) so it matches the arm's meter-long links. startSimulator goes through startPeer again, so it uses the configured peer type and publishes state. Startup. Auto-start peers are queued together with their own auto-start children, and those children are started first. startService runs only if that same instance is still registered. Release marks the instance first, so a start that loses the race does not launch threads for a service that has already been removed. Vosk, OakD, and installs. Microphone capture closes the line, then waits until the recognition thread has exited, before the native recognizer is freed. OakD runs pip install only when depthai, numpy, or cv2 is missing, and that install no longer passes --upgrade. Pip output is collected in a StringBuffer, and the reader is joined again if it is still alive after the process exits. After an Ivy install, jars from an older revision of that same artifact are deleted so LWJGL 3.2.3 and 3.3.2 cannot both sit on the classpath. |
…cache it, also fixing a unit test in http client
|
Second review pass, covering the two newest commits ( 1. A failed VinMoov download is retried on the render thread. (JMonkeyEngine.java#L4708) 2. The model is fetched from 3. Vosk: a capture started during Smaller issue in the same area (#L863): the loop reads the 4. Minor: the 30 s timeout in Everything else in these two commits looks good: the startup queue and 🤖 Generated with Claude Code |
The four review notes are fixed. VinMoovModelPinTest passes (2 tests). 1. VinMoov download stays off the render thread. loadDelayed still calls ensureVinMoovModel() before the scene work is queued. loadDefaultModels() skips that download when it is already on the jME thread, so a failed fetch is not retried inside the 120-second render task. A call from any other thread still downloads. 2. The mesh is pinned. The URL is commit 79cb407d20e40864dc87856057e00be58b126954, not master. The file must be 67,449,927 bytes and SHA-256 14f0f17fb550f64515eb2edfa7092a0e62779bc1727a97be734bf904af91bb2a. A Content-Length that does not match the body is rejected. A VinMoov5.j3o already in the models directory with the wrong size is replaced; other local meshes are left as they are. 3. Vosk cannot free a recognizer that a capture thread is using. loadModelFromPath and stopService check again under the lock. If a capture appeared in the gap, they stop it and retry before closeVosk(). acceptWaveForm, getResult, getPartialResult, and getFinalResult hold that same lock, and closeVosk() takes it too. The capture loop reads the microphone into a local once, and an error after stop is not published as a recognition failure. 4. The Py4j import check can actually time out. venvCanImport discards the child process output and then calls waitFor(30, SECONDS). A hung import is destroyed instead of blocking OakD.startService with no limit.
lots of stuff.. look at the commits.