Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Bug] SO-101 can make discontinuous joint jumps during Grip motion and stationary re-clutch

Abierto
  • ¿Algo no está bien?
#790 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
35/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
python
Área
robotics

Línea de trabajo

Comienza con el punto de entrada examples.isaac_teleop_to_so101.teleoperate y reproduce los dos escenarios de embrague utilizando los detalles del comando y de la plataforma proporcionados. Compara el comportamiento de Isaac Teleop con la corrección posterior f611bba y determina si un engage-frame hold o un joint-state rebase debe pertenecer al límite de integración; se considera terminado cuando el movimiento es continuo entre las articulaciones y no hay movimiento al volver a activar un controlador estacionario.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Describe the bug

When teleoperating an SO-101 with a Meta Quest 3S Touch Plus controller through CloudXR and Isaac Teleop, I observed sudden discontinuous robot motion in two situations:

  1. A joint jump while continuously holding Grip/Squeeze and moving the controller.
  2. A downward jump after moving the robot to a pose, releasing Grip, keeping the controller stationary, and pressing Grip again to re-engage the clutch.

The Quest controller pose and button input otherwise reached the Ubuntu host correctly. Grip/Squeeze acted as the clutch and Trigger controlled the gripper.

This was reproduced through the LeRobot SO-101 integration. I am filing it here because I would like guidance on the intended Isaac Teleop clutch/re-engagement behavior for underactuated arms and whether joint-continuity handling should be exposed or documented at the Isaac Teleop integration boundary.

To reproduce

  1. Start CloudXR Runtime 6.2.0 on Ubuntu and connect a Quest 3S through the official CloudXR web client.
  2. Connect an SO-101 follower and run the LeRobot Isaac Teleop example:
python -m examples.isaac_teleop_to_so101.teleoperate \
  --robot.type=so101_follower \
  --robot.port=/dev/serial/by-id/<SO101_SERIAL_DEVICE> \
  --robot.id=so101_follower_main \
  --teleop.type=xr_controller \
  --reset_to_origin=false

  1. Reproduction A:

    • Hold Grip continuously.
    • Move the right Quest controller through the workspace.
    • Observe that the arm can suddenly switch to a discontinuous elbow/wrist configuration.

  2. Reproduction B:

    • Hold Grip and move the arm to a reachable pose.
    • Release Grip and confirm that the arm holds.
    • Keep the controller stationary.
    • Press Grip again.
    • Observe an immediate downward jump on re-engagement.

Expected behavior

  • Motion should remain joint-continuous while Grip is held.
  • Releasing Grip should hold the robot.
  • Re-engaging Grip with a stationary controller should produce no robot movement on the engage frame.
  • The following IK frames should continue from the measured/current joint configuration rather than switching to another local solution.

Observed behavior

In a quantified re-clutch run before the downstream fix, three physical Grip re-engagements produced first-frame maximum joint-target changes of:

  • 6.422 degrees with 14.006 mm end-effector position change
  • 3.116 degrees with 6.815 mm end-effector position change
  • 3.186 degrees with 7.151 mm end-effector position change

The reported orientation change was 0 degrees in all three cases. The largest changes were mainly in elbow_flex, with wrist_flex also visibly involved in another event.

Videos

Platforms

  • Isaac Teleop Version: 1.3.131
  • CloudXR Runtime: 6.2.0
  • OS: Ubuntu 24.04 LTS, kernel 6.8.0-134-generic, x86_64
  • GPU: NVIDIA GeForce RTX 4080, driver 595.71.05
  • Integration: LeRobot v0.6.0, examples/isaac_teleop_to_so101
  • XR Device: Meta Quest 3S with Touch Plus controllers
  • Robot: SO-101 follower, 5-DoF arm plus gripper
  • Control target rate: 30 Hz

Investigation and downstream workaround

The investigation in the LeRobot integration found two relevant behaviors:

  • A calibrated/measured joint value can be slightly outside the URDF nominal limit (for example, shoulder_lift=-103 degrees while the URDF lower limit is approximately -100 degrees). A same-pose q -> FK(q) -> IK(FK(q)) operation can therefore project the state into the solver's feasible joint space instead of returning the measured joint configuration.
  • On this 5-DoF arm, the same-pose projection may require multiple solves to settle, and a small first controller displacement near an underconstrained or near-singular pose can select a more distant local joint solution.

A downstream fix was implemented in commit f611bba, currently included in the independent draft LeRobot PR #4046.

The workaround keeps the measured arm joints exactly unchanged on the engage frame, pre-converges the same-pose IK state, adds native dq regularization, and rebases subsequent solver increments onto the measured joint configuration. The gripper remains an absolute 0-100 command.

After the fix, 47 physical re-engagements on the SO-101 produced a 0.000000-degree engage-frame joint jump, and the original elbow/wrist jump did not recur during small and larger XYZ motions.

Questions

  1. Is there an intended Isaac Teleop pattern for guaranteeing joint-space continuity at clutch re-engagement on underactuated arms?
  2. Should the clutch/integration API provide an engage-frame hold or joint-state rebase hook before Cartesian IK resumes?
  3. Would NVIDIA prefer this behavior to remain entirely integration-specific, or should the Isaac Teleop documentation/example cover it?

Additional context

No hand tracking was used. The test used only the Quest 3S right Touch Plus controller. No cameras, data recording, or leader arm were involved.

Lenguaje dominante
Python
Estrellas
403
Forks
94
Merge medio
2 d 17 min
PR fusionados (30 d)
121

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/IsaacCapture

Todos los issues de NVIDIA/IsaacCapture

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.