Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Feature Request] Supported Unity/Android client path for a native Meta Quest robot teleoperation app

Abierto
  • ¿Algo no está bien?
#784 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Necesita aclaración
Estado de actividad
Tranquilo
Stack tecnológico
android, unity
Área
mobile, robotics

Línea de trabajo

Comienza revisando el cliente de navegador existente y el flujo de trabajo examples/isaac_teleop_to_so101, junto con la documentación de CloudXR.js mencionada. Determina si existe una ruta de cliente Unity/Android compatible con CloudXR Runtime 6.x e Isaac Teleop. Se consideraría terminado si hubiera una ruta nativa para Quest documentada o respaldada por un ejemplo que cubriera el streaming, las acciones de los controladores, las vistas de la cámara del robot y la desconexión segura.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Is your feature request related to a problem? Please describe.

I would like to build a native Unity/OpenXR application for the Meta Quest Store that allows users to:

  • view low-latency live video from a real robot;
  • control the robot using Quest Touch controllers;
  • use clutch, gripper, stop/hold, and disconnect controls inside the application.

I have already successfully tested the following real-hardware workflow:

Quest 3S Touch Plus controllers
→ CloudXR Runtime 6.2.0
→ NVIDIA Isaac Teleop 1.3.131
→ LeRobot v0.6.0 official examples/isaac_teleop_to_so101 example
→ SO-101 Follower robot

Test system:

  • Ubuntu 24.04
  • NVIDIA RTX 4080
  • Quest 3S
  • SO-101 robot arm

The Quest controller poses and buttons reached the Ubuntu host correctly. Grip/Squeeze worked as the clutch, Trigger controlled the gripper, and the SO-101 followed the controller successfully.

The existing browser client is excellent for validating the workflow. However, for an application that can be distributed through the Meta Quest Store, I need a native Unity/Android client with its own UI, connection flow, camera layout, robot selection, and safe-disconnection behavior.

My understanding is that the current CloudXR 6.

https://github.com/user-attachments/assets/e5cf9dcf-1734-42d9-aee0-9f998278e5ad

x documentation supports Quest through CloudXR.js in the browser, but I could not find a supported, redistributable native Unity/Android client path compatible with CloudXR Runtime 6.x and Isaac Teleop.

Please correct me if an existing supported option already covers this use case.

Describe the solution you'd like

Ideally, NVIDIA would provide one of the following:

  1. A supported CloudXR Unity package or Android client SDK compatible with CloudXR Runtime 6.x; or
  2. A stable and documented client API, together with a minimal Unity/OpenXR sample.

The Quest client should ideally support:

  • receiving the low-latency streamed XR view;
  • displaying one or two real robot camera views;
  • sending HMD and Touch-controller poses and actions upstream;
  • Unity OpenXR integration;
  • connection, reconnection, latency, and streaming-quality status;
  • Grip-to-clutch and Trigger-to-gripper mappings;
  • a clear stop/hold state after clutch release or connection loss;
  • redistribution as part of a Meta Quest Store application.

A first MVP could be LAN-only:

Robot camera(s)
    → camera_viz / Televiz
    → CloudXR Runtime
    → Native Unity app on Quest

Quest controller poses and buttons
    → CloudXR
    → Isaac Teleop
    → LeRobot
    → SO-101
Lenguaje dominante
Python
Estrellas
393
Forks
93
Merge medio
2 d 5 h
PR fusionados (30 d)
95

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/IsaacCapture

Todos los issues de NVIDIA/IsaacCapture

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.