Release Notes Draft: 1.5

This is a maintainer's draft of the 1.5 release notes, listing commits and PRs from release/1.5.x since v1.4.145. Start by checking each linked entry against its commit or PR to confirm the wording and that the change really landed. The draft mentions python -m isaacteleop.cloudxr.service and scikit-build-core, which are worth verifying against the code. Done means the notes are accurate and their sections are complete.

Release Notes — Isaac Teleop 1.5

Everything that landed in release/1.5.x since v1.4.145.

Build & Packaging

Standard Python tooling can build and install Isaac Teleop from source

#735

A scikit-build-core backend now supports pip install ., editable installs, and source distributions while driving the same top-level CMake build. Retargeting, DeviceIO live-view, and session-manager examples are also installable under the shared isaacteleop_examples namespace and can be launched with python -m without relying on the checkout directory being on sys.path.

Authored Python sources moved under src/python

4bff0fd

The installed isaacteleop package is unchanged, but its authored sources now live under src/python/isaacteleop/ with standard __init__.py files instead of the former *_init.py shims. Out-of-tree tooling and editable-install helpers that read the repository by path must update their source locations; tests also moved into their own top-level directory.

CloudXR Runtime & Session Management

The CloudXR runtime is now a host service that survives application restarts

426c813, 1bffe26, 17be1cf

CloudXRService is now the single owner of the runtime process and WSS proxy. Applications attach to the running service instead of launching a runtime tied to their own lifetime, so restarting a teleoperation application no longer forces the headset to reconnect.

Use python -m isaacteleop.cloudxr.service to start, stop, inspect status, or read logs. Startup is serialized, refuses to overwrite a live runtime, and reports bind or WSS failures before returning instead of leaving a partially started service behind. The service is detached, not supervised: if it crashes, status reports that state and an operator must start it again. Container entry points that must own the runtime for their entire lifetime can opt into embedded operation with run_embedded=True.

The WebXR client can be reopened without restarting the runtime

783200a, #978

python -m isaacteleop.cloudxr.webclient reopens the versioned WebXR client on a USB-connected headset with the host address and port prefilled, without restarting the launcher or CloudXR runtime. It also accepts an explicit client URL, can resolve the target with --print-only without touching ADB, and redacts access tokens from terminal output.

The ROS 2 integration exposes the CloudXR.js OOB client route as a node parameter, allowing deployments to select the appropriate application route without modifying the bundled client.

OpenXR sessions wait for a headset by default

32c9cd6

Creating an OpenXRSession before the headset connects is now treated as the normal startup path. Sessions wait for an OpenXR system instead of immediately failing with XR_ERROR_FORM_FACTOR_UNAVAILABLE, so operators can start the host application first and connect the headset afterward.

Devices, Plugins & Partner Integrations

VIVE Ultimate Trackers stream as independent SE3 pose sources

#986 (partner contribution — HTC)

A new VIVE plugin reads Ultimate Tracker poses from VIVEHub and publishes each physical tracker as its own generic Se3TrackerPose collection, keyed by the tracker serial number. Stale poses are marked invalid, unchanged samples are not duplicated, and a synthetic mode supports hardware-free testing.

Record and replay examples discover the active tracker collections automatically, write them to MCAP, and visualize replayed poses with viser. The full live → record → replay path was verified with VIVEHub 1.0.1 hardware.

Wuji gloves provide OpenXR hand input and 20-DoF hand retargeting

#791, #997, #1027, #1090, #1099 (partner contribution — Wuji)

The optional Wuji plugin maps glove skeletons into the standard OpenXR 26-joint hand layout, fuses a wrist pose from optical tracking or a calibrated controller pose, and injects the result through the existing hand-tracking path. WujiHandRetargeter converts that input into 20-DoF commands for Wuji Hand and Wuji Hand 2, with live, synthetic, and replay workflows.

The ROS 2 integration can now install, launch, and supervise Wuji or MANUS glove plugins through TeleopSession, while externally managed and replay providers keep their selected routing.

Noitom full-body capture drives G1 teleoperation, recording, and replay

#742 (partner contribution — Noitom)

The new Noitom plugin reads Hybrid Data Server updates through MocapApi and publishes the standard vendor-neutral FullBodyPose layout. A complete example records the stream to the existing MCAP schema, replays it through the generic full-body workflow, and maps shoulder, elbow, and wrist motion into a G1 upper-body Pink IK task in Isaac Lab.

Sharpa Avatar gloves provide hand tracking, joint streams, and per-finger haptics

#1150, #1185 (partner contribution — Sharpa)

The optional Linux plugin connects Sharpa Avatar gloves through the pinned Avatar SDK and publishes HUMAN landmarks as standard OpenXR 26-joint hands. It also exposes the RAW and ROBOT 22-DoF joint streams through DeviceIO and consumes per-finger vibration commands through the existing haptic interface.

Left and right gloves connect independently, recover after disconnection, and can operate one side at a time. A TeleopSession example provides live browser visualization and pinch-triggered haptic feedback, while the Sharpa Pink retargeter converts OpenXR glove input into robot-hand commands.

The SO-101 clutch retargeter uses an engage-relative full pose

f079c85

home_base_T_ee is now required and supplies both the position and orientation used at engagement. Engagement also requires squeeze input above squeeze_threshold; orientation_offset and debug_log_engage were removed. Put the desired gripper orientation into home_base_T_ee and update constructor calls accordingly. Custom graphs can use ENGAGE_PERMITTED_INPUT to gate engagement and select either the controller grip or aim pose through controller_pose.

The constructor also accepts position_scale, allowing each Cartesian axis of engage-relative controller motion to be scaled before it is mapped to the follower arm.

Rate limiters guard position-servo followers from tracking jumps and IK divergence

860018e (from #727)

EePoseRateLimiter and JointRateLimiter bound the change in each command before it reaches a position-controlled follower. Motion within the configured limits passes unchanged, larger steps are clamped, and implausible input velocities are rejected instead of being approached at full speed.

reBot B601-RS can be brought up from Jetson AGX Orin's built-in CAN controller

f25369a

A new reBot workflow connects the seven-motor B601-RS arm to Jetson AGX Orin through its built-in mttcan controller and an external SN65HVD230 transceiver, avoiding a USB-CAN or PCAN adapter. It includes verified wiring guidance, repeatable CAN pinmux and interface setup, a systemd boot service, Motorbridge gateway startup, and an agent-readable troubleshooting procedure.

The MANUS integration moves to SDK 3.2.0

#925, #963, #1066 (SDK 3.2.0 update contributed by MANUS in #1066)

The installer now downloads and verifies MANUS SDK 3.2.0 for both x86_64 and aarch64, pins the build to that SDK instead of reusing a stale cached path, and retains the archive for reliable rebuilds. Applications can also supply MANUS calibration files through the plugin interface, and the plugin handles SIGINT and SIGTERM so the SDK disconnects cleanly on process shutdown.

Visualization, Robot Preview & Safety

camera_viz can be controlled entirely from the headset

#969

Quest controller bindings now adjust stereo distance and surface parameters, cycle lock mode, mono/stereo presentation and surface shape, aim or recenter equirectangular content, and reset the active view. A head-locked HUD shows the current values in XR, while a live terminal panel reports source and render status without scrolling.

CloudXR runtime settings move into the camera YAML and are written through an environment config that takes precedence over stale shell values. The sample also enables foveation correctly and keeps blocking terminal output off the video submit thread, preventing a paused SSH or tmux terminal from stalling the camera feed.

A reusable MuJoCo robot twin previews the commanded robot before engagement

0467dc5, 322ad9a, 7513288, 8084d65, 672fa29

The MuJoCo-backed robot preview is now part of isaacteleop.viz.robot instead of being buried in one example. It shares one OpenXR session for rendering and input, keeps frames on the GPU, and provides clutch previews for the SO-101 follower and reBot DevArm.

Before engagement, the operator can align the preview with the physical arm and see whether a command is accepted, rate-limited, or rejected. The operator frame measures the XR-to-robot yaw rather than assuming where the user stands relative to the arm. A shared in-headset controller guide now shows the controls for both engaged and disengaged states.

Recording & Tracker APIs

DeviceIO tracker stacks are generated from a declarative manifest

#853

Schema-based FlatBuffer trackers can now be declared in TOML instead of requiring a hand-written facade plus separate live, replay, Python-binding, and MCAP plumbing. CMake generates the tracker stack at configure time, expands shared defaults, registers the live and replay factories, and prunes stale generated outputs.

This makes partner-device onboarding more repeatable while preserving hand-written implementations for trackers that require OpenXR xrLocate* calls, opaque channels, custom connection state, or multi-endpoint haptic behavior.

MCAP replay rejects incompatible schemas before reading the first frame

#1026

Every recording already carried its FlatBuffer schema, but replay did not compare it with the running build. A recording made before an incompatible schema change could therefore decode successfully while returning incorrect field values.

Build-time conformance tests now prevent incompatible schema changes, and replay validates every embedded schema when the session opens. Incompatible recordings fail with a clear diagnostic before any frames are consumed; compatible additive changes continue to work.

Tracker data is published as serialized FlatBuffers with zero-copy joint access

#865, #836

The tracker API now carries immutable serialized FlatBuffer views instead of repeatedly unpacking and copying native-table wrappers. Python hand and body joint fields are exposed as strided NumPy views over the serialized storage, reducing per-frame Python/C++ crossings and copies while preserving each returned hand or body frame's backing storage.

Notable Fixes

camera_viz no longer crashes while waiting for a headset or running without controllers

#1171

XR camera streaming now waits for the headset by default instead of losing the startup race and failing with XR_ERROR_FORM_FACTOR_UNAVAILABLE. The wait is configurable with display.xr.system_wait_seconds or --xr-wait; negative waits indefinitely and 0 restores fail-fast behavior.

Controller handling now uses the snapshot returned by ControllerTracker directly. This prevents the render thread from crashing on its first frame when no controller is bound, including hand-tracking-only headset sessions.

Jetson Thor RTP streaming produces video on JetPack R39

#1190

The GStreamer NVENC sender previously reported a healthy frame rate on JetPack R39 while emitting no packets, leaving the receiver black. It now stages system-memory NV12 frames through nvvidconv into NVMM and only sets maxperf-enable when the installed encoder supports it. The fix was verified from a RealSense D435 on Thor R39 to an RTX PRO 6000 workstation at 1280×720 and 30 FPS.

Consumers can distinguish readable poses from actively tracked poses

#909

Head, controller, and hand data now expose OpenXR's tracked state separately from pose validity. A pose can remain readable while optical tracking is lost, so applications can detect the dropout and hold, warn, or disengage instead of treating a predicted or stale pose as live input. The status is propagated through the standard retargeting tensors for downstream LeRobot and Gear workflows.

A glove tracking dropout no longer commands the hand to snap open

#1029 (community-submitted fix)

The ROS 2 example previously published all-zero finger joints when Wuji tracking was lost. Because the same values are also a valid flat-open hand, downstream controllers could not distinguish the dropout and visibly drove the hand open mid-grasp. Tracking validity is now applied per side: an untracked side is omitted so controllers can hold the last valid command, while the other hand continues updating.

Starting a second CloudXR instance no longer kills or corrupts a live session

#908, 53a1d7a, 1bffe26

The launcher used to treat the runtime socket as stale and terminate its owner, even when it belonged to a healthy session. It now probes liveness, refuses the second start with actionable guidance, checks before rewriting the shared environment, and serializes concurrent starts so two runtimes cannot race for the same socket.

The partner OOB client loads reliably from the built-in server

#791 (partner fix — Wuji)

Static /client responses now include Content-Length, so the headset browser can determine where each response ends instead of waiting for the connection to close and failing to finish loading the UI.

Reusing a build directory no longer breaks vendor plugin installation

#1022

The MANUS and Wuji installers now explicitly enable their plugins when they reuse an existing top-level build directory, and Wuji installs through its own component into the canonical plugin layout.

The hosted WebXR client loads correctly without a trailing slash

#949

Opening /client previously caused the browser to resolve bundle.js outside the client directory, preventing the UI from loading. The built-in server now redirects /client to /client/ so relative assets resolve correctly.

Breaking Changes

Python 3.10 and ROS 2 Humble are no longer supported

#848

Isaac Teleop now requires Python 3.11–3.13. ROS 2 examples and containers move to Jazzy with Python 3.12 because Humble's rclpy is built for Python 3.10 and cannot consume the 1.5 wheel.

Tracker accessors return serialized schema objects

#865

The generated Python *T native-table types and most C++ *Tracked wrapper types were removed. Update consumers to use the schema objects returned by tracker accessors directly; C++ extensions should accept Serialized<T> and retain that handle for as long as they access the FlatBuffer view.

The retargeting head tensor is named HeadInput

#862

The retargeting tensor type formerly named HeadPose is now HeadInput, with corresponding HeadInputIndex and helper updates. Rename imports, tensor specifications, selectors, and index references in custom retargeting graphs; the DeviceIO FlatBuffer table remains HeadPose.

Source builds require CMake 3.24 or newer and no longer use presets

962abd5

Ubuntu 22.04's default CMake 3.22 is no longer sufficient. Install CMake from Kitware or PyPI, or let the scikit-build-core source-install path provide a compatible CMake. CMakePresets.json was removed; use the documented cmake -B build flow instead. Changing ISAAC_TELEOP_PYTHON_VERSION in an existing build tree now fails explicitly, and wheel builds use build-wheel/<cache-tag> rather than sharing build/.