Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

isaac_ros_foundationpose on zed camera

Open
  • Is something wrong?
#51 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Domain
ai, computer-vision, robotics

Research direction

Start with the FoundationPose documentation and reproduce the supplied isaac_ros_examples launch command using the zed_mono_rect_depth and foundationpose launch fragments. Inspect the reported shutdown output and the foundationpose_zed.rviz configuration, then compare the ZED setup with the documented rosbag, RealSense, and Hawk camera paths. Done means identifying the failure and clarifying ZED and alternative detector support.

Written by the indexing model from the issue text.

Description

Hi, I am trying to run foundationpose on zed camera.

In the documentation, I don't see a column for zed camera, but there is for rosbag, realsense camera, and hawk camera. I managed to look at centerpose and compared the launch arguments.
So I ran this launch command

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=zed_mono_rect_depth,foundationpose mesh_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/Mac_and_cheese_0_1/Mac_and_cheese_0_1.obj texture_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/Mac_and_cheese_0_1/materials/textures/baked_mesh_tex0.png score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan rt_detr_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/synthetica_detr/sdetr_grasp.plan interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_centerpose/zed2_quickstart_interface_specs.json

to run zed mono rect depth.
To visualize, I ran rviz2 -d $(ros2 pkg prefix isaac_ros_foundationpose --share)/rviz/foundationpose_zed.rviz, to visualize

The correct feed shows up but somehow when it detects my tumblr (even though I am running Mac and cheese?) the program shuts down by saying below

Screenshot from 2024-10-30 15-31-25

What would be the solution to this problem?

Also, do we have to only use DETR object detection? What if we want to use yolo model or other object detection models, how would I go about this?

Thank you so much for your help!

Dominant language
C++
Stars
515
Forks
57
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation

All issues in NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.