FoundationPose with custom object
@yuanknv is already working on this.
Since May 20, 2025.
Assessment
This issue has not been assessed yet.
Description
For performing pose estimation on a novel object I understand that I need -
-
A object detection model trained on the new object. For this I used : rtdetr_r101vd which seems to be the same one that the sytheticaDETR in the quickstart is using. This was then converted to engine plan model.
/usr/src/tensorrt/bin/trtexec --onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/custom/custom.onnx --saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/custom/custom.plan -
A .obj file for the object. I dont have a uv texture map for the object so I removed that argument from the ros2 launch command.
ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono_rect_depth,foundationpose mesh_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/custom/custom.obj score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan rt_detr_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/rtdetr/custom.plan
However I was not able to pose estimate on the new object. I am using realsense 435 for the project. I also tried detection alone using isaac ros rtdetr tutorial which also seems to be not working.
I had tested the onnx model seperately and it works. I guess this must be something to do with converting to tensorRT.
- Dominant language
- C++
- Stars
- 515
- Forks
- 57
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
License clarification — vendored nvdiffrast sources in isaac_ros_pose_estimation (Apache-2.0)May be free again @yuanknv claimed this 43 days ago, and no pull request is open. Openverify to close
NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation#98 · 2 comments · 2 assignees ·
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
Not getting the same result as offline foundationposeMay be free again @yuanknv claimed this 170 days ago, and no pull request is open. Opencoming soon!
NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation#95 · 8 comments · 1 assignee ·
All issues in NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation
Similar issues
-
launch-client --save-creds creates creds/<account>/ with mode 755, not 700Possibly taken @metis-sprock claimed this today. Openbacklog bug
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
EchoTools/nevr-runtime#1471 ·
Maintainers usually reply within 1 day
-
.NET ep:WebGPU platform:web
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
microsoft/onnxruntime#33755 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
grumpycoders/pcsx-redux#2530 ·
-
A-tools P-medium S-ready-to-start studio task
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Status: Needs confirmation Status: Needs triage Type: Code Quality
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day