[visualization] PCLVisualizer::addCorrespondences/updateCorrespondences does not show all correspondences

未关闭
#5,749 9 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 3 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
38/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
停滞
技术栈
cpp

调研方向

从 mwe.cpp 复现程序以及 visualization/include/pcl/visualization/impl/pcl_visualizer.hpp 中的 addCorrespondences()/updateCorrespondences() 实现开始,包括第 1259 行附近被引用的区域。将其渲染行为与正常工作的 addLine() 循环进行比较,并调查为什么并非每个迭代到的对应关系都会变得可见;当所有对应关系都通过对应关系 API 显示出来时即视为完成。

由索引模型根据 Issue 内容生成。

描述

kind: bug module: visualization

Describe the bug

I use Registration::registerVisualizationCallback() to show correspondences. The code employs PCLVisualizer::addCorrespondences(). This does not lead to all correspondences getting shown. If instead I iterate over all correspondences manually and addLine() for each, all are shown as expected. Somewhere inside PCLVisualizer, items get lost.

Expected behavior

All correspondences passed to addCorrespondences() function should be shown.

Current Behavior

Not all correspondences passed to addCorrespondences() function are shown.

To Reproduce

mwe.cpp

#include <pcl/io/pcd_io.h>
#include <pcl/registration/icp.h>

#include <pcl/visualization/pcl_visualizer.h>

using namespace std;
using namespace pcl;
using namespace pcl::visualization;

using PointT = PointXYZ;
using CloudT = PointCloud<PointT>;
using namespace Eigen;

struct RegistrationVisualizer {

  pcl::visualization::PCLVisualizer::Ptr optimization_viewer;

  RegistrationVisualizer();

  void visualize_registration(pcl::Registration<PointT, PointT> &reg);
};

int main(int argc, char *argv[]) {
  Matrix4f transformation;
  transformation << 1, 0, 0, 6.7, 0, 1, 0, 12.3, 0, 0, 1, 0.7, 0, 0, 0, 1;

  CloudT::Ptr source{new CloudT};
  CloudT::Ptr target{new CloudT};
  io::loadPCDFile("../scene_filtered.pcd", *target);
  io::loadPCDFile("../template.pcd", *source);

  IterativeClosestPoint<PointT, PointT> icp;
  icp.setInputSource(source);
  icp.setInputTarget(target);
  icp.setMaxCorrespondenceDistance(0.1);
  RegistrationVisualizer regvis;
  regvis.visualize_registration(icp);

  icp.align(*source, transformation);
  return 0;
}

RegistrationVisualizer::RegistrationVisualizer() {
  optimization_viewer = pcl::make_shared<pcl::visualization::PCLVisualizer>();
  optimization_viewer->setWindowName("Correspondences");
}

void RegistrationVisualizer::visualize_registration(
    pcl::Registration<PointT, PointT> &reg) {
  using namespace pcl;
  using namespace pcl::visualization;

  std::function<
      pcl::Registration<PointT, PointT>::UpdateVisualizerCallbackSignature>
      callback([this](const CloudT &c1, const pcl::Indices &idxs1,
                      const CloudT &c2, const pcl::Indices &idxs2) {
        if (!idxs1.size() || !idxs2.size()) {
          return;
        }

        // turn idxs into one correspondence vector
        Correspondences correspondences;
        for (size_t i = 0; i < idxs1.size(); ++i) {
          correspondences.push_back(Correspondence(idxs1[i], idxs2[i], 1));
        }

        // viewer needs ptrs, not values (expensiveish)
        const CloudT::Ptr c1_correspondences = c1.makeShared();
        const CloudT::Ptr c2_correspondences = c2.makeShared();

        if (!this->optimization_viewer->updatePointCloud<PointT>(
                c1_correspondences, "c1")) {
          this->optimization_viewer->addPointCloud<PointT>(c1_correspondences,
                                                           "c1");
          this->optimization_viewer->setPointCloudRenderingProperties(
              PCL_VISUALIZER_COLOR, 0, 1, 1, "c1");
          this->optimization_viewer->setPointCloudRenderingProperties(
              PCL_VISUALIZER_POINT_SIZE, 3, "c1");
        }
        if (!this->optimization_viewer->updateText("Source", 10, 10)) {
          this->optimization_viewer->addText("Source", 10, 10, 20, 0, 1, 1);
        }
        if (!this->optimization_viewer->updatePointCloud<PointT>(
                c2_correspondences, "c2")) {
          this->optimization_viewer->addPointCloud<PointT>(c2_correspondences,
                                                           "c2");
          this->optimization_viewer->setPointCloudRenderingProperties(
              PCL_VISUALIZER_COLOR, 1, 1, 0, "c2");
          this->optimization_viewer->setPointCloudRenderingProperties(
              PCL_VISUALIZER_POINT_SIZE, 3, "c2");
        }
        if (!this->optimization_viewer->updateText("Target", 10, 40)) {
          this->optimization_viewer->addText("Target", 10, 40, 20, 1, 1, 0);
        }

        // works as expected
        /* int c = 0; */
        /* for (const auto& corr : correspondences) { */
        /*   const auto p1     = c1_correspondences->points[corr.index_query]; */
        /*   const auto p2     = c2_correspondences->points[corr.index_match]; */
        /*   const string name = "line" + to_string(c); */
        /*   this->optimization_viewer->removeShape(name); */
        /*   this->optimization_viewer->addLine(p1, p2, 1, 0, 0, name); */
        /*   c++; */
        /* } */

        // many lines missing
        if (!this->optimization_viewer->updateCorrespondences<PointT>(
                c1_correspondences, c2_correspondences, correspondences)) {
          this->optimization_viewer->addCorrespondences<PointT>(
              c1_correspondences, c2_correspondences, correspondences,
              "correspondences");
        }
        this->optimization_viewer->setShapeRenderingProperties(
            PCL_VISUALIZER_LINE_WIDTH, 5, "correspondences");
        this->optimization_viewer->spin();
      });
  reg.registerVisualizationCallback(callback);
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.19.8)
set(CMAKE_CXX_STANDARD 20)
find_package(PCL REQUIRED)
add_executable(mwe mwe.cpp)
target_link_libraries(mwe ${PCL_LIBRARIES})
target_include_directories(mwe PRIVATE ${PCL_INCLUDE_DIRS})

pcds:

pcds.zip

First screenshot is using updateCorrespondences(). Notice how on the right there are no correspondences.

Screenshot 2023-06-18 at 20 14 55

Second screenshot I add them by hand with addLine(). Many correspondences on the right.

Screenshot 2023-06-18 at 20 15 18

Your Environment (please complete the following information):

  • OS: macos 12.6.5
  • Compiler: Apple clang version 14.0.0 (clang-1400.0.29.202) (arm)
  • PCL Version 1.13 (4d8f60706c5d97c4a5b3b3da65f107cc7716ce1e)

I'm at a loss here. I don't know VTK, but the code in addCorrespondences() does not look wrong. I could verify that in https://github.com/PointCloudLibrary/pcl/blob/master/visualization/include/pcl/visualization/impl/pcl_visualizer.hpp#L1259, the full number of found correspondences is iterated, but somehow not everything shows up.
This led me on a merry chase as I had first assumed the correspondence estimation to somehow fail, but it seems to be just the visualization.

主要语言
C++
星标
11.2k
派生
4.7k
平均合并
6 天 13 小时
30 天内合并 PR
10

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

PointCloudLibrary/pcl 的其他 Issue

查看 PointCloudLibrary/pcl 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。