[visualization] PCLVisualizer::addCorrespondences/updateCorrespondences does not show all correspondences

オープン
#5,749 コメント 9 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 3 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
38/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
停滞
技術スタック
cpp

調査の方向性

mwe.cpp の再現プログラムと、visualization/include/pcl/visualization/impl/pcl_visualizer.hpp にある addCorrespondences()/updateCorrespondences() の実装から始め、1259 行付近の参照されている領域も確認してください。そのレンダリング動作を、動作している addLine() ループと比較し、反復処理されたすべての対応が表示可能になるわけではない理由を調査してください。対応付け API を通じてすべての対応が表示されれば完了です。

索引モデルが issue の本文から書いたものです。

説明

kind: bug module: visualization

Describe the bug

I use Registration::registerVisualizationCallback() to show correspondences. The code employs PCLVisualizer::addCorrespondences(). This does not lead to all correspondences getting shown. If instead I iterate over all correspondences manually and addLine() for each, all are shown as expected. Somewhere inside PCLVisualizer, items get lost.

Expected behavior

All correspondences passed to addCorrespondences() function should be shown.

Current Behavior

Not all correspondences passed to addCorrespondences() function are shown.

To Reproduce

mwe.cpp

#include <pcl/io/pcd_io.h>
#include <pcl/registration/icp.h>

#include <pcl/visualization/pcl_visualizer.h>

using namespace std;
using namespace pcl;
using namespace pcl::visualization;

using PointT = PointXYZ;
using CloudT = PointCloud<PointT>;
using namespace Eigen;

struct RegistrationVisualizer {

  pcl::visualization::PCLVisualizer::Ptr optimization_viewer;

  RegistrationVisualizer();

  void visualize_registration(pcl::Registration<PointT, PointT> &reg);
};

int main(int argc, char *argv[]) {
  Matrix4f transformation;
  transformation << 1, 0, 0, 6.7, 0, 1, 0, 12.3, 0, 0, 1, 0.7, 0, 0, 0, 1;

  CloudT::Ptr source{new CloudT};
  CloudT::Ptr target{new CloudT};
  io::loadPCDFile("../scene_filtered.pcd", *target);
  io::loadPCDFile("../template.pcd", *source);

  IterativeClosestPoint<PointT, PointT> icp;
  icp.setInputSource(source);
  icp.setInputTarget(target);
  icp.setMaxCorrespondenceDistance(0.1);
  RegistrationVisualizer regvis;
  regvis.visualize_registration(icp);

  icp.align(*source, transformation);
  return 0;
}

RegistrationVisualizer::RegistrationVisualizer() {
  optimization_viewer = pcl::make_shared<pcl::visualization::PCLVisualizer>();
  optimization_viewer->setWindowName("Correspondences");
}

void RegistrationVisualizer::visualize_registration(
    pcl::Registration<PointT, PointT> &reg) {
  using namespace pcl;
  using namespace pcl::visualization;

  std::function<
      pcl::Registration<PointT, PointT>::UpdateVisualizerCallbackSignature>
      callback([this](const CloudT &c1, const pcl::Indices &idxs1,
                      const CloudT &c2, const pcl::Indices &idxs2) {
        if (!idxs1.size() || !idxs2.size()) {
          return;
        }

        // turn idxs into one correspondence vector
        Correspondences correspondences;
        for (size_t i = 0; i < idxs1.size(); ++i) {
          correspondences.push_back(Correspondence(idxs1[i], idxs2[i], 1));
        }

        // viewer needs ptrs, not values (expensiveish)
        const CloudT::Ptr c1_correspondences = c1.makeShared();
        const CloudT::Ptr c2_correspondences = c2.makeShared();

        if (!this->optimization_viewer->updatePointCloud<PointT>(
                c1_correspondences, "c1")) {
          this->optimization_viewer->addPointCloud<PointT>(c1_correspondences,
                                                           "c1");
          this->optimization_viewer->setPointCloudRenderingProperties(
              PCL_VISUALIZER_COLOR, 0, 1, 1, "c1");
          this->optimization_viewer->setPointCloudRenderingProperties(
              PCL_VISUALIZER_POINT_SIZE, 3, "c1");
        }
        if (!this->optimization_viewer->updateText("Source", 10, 10)) {
          this->optimization_viewer->addText("Source", 10, 10, 20, 0, 1, 1);
        }
        if (!this->optimization_viewer->updatePointCloud<PointT>(
                c2_correspondences, "c2")) {
          this->optimization_viewer->addPointCloud<PointT>(c2_correspondences,
                                                           "c2");
          this->optimization_viewer->setPointCloudRenderingProperties(
              PCL_VISUALIZER_COLOR, 1, 1, 0, "c2");
          this->optimization_viewer->setPointCloudRenderingProperties(
              PCL_VISUALIZER_POINT_SIZE, 3, "c2");
        }
        if (!this->optimization_viewer->updateText("Target", 10, 40)) {
          this->optimization_viewer->addText("Target", 10, 40, 20, 1, 1, 0);
        }

        // works as expected
        /* int c = 0; */
        /* for (const auto& corr : correspondences) { */
        /*   const auto p1     = c1_correspondences->points[corr.index_query]; */
        /*   const auto p2     = c2_correspondences->points[corr.index_match]; */
        /*   const string name = "line" + to_string(c); */
        /*   this->optimization_viewer->removeShape(name); */
        /*   this->optimization_viewer->addLine(p1, p2, 1, 0, 0, name); */
        /*   c++; */
        /* } */

        // many lines missing
        if (!this->optimization_viewer->updateCorrespondences<PointT>(
                c1_correspondences, c2_correspondences, correspondences)) {
          this->optimization_viewer->addCorrespondences<PointT>(
              c1_correspondences, c2_correspondences, correspondences,
              "correspondences");
        }
        this->optimization_viewer->setShapeRenderingProperties(
            PCL_VISUALIZER_LINE_WIDTH, 5, "correspondences");
        this->optimization_viewer->spin();
      });
  reg.registerVisualizationCallback(callback);
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.19.8)
set(CMAKE_CXX_STANDARD 20)
find_package(PCL REQUIRED)
add_executable(mwe mwe.cpp)
target_link_libraries(mwe ${PCL_LIBRARIES})
target_include_directories(mwe PRIVATE ${PCL_INCLUDE_DIRS})

pcds:

pcds.zip

First screenshot is using updateCorrespondences(). Notice how on the right there are no correspondences.

Screenshot 2023-06-18 at 20 14 55

Second screenshot I add them by hand with addLine(). Many correspondences on the right.

Screenshot 2023-06-18 at 20 15 18

Your Environment (please complete the following information):

  • OS: macos 12.6.5
  • Compiler: Apple clang version 14.0.0 (clang-1400.0.29.202) (arm)
  • PCL Version 1.13 (4d8f60706c5d97c4a5b3b3da65f107cc7716ce1e)

I'm at a loss here. I don't know VTK, but the code in addCorrespondences() does not look wrong. I could verify that in https://github.com/PointCloudLibrary/pcl/blob/master/visualization/include/pcl/visualization/impl/pcl_visualizer.hpp#L1259, the full number of found correspondences is iterated, but somehow not everything shows up.
This led me on a merry chase as I had first assumed the correspondence estimation to somehow fail, but it seems to be just the visualization.

主要言語
C++
スター
11.2k
フォーク
4.7k
平均マージ
6日 13時間
マージ済み PR(30日)
10

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

PointCloudLibrary/pcl のほかの issue

PointCloudLibrary/pcl の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。