BUG: CTF writer keeps only four decimal places of every float column

オープン 初心者向け
#65 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
2/5
見積もり時間
1〜3時間
初心者へのやさしさ
82/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
cpp
領域
backend

調査の方向性

Source/EbsdLib/IO/HKL/CtfReader.cppで、1071~1075行付近にあるCtfReader::writeFile()のfloat列の書式設定を調べてください。まずissueに記載された再現手順を実行し、次に提案されているラウンドトリップテストについてSource/Test/CtfReaderTest.cppを確認してください。Euler1、Euler2、Xの値が保存時と完全に同じ値として読み戻されれば完了です。

索引モデルが issue の本文から書いたものです。

説明

bug

Problem

CtfReader::writeFile() writes every floating-point column of a .ctf file with four decimal places. Values that need more digits change without an error: an Euler angle or X position of 1e-5 is written as 0.0000, and 12.3456789 is written as 12.3457. Reading the written file back gives different data than the reader held. CtfReader itself reads float columns with std::stof, so it would accept the full value.

Expected: a written float column reads back to the same float. Actual: values are rounded to four decimals, and values below 5e-5 become zero.

Occurs at 34844ed31dfb8660c67edb6e019ca708e04beeb4 (develop).

Steps to Reproduce

  1. Read a .ctf file with CtfReader (the test below uses the test fixture CtfReaderTest::USInputFile1).
  2. Set the first Euler1 value to 1.0e-5f, the first Euler2 value to 12.3456789f and the first X value to 1.0e-5f.
  3. Call writeFile() to a new path.
  4. Read that file with a new CtfReader and compare the three values with the stored floats.

Cause

The data rows are written with a fixed four-decimal format for every float column (CtfReader.cpp#L1071-L1075):

else if(1 == colInfo.aType) // float pointer
{
  float* f32Ptr = reinterpret_cast<float*>(colInfo.ptr);
  fprintf(f, "%0.4f\t", f32Ptr[counter]);
}

%0.4f keeps four digits after the decimal point, so the written text cannot hold a float that needs more. A float needs up to 9 significant digits to read back unchanged.

Verification Unit Test

File: Source/Test/CtfReaderTest.cpp. The test sets the original header from the fixture file before writing, so the written file can be read back (see Notes).

TEST_CASE("ebsdlib::CtfReaderTest::WriteFilePreservesFloatPrecision", "[EbsdLib][CtfReaderTest]")
{
  CtfReader reader;
  reader.setFileName(ebsdlib::unit_test::CtfReaderTest::USInputFile1);
  REQUIRE(reader.readFile() == 0);
  REQUIRE(reader.getNumberOfElements() > 0);

  // Preserve fixture header line breaks to isolate float precision from header reconstruction.
  std::ifstream input(ebsdlib::unit_test::CtfReaderTest::USInputFile1);
  REQUIRE(input.is_open());
  std::string header;
  std::string line;
  bool foundDataHeader = false;
  while(std::getline(input, line))
  {
    header += line + "\n";
    if(line.find("Phase\tX\tY\t") == 0)
    {
      foundDataHeader = true;
      break;
    }
  }
  REQUIRE(foundDataHeader);
  reader.setOriginalHeader(header);

  auto* euler1Ptr = static_cast<float*>(reader.getPointerByName(ebsdlib::Ctf::Euler1));
  auto* euler2Ptr = static_cast<float*>(reader.getPointerByName(ebsdlib::Ctf::Euler2));
  auto* xPtr = static_cast<float*>(reader.getPointerByName(ebsdlib::Ctf::X));
  REQUIRE(euler1Ptr != nullptr);
  REQUIRE(euler2Ptr != nullptr);
  REQUIRE(xPtr != nullptr);

  euler1Ptr[0] = 1.0e-5f;
  euler2Ptr[0] = 12.3456789f;
  xPtr[0] = 1.0e-5f;
  const float storedEuler1 = euler1Ptr[0];
  const float storedEuler2 = euler2Ptr[0];
  const float storedX = xPtr[0];

  std::stringstream filePath;
  filePath << ebsdlib::unit_test::k_TestTempDir << "/CTF_WriteFile_float_precision_test.ctf";
  REQUIRE(reader.writeFile(filePath.str()) == 0);

  CtfReader readBack;
  readBack.setFileName(filePath.str());
  REQUIRE(readBack.readFile() == 0);
  REQUIRE(readBack.getNumberOfElements() == reader.getNumberOfElements());

  const auto* actualEuler1Ptr = static_cast<const float*>(readBack.getPointerByName(ebsdlib::Ctf::Euler1));
  const auto* actualEuler2Ptr = static_cast<const float*>(readBack.getPointerByName(ebsdlib::Ctf::Euler2));
  const auto* actualXPtr = static_cast<const float*>(readBack.getPointerByName(ebsdlib::Ctf::X));
  REQUIRE(actualEuler1Ptr != nullptr);
  REQUIRE(actualEuler2Ptr != nullptr);
  REQUIRE(actualXPtr != nullptr);

  SECTION("Euler1")
  {
    REQUIRE(actualEuler1Ptr[0] == storedEuler1);
  }
  SECTION("Euler2")
  {
    REQUIRE(actualEuler2Ptr[0] == storedEuler2);
  }
  SECTION("X")
  {
    REQUIRE(actualXPtr[0] == storedX);
  }
}

Command:

ctest --test-dir build/Release -R '^EbsdLib::ebsdlib::CtfReaderTest::WriteFilePreservesFloatPrecision$' -j 1 --output-on-failure --timeout 1500 --no-tests=error

Output on 34844ed (Release, macOS arm64):

Source/Test/CtfReaderTest.cpp:339: FAILED:
  REQUIRE( actualEuler1Ptr[0] == storedEuler1 )
with expansion:
  0.0f == 0.00001f
Source/Test/CtfReaderTest.cpp:343: FAILED:
  REQUIRE( actualEuler2Ptr[0] == storedEuler2 )
with expansion:
  12.3457f == 12.34568f
Source/Test/CtfReaderTest.cpp:347: FAILED:
  REQUIRE( actualXPtr[0] == storedX )
with expansion:
  0.0f == 0.00001f
test cases:  1 |  0 passed | 1 failed
0% tests passed, 1 tests failed out of 1

Proposed Fix

In CtfReader::writeFile(), write float columns with enough significant digits to read back unchanged, for example:

-            fprintf(f, "%0.4f\t", f32Ptr[counter]);
+            fprintf(f, "%.*g\t", std::numeric_limits<float>::max_digits10, static_cast<double>(f32Ptr[counter]));

(#include <limits>.) %g drops trailing zeros, so typical values such as 12.5 stay short. The test above then passes. If a consumer of .ctf files requires a fixed number of decimals per column, that requirement should be named in the code instead.

Notes

  • Found by an audit of floating-point text output across simplnx, DREAM3D-NX and EbsdLib. The same defect class in simplnx is tracked in BlueQuartzSoftware/simplnx#1927.
  • To read the written file back at all, the test restores the header line breaks with setOriginalHeader(). Without that, writeFile() writes the original header without line separators (getHeaderLines() through appendOriginalHeader()), and the new reader throws stoi: no conversion. That is a separate defect and is not part of this issue.
主要言語
C++
スター
8
フォーク
16
平均マージ
4時間 35分
マージ済み PR(30日)
3

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

BlueQuartzSoftware/EbsdLib のほかの issue

BlueQuartzSoftware/EbsdLib の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。