BUG: HDF5::FileIO::WriteFile removes the existing file before it creates the new file

オープン
#1,855 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 6 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
48/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
cpp
領域
backend, database

調査の方向性

Start with FileIO::WriteFile in src/simplnx/Utilities/Parsing/HDF5/IO/FileIO.cpp and its contract in FileIO.hpp; inspect the existing DelayVfd setup and test/H5IOTest.cpp. Build simplnx_test and run the named test to confirm the failure. Done means a failed WriteFile preserves the original file and the header documents the resulting behavior; the proposed replacement strategy still needs evaluation.

索引モデルが issue の本文から書いたものです。

説明

bug

Problem

HDF5::FileIO::WriteFile deletes the existing file first. Then it creates the new HDF5 file. If the creation fails, the original file is already gone. The caller gets an invalid FileIO and has lost the user's earlier data.

Expected: when WriteFile fails, the existing file is not changed.

Actual: when H5Fcreate fails, the existing file does not exist any more.

Found at commit 01809c236527ba812055cd6ab46e142317670c77.

Steps to Reproduce

  1. Build simplnx_test at commit 01809c236527ba812055cd6ab46e142317670c77.
  2. Create a file write_file_create_failure.dream3d that has some content.
  3. Install an HDF5 file access property list configurator (HDF5::FileIO::SetFaplConfigurator) that sets a read-only virtual file driver. The test suite already has such a driver (DelayVfd). With this driver, H5Fcreate fails.
  4. Call HDF5::FileIO::WriteFile(<that path>). It returns an invalid FileIO.
  5. Check the original file. It does not exist.

In a real run, other failures after the removal give the same result: a full disk, a read-only file system, or a path that the narrow string conversion in the H5Fcreate call cannot represent (see Notes).

Cause

WriteFile calls std::filesystem::remove on the target. Only then does it call H5Fcreate. There is no temporary file and no rename. If any step after the removal fails, nothing restores the old file.

https://github.com/BlueQuartzSoftware/simplnx/blob/01809c236527ba812055cd6ab46e142317670c77/src/simplnx/Utilities/Parsing/HDF5/IO/FileIO.cpp#L84-L111

FileIO FileIO::WriteFile(const std::filesystem::path& filepath)
{
  // Filesystem calls do not use HDF5. Keep them outside the HDF5 critical section.
  if(std::filesystem::exists(filepath))
  {
    try
    {
      std::filesystem::remove(filepath);
    } catch(const std::exception& e)
    {
      ...
    }
  }

  hid_t fileId = H5I_INVALID_HID;
  {
    std::lock_guard<std::mutex> hdf5Lock(Support::ApiLock());
    hid_t faplId = MakeFaplId();
    fileId = H5Fcreate(filepath.string().c_str(), H5F_ACC_TRUNC, H5P_DEFAULT, faplId);
    CloseFaplId(faplId);
  }
  ...

The header documents this behavior as a limit (https://github.com/BlueQuartzSoftware/simplnx/blob/01809c236527ba812055cd6ab46e142317670c77/src/simplnx/Utilities/Parsing/HDF5/IO/FileIO.hpp#L38-L45):

Filesystem removal occurs before HDF5 creation and is not recoverable through this API.

The removal is also not needed. H5Fcreate with H5F_ACC_TRUNC replaces an existing file by itself.

Verification Unit Test

File: test/H5IOTest.cpp. Add these includes at the top of the file, in the matching include groups. First, before the other includes:

#include "DelayVfd.hpp"

Then, among the existing includes:

#include "simplnx/unit_test/simplnx_test_dirs.hpp"

#include <H5Ppublic.h>

#include <fstream>

Add this test at the end of the file:

TEST_CASE("HDF5 FileIO::WriteFile keeps the existing file when HDF5 creation fails", "[simplnx][HDF5][FileIO]")
{
  const std::filesystem::path filePath = std::filesystem::path(unit_test::k_BinaryTestOutputDir.view()) / "write_file_create_failure.dream3d";
  const std::string originalContent = "original file content that must survive a failed replacement";
  {
    std::ofstream original(filePath, std::ios::binary | std::ios::trunc);
    REQUIRE(original.is_open());
    original << originalContent;
  }
  REQUIRE(std::filesystem::exists(filePath));

  // Fault injection: the test's delay driver is read-only, so H5Fcreate fails
  // deterministically. This stands in for any failure after FileIO::WriteFile has
  // started (H5Fcreate error, disk full, or a path the narrow conversion cannot represent).
  const hid_t driverId = DelayVfd::Register();
  REQUIRE(driverId != H5I_INVALID_HID);
  struct ClearFaplConfigurator
  {
    ~ClearFaplConfigurator()
    {
      HDF5::FileIO::SetFaplConfigurator({});
    }
  } clearFaplConfigurator;
  HDF5::FileIO::SetFaplConfigurator([driverId](hid_t faplId) { H5Pset_driver(faplId, driverId, nullptr); });

  bool writerValid = true;
  {
    auto fileWriter = HDF5::FileIO::WriteFile(filePath);
    writerValid = fileWriter.isValid();
  }
  HDF5::FileIO::SetFaplConfigurator({});
  REQUIRE_FALSE(writerValid); // the injected creation failure took effect

  // Expected: a failed WriteFile leaves the existing file untouched.
  REQUIRE(std::filesystem::exists(filePath));
  std::string content;
  {
    std::ifstream reread(filePath, std::ios::binary);
    content.assign(std::istreambuf_iterator<char>(reread), std::istreambuf_iterator<char>());
  }
  REQUIRE(content == originalContent);
}

The test uses the FileIO::SetFaplConfigurator hook and the test driver DelayVfd that already exist in the repository. The driver is read-only, so H5Fcreate fails in a deterministic way. This stands in for any failure after the removal.

Build and run only this test. Use the build directory of a configured simplnx build tree.

cmake --build <build-dir> -j --target simplnx_test
cd <build-dir> && ctest -R "^HDF5 FileIO::WriteFile keeps the existing file when HDF5 creation fails$" --output-on-failure

Output at commit 01809c236527ba812055cd6ab46e142317670c77:

test/H5IOTest.cpp:437: FAILED:
  REQUIRE( std::filesystem::exists(filePath) )
with expansion:
  false

test cases: 1 | 1 failed
assertions: 5 | 4 passed | 1 failed

The four assertions that pass confirm that the creation failure took effect (writerValid is false).

Proposed Fix

Create the new file at a temporary path in the same directory. Replace the target only when the creation succeeded.

  1. In FileIO::WriteFile, do not remove the target at the start. Create the HDF5 file at <filepath>.tmp (or a unique name in the same directory). The caller can write to the returned object as before.
  2. Replace the target with the temporary file after the file is closed and complete. This is a change of the FileIO lifetime contract. Decide where the rename happens, for example in the FileIO destructor or close(). Check the callers that keep the object open for a long time.
  3. A simpler option: do not call std::filesystem::remove. Call H5Fcreate with H5F_ACC_TRUNC on the target. This option was not tested. It does not protect against a failure in the middle of a write. Check it against the test first, because HDF5 can also truncate the old file before it reports a failure.
  4. Update the Doxygen comment on WriteFile to match the new behavior.
  5. Use the std::filesystem::path native string for the HDF5 call on Windows (a wide-character path function or UTF-8 conversion), so that the narrow .string() conversion cannot fail for a valid path.

Notes

  • Windows (inferred, not reproduced): filepath.string() converts the path to the active code page. For a file name that the code page cannot represent, the conversion can throw or give a wrong name. This is a second failure point after the removal. This was not run on Windows. The test above uses the read-only driver as a stand-in.
  • The failure paths that were verified: H5Fcreate failure through a read-only file driver (macOS).
  • FileIO::WriteFile is used by DREAM3D::WriteFile and by DREAM3D::WriteRecoveryFile. A failed save currently loses the previous file at the same path.
主要言語
C++
スター
18
フォーク
15
平均マージ
4日 23時間
マージ済み PR(30日)
42

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

BlueQuartzSoftware/simplnx のほかの issue

BlueQuartzSoftware/simplnx の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。