BUG: HDF5::FileIO::WriteFile removes the existing file before it creates the new file

Đang mở
#1,855 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 6 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
48/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
cpp
Lĩnh vực
backend, database

Hướng nghiên cứu

Start with FileIO::WriteFile in src/simplnx/Utilities/Parsing/HDF5/IO/FileIO.cpp and its contract in FileIO.hpp; inspect the existing DelayVfd setup and test/H5IOTest.cpp. Build simplnx_test and run the named test to confirm the failure. Done means a failed WriteFile preserves the original file and the header documents the resulting behavior; the proposed replacement strategy still needs evaluation.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

bug

Problem

HDF5::FileIO::WriteFile deletes the existing file first. Then it creates the new HDF5 file. If the creation fails, the original file is already gone. The caller gets an invalid FileIO and has lost the user's earlier data.

Expected: when WriteFile fails, the existing file is not changed.

Actual: when H5Fcreate fails, the existing file does not exist any more.

Found at commit 01809c236527ba812055cd6ab46e142317670c77.

Steps to Reproduce

  1. Build simplnx_test at commit 01809c236527ba812055cd6ab46e142317670c77.
  2. Create a file write_file_create_failure.dream3d that has some content.
  3. Install an HDF5 file access property list configurator (HDF5::FileIO::SetFaplConfigurator) that sets a read-only virtual file driver. The test suite already has such a driver (DelayVfd). With this driver, H5Fcreate fails.
  4. Call HDF5::FileIO::WriteFile(<that path>). It returns an invalid FileIO.
  5. Check the original file. It does not exist.

In a real run, other failures after the removal give the same result: a full disk, a read-only file system, or a path that the narrow string conversion in the H5Fcreate call cannot represent (see Notes).

Cause

WriteFile calls std::filesystem::remove on the target. Only then does it call H5Fcreate. There is no temporary file and no rename. If any step after the removal fails, nothing restores the old file.

https://github.com/BlueQuartzSoftware/simplnx/blob/01809c236527ba812055cd6ab46e142317670c77/src/simplnx/Utilities/Parsing/HDF5/IO/FileIO.cpp#L84-L111

FileIO FileIO::WriteFile(const std::filesystem::path& filepath)
{
  // Filesystem calls do not use HDF5. Keep them outside the HDF5 critical section.
  if(std::filesystem::exists(filepath))
  {
    try
    {
      std::filesystem::remove(filepath);
    } catch(const std::exception& e)
    {
      ...
    }
  }

  hid_t fileId = H5I_INVALID_HID;
  {
    std::lock_guard<std::mutex> hdf5Lock(Support::ApiLock());
    hid_t faplId = MakeFaplId();
    fileId = H5Fcreate(filepath.string().c_str(), H5F_ACC_TRUNC, H5P_DEFAULT, faplId);
    CloseFaplId(faplId);
  }
  ...

The header documents this behavior as a limit (https://github.com/BlueQuartzSoftware/simplnx/blob/01809c236527ba812055cd6ab46e142317670c77/src/simplnx/Utilities/Parsing/HDF5/IO/FileIO.hpp#L38-L45):

Filesystem removal occurs before HDF5 creation and is not recoverable through this API.

The removal is also not needed. H5Fcreate with H5F_ACC_TRUNC replaces an existing file by itself.

Verification Unit Test

File: test/H5IOTest.cpp. Add these includes at the top of the file, in the matching include groups. First, before the other includes:

#include "DelayVfd.hpp"

Then, among the existing includes:

#include "simplnx/unit_test/simplnx_test_dirs.hpp"

#include <H5Ppublic.h>

#include <fstream>

Add this test at the end of the file:

TEST_CASE("HDF5 FileIO::WriteFile keeps the existing file when HDF5 creation fails", "[simplnx][HDF5][FileIO]")
{
  const std::filesystem::path filePath = std::filesystem::path(unit_test::k_BinaryTestOutputDir.view()) / "write_file_create_failure.dream3d";
  const std::string originalContent = "original file content that must survive a failed replacement";
  {
    std::ofstream original(filePath, std::ios::binary | std::ios::trunc);
    REQUIRE(original.is_open());
    original << originalContent;
  }
  REQUIRE(std::filesystem::exists(filePath));

  // Fault injection: the test's delay driver is read-only, so H5Fcreate fails
  // deterministically. This stands in for any failure after FileIO::WriteFile has
  // started (H5Fcreate error, disk full, or a path the narrow conversion cannot represent).
  const hid_t driverId = DelayVfd::Register();
  REQUIRE(driverId != H5I_INVALID_HID);
  struct ClearFaplConfigurator
  {
    ~ClearFaplConfigurator()
    {
      HDF5::FileIO::SetFaplConfigurator({});
    }
  } clearFaplConfigurator;
  HDF5::FileIO::SetFaplConfigurator([driverId](hid_t faplId) { H5Pset_driver(faplId, driverId, nullptr); });

  bool writerValid = true;
  {
    auto fileWriter = HDF5::FileIO::WriteFile(filePath);
    writerValid = fileWriter.isValid();
  }
  HDF5::FileIO::SetFaplConfigurator({});
  REQUIRE_FALSE(writerValid); // the injected creation failure took effect

  // Expected: a failed WriteFile leaves the existing file untouched.
  REQUIRE(std::filesystem::exists(filePath));
  std::string content;
  {
    std::ifstream reread(filePath, std::ios::binary);
    content.assign(std::istreambuf_iterator<char>(reread), std::istreambuf_iterator<char>());
  }
  REQUIRE(content == originalContent);
}

The test uses the FileIO::SetFaplConfigurator hook and the test driver DelayVfd that already exist in the repository. The driver is read-only, so H5Fcreate fails in a deterministic way. This stands in for any failure after the removal.

Build and run only this test. Use the build directory of a configured simplnx build tree.

cmake --build <build-dir> -j --target simplnx_test
cd <build-dir> && ctest -R "^HDF5 FileIO::WriteFile keeps the existing file when HDF5 creation fails$" --output-on-failure

Output at commit 01809c236527ba812055cd6ab46e142317670c77:

test/H5IOTest.cpp:437: FAILED:
  REQUIRE( std::filesystem::exists(filePath) )
with expansion:
  false

test cases: 1 | 1 failed
assertions: 5 | 4 passed | 1 failed

The four assertions that pass confirm that the creation failure took effect (writerValid is false).

Proposed Fix

Create the new file at a temporary path in the same directory. Replace the target only when the creation succeeded.

  1. In FileIO::WriteFile, do not remove the target at the start. Create the HDF5 file at <filepath>.tmp (or a unique name in the same directory). The caller can write to the returned object as before.
  2. Replace the target with the temporary file after the file is closed and complete. This is a change of the FileIO lifetime contract. Decide where the rename happens, for example in the FileIO destructor or close(). Check the callers that keep the object open for a long time.
  3. A simpler option: do not call std::filesystem::remove. Call H5Fcreate with H5F_ACC_TRUNC on the target. This option was not tested. It does not protect against a failure in the middle of a write. Check it against the test first, because HDF5 can also truncate the old file before it reports a failure.
  4. Update the Doxygen comment on WriteFile to match the new behavior.
  5. Use the std::filesystem::path native string for the HDF5 call on Windows (a wide-character path function or UTF-8 conversion), so that the narrow .string() conversion cannot fail for a valid path.

Notes

  • Windows (inferred, not reproduced): filepath.string() converts the path to the active code page. For a file name that the code page cannot represent, the conversion can throw or give a wrong name. This is a second failure point after the removal. This was not run on Windows. The test above uses the read-only driver as a stand-in.
  • The failure paths that were verified: H5Fcreate failure through a read-only file driver (macOS).
  • FileIO::WriteFile is used by DREAM3D::WriteFile and by DREAM3D::WriteRecoveryFile. A failed save currently loses the previous file at the same path.
Ngôn ngữ chính
C++
Star
18
Fork
15
Merge trung bình
4 ngày 23 giờ
Pull request đã merge (30 ngày)
42

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của BlueQuartzSoftware/simplnx

Tất cả issue của BlueQuartzSoftware/simplnx

Issue tương tự

Thêm issue về C++

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.