BUG: Writers save floating-point data as text with too few digits
I maintainer di solito rispondono entro 6 giorni
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 35/100
Direzione di ricerca
Start with the listed writer locations, including WriteAvizoUniformCoordinate.cpp, WriteLAMMPSFile.cpp, WriteINLFile.cpp, ComputeGBCDMetricBased.cpp, and simplnx_utilities.py, then compare their formatting with the reported failing values. The issue describes a broad audit across many writers and output formats, so identify which cases remain unaddressed and check for format-specific precision requirements. Done means text output round-trips to the written floating-point type, except where user-selected or format-required precision applies.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Problem
Several writers save floating-point data as text with a format that drops digits. The output file then holds different values than the data structure, without an error:
- Fixed decimal places (
%f,{:f},{:.Nf},std::fixed) turn small nonzero values into zero. For example, an Image Geometry with spacing1e-7is written with bounds or coordinates0.000000. - Too few significant digits (default stream precision,
digits10, 9 digits for a float64) change the value that a reader parses back.
Expected: every floating-point value that a writer saves as text reads back to the same value of the type that was written. A float32 needs up to 9 significant digits, a float64 up to 17. Fmt {} writes the shortest text that reads back exactly. The exceptions are a precision that the user selects and a precision that a file-format specification fixes.
Occurs at aacedb4dc5c009d21045d715ac8f09a0d213d42e (develop). An audit of all simplnx writers found these places. "Code reading" means the place was confirmed by reading the code and computing the output for the input shown. The two places marked "test below" also have a unit test that was run and seen to fail.
Values become zero or lose all small digits:
| Filter | Function | Location | Values written | Format | Failing input → output; how confirmed |
|---|---|---|---|---|---|
generateHeader() |
WriteAvizoUniformCoordinate.cpp:69 | BoundingBox, in ASCII and binary files |
%f |
spacing 1e-7 → bounds 0.000000; test below |
|
writeData() (ASCII) |
WriteAvizoRectilinearCoordinate.cpp:163 | X/Y/Z coordinates | %f |
spacing 1e-7 → coordinates 0.000000; test below |
|
operator() |
WriteLAMMPSFile.cpp:123-125, WriteLAMMPSFile.cpp:146 | box bounds, atom coordinates | {:f} |
bound 1e-7 → 0.000000; code reading |
|
operator() |
WriteINLFile.cpp:192-202, WriteINLFile.cpp:232 | header steps/origin/bounds; Euler angles and positions in every row | std::fixed, precision 6 |
X_STEP 1e-7 → 0.000000; code reading |
|
WriteChunkedData(), WriteDirectData() |
WriteLosAlamosFFT.cpp:144, WriteLosAlamosFFT.cpp:218 | Euler angles (degrees) | {:.3f} |
1e-7 rad → 0.000; code reading |
|
operator() |
WriteGBCDTriangleData.cpp:95 | Euler angles, normals, face area | {:0.4f} |
area 1e-5 → 0.0000; code reading |
|
operator() |
WriteGBCDGMTFile.cpp:307, WriteGBCDGMTFile.cpp:313 | misorientation header; angles and intensity rows | %.1f, %f |
angle 0.04° → 0.0; intensity 1e-7 → 0.000000; code reading |
|
WriteEulerLine() |
WriteStatsGenOdfAngleFile.cpp:63 | Euler angles, weight, sigma | std::fixed, precision 8 |
Euler 1e-9 (radians kept) → 0.00000000; code reading |
|
WriteValue<T>() |
WriteNodesAndElementsFiles.cpp:28 | vertex coordinates | std::fixed, precision 4 |
vertex 1e-5 → 0.0000; code reading |
|
operator() |
ComputeGBCDMetricBased.cpp:654 | misorientation header of both output files | {:.1F} |
angle 0.04° → 0.0; code reading |
|
| Python pipeline code export | _encode_float() |
simplnx_utilities.py:107 | numeric parameters in generated Python | round() then fixed decimals |
origin 1e-13 → 0.0; code reading |
Values lose trailing significant digits:
| Filter | Function | Location | Values written | Format | Failing input → output; how confirmed |
|---|---|---|---|---|---|
Converter::operator() |
ConvertHexGridToSquareGrid.cpp:210 | .ang rows: Euler angles, X/Y, IQ, CI, SEM signal, fit |
default stream, 6 significant digits | 1.0000001192f → 1; code reading |
|
modifyAngHeaderLine() |
ConvertHexGridToSquareGrid.cpp:309, ConvertHexGridToSquareGrid.cpp:313 | XSTEP, YSTEP | StringUtilities::number() with digits10 (6) |
step 1.0000001192f → 1; code reading |
|
WriteGeomXdmf() (ImageGeom) |
Dream3dIO.cpp:175-177 | origin, spacing | default stream, 6 significant digits | spacing 1.0000001192f → 1; code reading |
|
writeNodes() |
WriteAbaqusHexahedron.cpp:103 | node coordinates computed in float64 | {:.9g} |
1 + 2^-24 → 1.00000006 (a different double); code reading |
|
WriteNodes() |
WriteAbaqusCrystalPlasticity.cpp:100 | node coordinates computed in float64 | {:.9g} |
same as above; code reading | |
| distribution and error outputs | ComputeGBCDMetricBased.cpp:667, ComputeGBCDMetricBased.cpp:672, ComputeGBCDMetricBased.cpp:682 | float64 angles, distribution, errors | {:1.8E} (9 significant digits) |
141.93951659864035 → 1.41939517E+02; code reading |
|
| distribution and error outputs | ComputeGBPDMetricBased.cpp:717, ComputeGBPDMetricBased.cpp:722, ComputeGBPDMetricBased.cpp:732 | float64 angles, distribution, errors | {:.8E} (9 significant digits) |
35.264389682754654 → 3.52643897E+01; code reading |
Already reported:
| Filter | Function | Location | Values written | Format | Report |
|---|---|---|---|---|---|
writeCoords() |
WriteVtkRectilinearGrid.cpp:133 | ASCII coordinates | %f |
#1908 item L29; PR #1915 changes it to max_digits10 |
|
TupleWriter) |
TupleWriter::write() |
OStreamUtilities.cpp:464, OStreamUtilities.cpp:468 | float32 / float64 values | precision 8 / 16 | #1867 item WriteASCIIDataFilter#3 |
The ImageProcessing developer benchmarks also write timing CSV columns with {:.3f} (BenchmarkRunner.cpp:202, BenchmarkRunner.cpp:209, WriteImageBenchmark.cpp:511-514). They are developer tools, not filter output.
Steps to Reproduce
- Create an Image Geometry with dimensions
3 x 2 x 1, origin(0, 0, 0)and spacing(1e-7, 2e-7, 3e-7), and an int32 Feature Ids cell array. - Run Write Avizo Uniform Coordinate with Write Binary File off, then on.
- Open the output files. Both headers contain
BoundingBox 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000. The expected first bound is5e-08. - Run Write Avizo Rectilinear Coordinate on the same geometry with Write Binary File off. The X coordinates line is
0.000000 0.000000 0.000000; the expected second value is1e-07.
Cause
Each place formats the value with a fixed number of decimal places or too few significant digits. For example, WriteAvizoUniformCoordinate.cpp#L69-L70:
fprintf(outputFile, " BoundingBox %f %f %f %f %f %f\n", origin[0] + 0.5 * res[0], origin[0] + (static_cast<float64>(dims[0]) - 0.5) * res[0], origin[1] + 0.5 * res[1],
origin[1] + (static_cast<float64>(dims[1]) - 0.5) * res[1], origin[2] + 0.5 * res[2], origin[2] + (static_cast<float64>(dims[2]) - 0.5) * res[2]);
and WriteAvizoRectilinearCoordinate.cpp#L163:
fprintf(outputFile, "%f ", origin[d] + (res[d] * i));
%f always prints six digits after the decimal point, so any value below 5e-7 is written as 0.000000. The truncating places use a precision below what the written type needs to round-trip: digits10 (6 for float32) is the decimal-to-binary guarantee, not max_digits10 (9). The Abaqus writers apply float32's 9 digits to coordinates that they compute in float64.
Verification Unit Test
1. The fixed instance: Write Avizo Uniform Coordinate. File src/Plugins/SimplnxCore/test/WriteAvizoUniformCoordinateTest.cpp. This test and the fix are being added to PR #1919.
TEST_CASE("SimplnxCore::WriteAvizoUniformCoordinateFilter: Tiny spacing bounds round trip", "[SimplnxCore][WriteAvizoUniformCoordinateFilter]")
{
const bool binary = GENERATE(false, true);
CAPTURE(binary);
DataStructure dataStructure;
const SizeVec3 dims = {3, 2, 1};
const DataPath labelsPath = CreateUniformWriterInput(dataStructure, dims, {101, 102, 103, 104, 105, 106});
auto* imageGeomPtr = dataStructure.getDataAs<ImageGeom>(DataPath({"Image"}));
REQUIRE(imageGeomPtr != nullptr);
imageGeomPtr->setOrigin({0, 0, 0});
imageGeomPtr->setSpacing({1e-7f, 2e-7f, 3e-7f});
const std::string actual = WriteUniformFile(dataStructure, labelsPath, binary, binary ? "UniformTinySpacingBinary.am" : "UniformTinySpacingAscii.am");
const std::string marker = " BoundingBox ";
const usize boxStart = actual.find(marker);
REQUIRE(boxStart != std::string::npos);
const std::string boxLine = actual.substr(boxStart + marker.size(), actual.find('\n', boxStart) - boxStart - marker.size());
INFO("BoundingBox " << boxLine);
std::istringstream bounds(boxLine);
const std::array<float64, 3> origin = {0, 0, 0};
const std::array<float64, 3> spacing = {static_cast<float64>(1e-7f), static_cast<float64>(2e-7f), static_cast<float64>(3e-7f)};
for(usize axis = 0; axis < 3; ++axis)
{
const std::array<float64, 2> expectedBounds = {origin[axis] + 0.5 * spacing[axis], origin[axis] + (static_cast<float64>(dims[axis]) - 0.5) * spacing[axis]};
for(float64 expected : expectedBounds)
{
CAPTURE(axis, expected);
std::string token;
REQUIRE(static_cast<bool>(bounds >> token));
char* endPtr = nullptr;
const float64 value = std::strtod(token.c_str(), &endPtr);
REQUIRE(endPtr == token.c_str() + token.size());
REQUIRE(value > 0);
REQUIRE(value == expected);
}
}
UnitTest::CheckArraysInheritTupleDims(dataStructure);
}
Command: ctest --test-dir build/simplnx-Rel -R "WriteAvizoUniformCoordinateFilter: Tiny spacing bounds round trip" -j 1 --output-on-failure
Output before the fix (Release, macOS arm64; both encodings fail):
WriteAvizoUniformCoordinateTest.cpp:123: FAILED:
REQUIRE( value > 0 )
with expansion:
0.0 > 0
with messages:
binary := false
BoundingBox 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
axis := 0
expected := 0.00000005
...
binary := true
BoundingBox 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
test cases: 1 | 0 passed | 1 failed
After the fix the test passes.
2. Another place in the class: Write Avizo Rectilinear Coordinate. File src/Plugins/SimplnxCore/test/WriteAvizoRectilinearCoordinateTest.cpp (with #include <array>, <cstdlib>, <sstream> and simplnx/Utilities/DataStoreUtilities.hpp). The writer is unchanged from aacedb4dc.
TEST_CASE("SimplnxCore::WriteAvizoRectilinearCoordinateFilter: Tiny spacing coordinates round trip", "[SimplnxCore][WriteAvizoRectilinearCoordinateFilter]")
{
const SizeVec3 dims = {3, 2, 1};
const FloatVec3 origin = {0, 0, 0};
const FloatVec3 spacing = {1e-7f, 2e-7f, 3e-7f};
DataStructure dataStructure;
auto* imageGeomPtr = ImageGeom::Create(dataStructure, "Image");
REQUIRE(imageGeomPtr != nullptr);
imageGeomPtr->setDimensions(dims);
imageGeomPtr->setOrigin(origin);
imageGeomPtr->setSpacing(spacing);
const ShapeType shape = {dims[2], dims[1], dims[0]};
auto* cellsPtr = AttributeMatrix::Create(dataStructure, "Cells", shape, imageGeomPtr->getId());
REQUIRE(cellsPtr != nullptr);
imageGeomPtr->setCellData(cellsPtr->getId());
const DataPath featureIdsPath({"Image", "Cells", "FeatureIds"});
auto* featureIdsPtr = Int32Array::Create(dataStructure, "FeatureIds", DataStoreUtilities::CreateDataStore<int32>(dataStructure, featureIdsPath, shape, {1}), cellsPtr->getId());
REQUIRE(featureIdsPtr != nullptr);
const std::array<int32, 6> labels = {1, 2, 3, 4, 5, 6};
const auto writeResult = featureIdsPtr->getDataStoreRef().copyFromBuffer(0, labels);
SIMPLNX_RESULT_REQUIRE_VALID(writeResult);
WriteAvizoRectilinearCoordinateFilter filter;
Arguments args = filter.getDefaultArguments();
const fs::path path = fs::path(unit_test::k_BinaryTestOutputDir.view()) / "RectilinearTinySpacingAscii.am";
args.insertOrAssign(WriteAvizoRectilinearCoordinateFilter::k_OutputFile_Key, std::make_any<fs::path>(path));
args.insertOrAssign(WriteAvizoRectilinearCoordinateFilter::k_WriteBinaryFile_Key, std::make_any<bool>(false));
args.insertOrAssign(WriteAvizoRectilinearCoordinateFilter::k_GeometryPath_Key, std::make_any<DataPath>(DataPath({"Image"})));
args.insertOrAssign(WriteAvizoRectilinearCoordinateFilter::k_FeatureIdsArrayPath_Key, std::make_any<DataPath>(featureIdsPath));
const auto preflightResult = filter.preflight(dataStructure, args);
SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions);
const auto executeResult = filter.execute(dataStructure, args);
SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result);
std::ifstream file(path);
REQUIRE(file.is_open());
std::string line;
bool foundCoordinates = false;
while(std::getline(file, line))
{
if(line == "@2 # x coordinates, then y, then z")
{
foundCoordinates = true;
break;
}
}
REQUIRE(foundCoordinates);
REQUIRE(static_cast<bool>(std::getline(file, line)));
INFO("X coordinates: " << line);
std::istringstream coordinates(line);
for(usize i = 0; i < dims[0]; ++i)
{
const float32 expected = origin[0] + (spacing[0] * i);
CAPTURE(i, expected);
std::string token;
REQUIRE(static_cast<bool>(coordinates >> token));
char* endPtr = nullptr;
const float64 value = std::strtod(token.c_str(), &endPtr);
REQUIRE(endPtr == token.c_str() + token.size());
REQUIRE(value == static_cast<float64>(expected));
if(i > 0)
{
REQUIRE(value > 0);
}
}
UnitTest::CheckArraysInheritTupleDims(dataStructure);
}
Command: ctest --test-dir build/simplnx-Rel -R "WriteAvizoRectilinearCoordinateFilter: Tiny spacing coordinates round trip" -j 1 --output-on-failure
Output (Release, macOS arm64):
WriteAvizoRectilinearCoordinateTest.cpp:234: FAILED:
REQUIRE( value == static_cast<float64>(expected) )
with expansion:
0.0 == 0.0000001
with messages:
X coordinates: 0.000000 0.000000 0.000000
i := 1
test cases: 1 | 0 passed | 1 failed
Proposed Fix
Use the pattern from the fixed instance in PR #1919 at every place above:
- fprintf(outputFile, " BoundingBox %f %f %f %f %f %f\n", ...);
+ fmt::print(outputFile, " BoundingBox {} {} {} {} {} {}\n", ...);
- fmt: plain
{}(shortest round-trip text) for the type actually written. - printf:
fprintf(f, "%.*g", std::numeric_limits<T>::max_digits10, static_cast<double>(value)). - Streams:
std::defaultfloat << std::setprecision(std::numeric_limits<T>::max_digits10). Resetstd::fixedexplicitly. - Python:
repr(value); drop the rounding in_encode_float(). - Format the computed type: a float64 expression of float32 inputs needs 17 digits or
{}.
Where a file format really fixes the precision, name that requirement in the code. Each place should get a round-trip test like the two above (tiny nonzero values, and values that need all 9 or 17 digits). PR #1919 adds the knowledge-base entry "Write floating-point data as text that reads back to the same value" to docs/Coding_Knowledge_Base.md.
Notes
- Found during the review of PR #1919 by an audit of floating-point text output in simplnx, SimplnxOoc, DREAM3D-NX and EbsdLib. SimplnxOoc has no affected place. DREAM3D-NX and EbsdLib places are reported in BlueQuartzSoftware/DREAM3DNX#1238 and BlueQuartzSoftware/EbsdLib#65.
- Open PRs that touch some of these writers keep the lossy formats: #1764 (both Avizo writers), #1915 (INL symmetry codes, ODF angle file delimiter).
- Related, not part of this issue:
NumberToStringFilterParameterConverter(StringParameter.cpp) converts legacy SIMPL numeric values withstd::to_string, which also writes six decimal places.
- Lingua principale
- C++
- Stelle
- 18
- Fork
- 15
- Merge medio
- 4g 23h
- PR unite (30g)
- 42
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di BlueQuartzSoftware/simplnx
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
BlueQuartzSoftware/simplnx#1987 ·
I maintainer di solito rispondono entro 6 giorni
-
documentation
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
BlueQuartzSoftware/simplnx#1980 ·
I maintainer di solito rispondono entro 6 giorni
-
Testing
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
BlueQuartzSoftware/simplnx#1979 ·
I maintainer di solito rispondono entro 6 giorni
-
documentation
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
BlueQuartzSoftware/simplnx#1978 ·
I maintainer di solito rispondono entro 6 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
BlueQuartzSoftware/simplnx#1945 ·
I maintainer di solito rispondono entro 6 giorni
Tutte le issue di BlueQuartzSoftware/simplnx
Issue simili
-
bug triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
I maintainer di solito rispondono entro 1 giorno
-
26.04 bug dark-mode Easy Hack impress symptom: visual
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
CollaboraOnline/online#16421 · 1 commento ·
-
area/cs bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
ydb-platform/ydb#59881 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
I maintainer di solito rispondono entro 1 giorno
-
documentation enhancement lua server
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100