BUG: C11: float32 accumulation loses precision in 16 filters
Los mantenedores suelen responder en 6 días
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 25/100
Línea de trabajo
Start with the checklist in the issue, which names the affected filters and source locations, then read the shared helpers in src/simplnx/Utilities/HistogramUtilities.hpp and the listed filter implementations. Add failing tests with independent expected results for each finding before changing code; the issue gives specific input cases and oracles for counters, sums, histograms, and the hanging loop. Done means all 16 findings are addressed and their tests pass, with any V&V package updates required by touched filters.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Summary
Many filters count or sum values in float32 variables. A float32 has a 24-bit significand. A counter incremented with ++ or += 1.0f stops at 16,777,216 (2^24), and a long sum loses precision. Large Features, large slices, large ranges or coordinates far from the origin then give wrong output without an error. In one case a loop variable stops advancing and the filter hangs.
As of develop 419f9db35 (2026-10-21) there are 17 open findings in 16 filters: 1 CRASH (hang), 15 WRONG_DATA and 1 MINOR. Each finding is a candidate. It was found and re-checked by reading code, not by running it. The exception is ComputeShapesFilter#9, which was added on 2026-11-02 from legacy DREAM3D issue BlueQuartzSoftware/DREAM3D#425 and was reproduced with a failing unit test on develop b51d6c353.
Mechanism
float32 can store every integer only up to 2^24. Then 16777216.0f + 1.0f == 16777216.0f, so a counter stops. A sum of many small values stops growing when each value is smaller than half the spacing between floats at the current total. The forms are:
- Counts in
float32. Voxel or sample counters arefloat32and are incremented one at a time. A Feature with more than 16,777,216 cells (for example a matrix Feature in a 300^3 Image Geometry) gets a capped count, so volumes, fractions and means are wrong. - Sums in
float32. Centroids, means and threshold sums usefloat32accumulators. - Intermediate math in
float32. Histogram increments and bin indices, color-map normalization and triangle volume determinants usefloat32forint32,int64orfloat64input, or for coordinates far from the origin (cancellation). - Loop variable in
float32.lineY += lineSpacingstops advancing when the spacing is smaller than half the float spacing atlineY, and the loop never ends.
Example (ComputeBoundaryElementFractions). Both counters are float32. They are incremented with ++ for every cell (lines 70 and 73), so they stop at 2^24 and the fraction is wrong for large Features:
std::vector<float32> surfVoxCounts(numFeatures, 0);
std::vector<float32> voxCounts(numFeatures, 0);
Fixed form. PR #1817 (commit 01809c236) changed ComputeSurfaceAreaToVolume to compute and accumulate in float64 and to convert to float32 only when it writes the output:
const float64 voxelVol = static_cast<float64>(spacing[0]) * spacing[1] * spacing[2];
const float64 xyFaceArea = static_cast<float64>(spacing[0]) * spacing[1];
const float64 yzFaceArea = static_cast<float64>(spacing[1]) * spacing[2];
const float64 zxFaceArea = static_cast<float64>(spacing[2]) * spacing[0];
// Multiple cells contribute to one feature, so accumulation stays local until
// the cell scan is complete.
std::vector<float64> featureSurfaceArea(numFeatures, 0.0);
Rule
- Count in an integer type (
usizeoruint64). Sum and do intermediate math infloat64. Convert to the output type only when you write the output. - Loops over a real-valued range use an integer step counter
kand compute the value asstart + k * stepinfloat64. - Moving accumulators to
float64changes exemplar outputs. Treat this class as V&V-style work: prove each change with an independent oracle. Do NOT regenerate exemplars just to absorb the change. If an exemplar comparison fails after the fix, report the difference and the oracle evidence in the pull request, and let the project lead decide about the exemplar. - Checks belong in preflight unless they need DataArray values.
How to fix
- Change each
float32counter tousizeoruint64. Compute fractions and means infloat64, then cast the result. - Change each
float32sum tofloat64. If the order of summation matters for exact results, keep the existing order. - In
src/simplnx/Utilities/HistogramUtilities.hpp, computeCalculateIncrement,FillBinRangesandCalculateBininfloat64(lines 21-24 and 84-90 at 419f9db35). The per-featuresummationat line 395 is alsovector<float32>. These helpers are shared, so one change fixes ComputeArrayHistogram, ComputeArrayHistogramByFeature and part of ComputeArrayStatistics. Check every caller. - In
src/simplnx/Utilities/ColorTableUtilities.hppline 151, divide infloat64, then cast the [0,1] result tofloat32. - In
src/simplnx/Utilities/Meshing/TriangleUtilities.cpp(FindTriangleVolume, lines 922-938), compute the determinant infloat64relative to a reference point near the Feature, and accumulate infloat64. - In
CreateAMScanPaths.cppline 190, iterate an integerkand computelineY = startLineY + k * lineSpacinginfloat64. Add a cancel check in the loop. - Fix every copy of the code in the same filter: Direct and Scanline algorithms, 2D and 3D paths, and in-core and out-of-core paths (the checklist names the other lines).
- New error codes (if any): use the filter's existing error-code range. Codes are more negative than -10000, and each filter uses one range of 100 codes. Check uniqueness with
git grep -nE -- "-NNNN([^0-9]|$)"(not\b) on develop and on open pull request refs.
How to test
Write a unit test that fails on develop first. Then fix the code.
- Counter saturation: a test with more than 2^24 elements in one Feature or slice. A uniform input (for example a 257 x 257 x 257 Image Geometry with Feature Id 1 everywhere, 16,974,593 cells) has an exact oracle: the count is the number of cells, and the mean of a constant is the constant. If the memory cost is too high for the normal test suite, test the extracted helper with a synthetic stream of 2^24 + N values, or mark the test as a long test. Write the choice in the pull request.
- Sum and math precision: use inputs with a closed-form answer (for example vertices offset by 1e5 to 1e6 from the origin around a unit cube, whose volume is 1 and whose centroid is known). Compare with a tolerance that is derived from
float64, not from the old output. - Histogram: an
int32DataArray {0, 100000000} with 10 bins. The oracle is the bin edges from integer arithmetic, and the maximum value is in the last bin. - Hang: an Edge Geometry with coordinates near 1e5 and Hatch Spacing < 0.004. Assert that the filter finishes and that the line count equals
floor((maxY - startY) / spacing) + 1. - If the filter has Direct and Scanline algorithms, test both through the existing algorithm-selection test scope.
- Value checks need an independent expected result (oracle), not the output of the code under test.
Conventions
- One pull request for the whole class. One commit per filter.
- Commit prefix
BUG:. Add aSigned-off-byline. Do not add AI attribution. - Every fix starts with a unit test that fails on develop. If you cannot reproduce a finding, write that in the pull request and on this issue with the reason. Do not force a fix.
- Value checks need an independent expected result (oracle).
- Run ctest serially. Never use
-j; the tests depend on each other. - Format with clang-format 19.1.1 and the repository
.clang-format. - Do not add per-filter documentation sentences about input requirements. The generated parameter table documents them.
- If a fix touches a filter that has a V&V report (
src/Plugins/<Plugin>/vv/<Filter>.md), update every file of that filter's V&V package (report, deviations, provenance, test tables). V&V documents reference DREAM3D 6.5.171 only. - Rebase on develop. Never merge develop into the branch.
Checklist
- CRASH · CreateAMScanPathsFilter · CreateAMScanPaths.cpp:190 —
float lineY += lineSpacing: with coordinates near 1e5 and Hatch Spacing < 0.004,lineYstops advancing and the loop never ends(no cancel check). Otherwise the spacing is quantized.CreateAMScanPathsFilter#6— cancellation handling moved to #1876; floating-point loop progress and spacing precision remain here. - WRONG_DATA · AlignGeometriesFilter · AlignGeometries.cpp:205 — With Alignment Type = Centroid, vertex centroids are summed in
float32(also lines 220-225, 239-244, 258-263), so a mesh with millions of vertices far from the origin gets a slightly wrong translation.AlignGeometriesFilter#5 - WRONG_DATA · AlignSectionsFeatureCentroidFilter · AlignSectionsFeatureCentroid.cpp:105 — xCentroid/yCentroid are
float32sums over the slice (also the out-of-core path near line 322), so on a 2000 x 2000 slice with a nearly full mask the shift can change by one cell.AlignSectionsFeatureCentroidFilter#6 - WRONG_DATA · AlignSectionsMisorientationFilter · AlignSectionsMisorientation.cpp:137 —
disorientationandcountarefloat32and incremented with++(also lines 260-261 and 522-523), so they stop at 2^24 and bias the mismatch fraction on XY slices of more than about 268M cells.AlignSectionsMisorientationFilter#2 - WRONG_DATA · AlignSectionsMutualInformationFilter · AlignSectionsMutualInformation.cpp:371 — The histograms (lines 309-311) and
count(lines 371 and 513) arefloat32and incremented with++, so they stop at 2^24 and skew the mutual information on very large slices.AlignSectionsMutualInformationFilter#4 - WRONG_DATA · ComputeArrayHistogramByFeatureFilter · HistogramUtilities.hpp:23 —
CalculateIncrementreturnsfloat32andFillBinRanges/CalculateBinuse it forint32,int64andfloat64too, so for values over [0, 2^30) the bin edges and the assigned bins disagree near the edges.ComputeArrayHistogramByFeatureFilter#11 - WRONG_DATA · ComputeArrayHistogramFilter · HistogramUtilities.hpp:23 — Same
float32increment and bin index (lines 21-24, 88-89): for anint32DataArray {0, 100000000} with 10 bins, the maximum lands in overflow and the stored upper edges are wrong.ComputeArrayHistogramFilter#3 - WRONG_DATA · ComputeArrayStatisticsFilter · ComputeArrayStatistics.cpp:2036 — Per-group
sumsarefloat32(line 1907; also HistogramUtilities.hpp:395), so with Compute Statistics Per Feature/Ensemble on and a group of more than 2^24 tuples the Mean is wrong (0.62 instead of 1.0).ComputeArrayStatisticsFilter#4 - WRONG_DATA · ComputeBoundaryElementFractionsFilter · ComputeBoundaryElementFractions.cpp:43 — Cell counts are
vector<float32>incremented with++, so for a Feature of more than 16,777,216 cells the count stops and the fraction is overestimated.ComputeBoundaryElementFractionsFilter#3 - WRONG_DATA · ComputeBoundingBoxStatsFilter · ComputeBoundingBoxStatsDirect.cpp:588 —
float32input is summed into afloat32accumulator, so a box of more than about 1e7 cells (for example all 0.1) drifts. Scanline lines 593 and 651 are the same.ComputeBoundingBoxStatsFilter#6 - WRONG_DATA · ComputeLargestCrossSectionsFilter · ComputeLargestCrossSectionsDirect.cpp:199 —
featureCountsisvector<float32>incremented with++, so it stops at 2^24 per plane (for example 4200 x 4200 XY planes). Scanline line 46 is the same.ComputeLargestCrossSectionsFilter#2 - WRONG_DATA · ComputeShapesFilter · ComputeShapes.cpp:312 —
featureVoxelCounts[gnum] += 1.0fon afloat32vector stops at 2^24, which caps Volumes and corrupts Omega3s for large Features. Lines 470 and 515 (second path) are the same.ComputeShapesFilter#3 - WRONG_DATA · ComputeShapesFilter · ComputeShapes.cpp:290 — The per-voxel moment terms are computed in
float32: the eight corner distances andxx,yy,zz,xy,xz,yzarefloat(lines 195 and 212), and Omega3 is formed fromfloat32central moments (lines 390-397). For a filled 4096 × 8 × 4 cuboid Feature (131,072 cells, well below the 2^24 cap ofComputeShapesFilter#3), Omega3s is 0.8024827 but the exact value for the sampled points is 0.8035182 (relative error -1.3e-3). The same sums infloat64give the exact value. Axis Lengths stay correct to about 3e-8, so the eigensolve itself is not the problem. Legacy DREAM3D issue BlueQuartzSoftware/DREAM3D#425 reported this.ComputeShapesFilter#9 - WRONG_DATA · ComputeTriangleGeomVolumesFilter · TriangleUtilities.cpp:922 —
FindTriangleVolumebuilds the matrix relative to the origin infloat32and volumes are accumulated infloat32, so a small closed Feature at coordinates around 1e5-1e6 loses its volume to cancellation.ComputeTriangleGeomVolumesFilter#6 - WRONG_DATA · CreateColorMapFilter · ColorTableUtilities.hpp:151 — The non-integer path divides in
float32, so afloat64range larger than FLT_MAX (for example [0, 1e30, 1e40]) becomes inf, then 0 or NaN, and every value gets the first color.CreateColorMapFilter#4 - WRONG_DATA · RobustAutomaticThresholdFilter · RobustAutomaticThreshold.cpp:47 —
numeratoranddenominatorarefloatand summed per tuple (lines 69-70), so for 20M tuples the denominator stops at 2^24 and the threshold at line 75 is wrong.RobustAutomaticThresholdFilter#1 - MINOR · TriangleCentroidFilter · TriangleCentroid.cpp:40 — Three
float32coordinates are added and divided by3.0Fbefore the value goes into thefloat64output, so the result loses about 1 ulp offloat32precision (for example X = 16777216, 1, 1).TriangleCentroidFilter#1
Already fixed or in progress
- PR #1763 (merged, commit 8bed04ce9):
ComputeKMeansFilter#7,SliceTriangleGeometryFilter#3. - PR #1817 (merged, commit 01809c236):
ComputeSurfaceAreaToVolumeFilter#4.
Out of scope
- Findings in filters that have a V&V report are excluded from this issue: 7 findings, in ComputeAvgCAxesFilter, ComputeAvgOrientationsFilter, ComputeFeatureReferenceCAxisMisorientationsFilter, ComputeFeatureReferenceMisorientationsFilter, ComputeTriangleGeomCentroidsFilter and DBSCANFilter. A separate V&V branch handles them.
- Changes to exemplar archives only to make a test pass. See Rule.
- Lenguaje dominante
- C++
- Estrellas
- 18
- Forks
- 15
- Merge medio
- 4 d 23 h
- PR fusionados (30 d)
- 42
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de BlueQuartzSoftware/simplnx
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
BlueQuartzSoftware/simplnx#1987 ·
Los mantenedores suelen responder en 6 días
-
documentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
BlueQuartzSoftware/simplnx#1980 ·
Los mantenedores suelen responder en 6 días
-
Testing
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
BlueQuartzSoftware/simplnx#1979 ·
Los mantenedores suelen responder en 6 días
-
documentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
BlueQuartzSoftware/simplnx#1978 ·
Los mantenedores suelen responder en 6 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
BlueQuartzSoftware/simplnx#1945 ·
Los mantenedores suelen responder en 6 días
Todos los issues de BlueQuartzSoftware/simplnx
Issues similares
-
bug triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
Los mantenedores suelen responder en 1 día
-
26.04 bug dark-mode Easy Hack impress symptom: visual
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
CollaboraOnline/online#16421 · 1 comentario ·
-
area/cs bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
ydb-platform/ydb#59881 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 64/100
Los mantenedores suelen responder en 1 día
-
documentation enhancement lua server
Dificultad 2/5 1-3 horas Aptitud para principiantes 66/100