BUG: C11: float32 accumulation loses precision in 16 filters
维护者通常 6 天内回复
还没有人认领这个 Issue。
评估
调研方向
Start with the checklist in the issue, which names the affected filters and source locations, then read the shared helpers in src/simplnx/Utilities/HistogramUtilities.hpp and the listed filter implementations. Add failing tests with independent expected results for each finding before changing code; the issue gives specific input cases and oracles for counters, sums, histograms, and the hanging loop. Done means all 16 findings are addressed and their tests pass, with any V&V package updates required by touched filters.
由索引模型根据 Issue 内容生成。
描述
Summary
Many filters count or sum values in float32 variables. A float32 has a 24-bit significand. A counter incremented with ++ or += 1.0f stops at 16,777,216 (2^24), and a long sum loses precision. Large Features, large slices, large ranges or coordinates far from the origin then give wrong output without an error. In one case a loop variable stops advancing and the filter hangs.
As of develop 419f9db35 (2026-10-21) there are 17 open findings in 16 filters: 1 CRASH (hang), 15 WRONG_DATA and 1 MINOR. Each finding is a candidate. It was found and re-checked by reading code, not by running it. The exception is ComputeShapesFilter#9, which was added on 2026-11-02 from legacy DREAM3D issue BlueQuartzSoftware/DREAM3D#425 and was reproduced with a failing unit test on develop b51d6c353.
Mechanism
float32 can store every integer only up to 2^24. Then 16777216.0f + 1.0f == 16777216.0f, so a counter stops. A sum of many small values stops growing when each value is smaller than half the spacing between floats at the current total. The forms are:
- Counts in
float32. Voxel or sample counters arefloat32and are incremented one at a time. A Feature with more than 16,777,216 cells (for example a matrix Feature in a 300^3 Image Geometry) gets a capped count, so volumes, fractions and means are wrong. - Sums in
float32. Centroids, means and threshold sums usefloat32accumulators. - Intermediate math in
float32. Histogram increments and bin indices, color-map normalization and triangle volume determinants usefloat32forint32,int64orfloat64input, or for coordinates far from the origin (cancellation). - Loop variable in
float32.lineY += lineSpacingstops advancing when the spacing is smaller than half the float spacing atlineY, and the loop never ends.
Example (ComputeBoundaryElementFractions). Both counters are float32. They are incremented with ++ for every cell (lines 70 and 73), so they stop at 2^24 and the fraction is wrong for large Features:
std::vector<float32> surfVoxCounts(numFeatures, 0);
std::vector<float32> voxCounts(numFeatures, 0);
Fixed form. PR #1817 (commit 01809c236) changed ComputeSurfaceAreaToVolume to compute and accumulate in float64 and to convert to float32 only when it writes the output:
const float64 voxelVol = static_cast<float64>(spacing[0]) * spacing[1] * spacing[2];
const float64 xyFaceArea = static_cast<float64>(spacing[0]) * spacing[1];
const float64 yzFaceArea = static_cast<float64>(spacing[1]) * spacing[2];
const float64 zxFaceArea = static_cast<float64>(spacing[2]) * spacing[0];
// Multiple cells contribute to one feature, so accumulation stays local until
// the cell scan is complete.
std::vector<float64> featureSurfaceArea(numFeatures, 0.0);
Rule
- Count in an integer type (
usizeoruint64). Sum and do intermediate math infloat64. Convert to the output type only when you write the output. - Loops over a real-valued range use an integer step counter
kand compute the value asstart + k * stepinfloat64. - Moving accumulators to
float64changes exemplar outputs. Treat this class as V&V-style work: prove each change with an independent oracle. Do NOT regenerate exemplars just to absorb the change. If an exemplar comparison fails after the fix, report the difference and the oracle evidence in the pull request, and let the project lead decide about the exemplar. - Checks belong in preflight unless they need DataArray values.
How to fix
- Change each
float32counter tousizeoruint64. Compute fractions and means infloat64, then cast the result. - Change each
float32sum tofloat64. If the order of summation matters for exact results, keep the existing order. - In
src/simplnx/Utilities/HistogramUtilities.hpp, computeCalculateIncrement,FillBinRangesandCalculateBininfloat64(lines 21-24 and 84-90 at 419f9db35). The per-featuresummationat line 395 is alsovector<float32>. These helpers are shared, so one change fixes ComputeArrayHistogram, ComputeArrayHistogramByFeature and part of ComputeArrayStatistics. Check every caller. - In
src/simplnx/Utilities/ColorTableUtilities.hppline 151, divide infloat64, then cast the [0,1] result tofloat32. - In
src/simplnx/Utilities/Meshing/TriangleUtilities.cpp(FindTriangleVolume, lines 922-938), compute the determinant infloat64relative to a reference point near the Feature, and accumulate infloat64. - In
CreateAMScanPaths.cppline 190, iterate an integerkand computelineY = startLineY + k * lineSpacinginfloat64. Add a cancel check in the loop. - Fix every copy of the code in the same filter: Direct and Scanline algorithms, 2D and 3D paths, and in-core and out-of-core paths (the checklist names the other lines).
- New error codes (if any): use the filter's existing error-code range. Codes are more negative than -10000, and each filter uses one range of 100 codes. Check uniqueness with
git grep -nE -- "-NNNN([^0-9]|$)"(not\b) on develop and on open pull request refs.
How to test
Write a unit test that fails on develop first. Then fix the code.
- Counter saturation: a test with more than 2^24 elements in one Feature or slice. A uniform input (for example a 257 x 257 x 257 Image Geometry with Feature Id 1 everywhere, 16,974,593 cells) has an exact oracle: the count is the number of cells, and the mean of a constant is the constant. If the memory cost is too high for the normal test suite, test the extracted helper with a synthetic stream of 2^24 + N values, or mark the test as a long test. Write the choice in the pull request.
- Sum and math precision: use inputs with a closed-form answer (for example vertices offset by 1e5 to 1e6 from the origin around a unit cube, whose volume is 1 and whose centroid is known). Compare with a tolerance that is derived from
float64, not from the old output. - Histogram: an
int32DataArray {0, 100000000} with 10 bins. The oracle is the bin edges from integer arithmetic, and the maximum value is in the last bin. - Hang: an Edge Geometry with coordinates near 1e5 and Hatch Spacing < 0.004. Assert that the filter finishes and that the line count equals
floor((maxY - startY) / spacing) + 1. - If the filter has Direct and Scanline algorithms, test both through the existing algorithm-selection test scope.
- Value checks need an independent expected result (oracle), not the output of the code under test.
Conventions
- One pull request for the whole class. One commit per filter.
- Commit prefix
BUG:. Add aSigned-off-byline. Do not add AI attribution. - Every fix starts with a unit test that fails on develop. If you cannot reproduce a finding, write that in the pull request and on this issue with the reason. Do not force a fix.
- Value checks need an independent expected result (oracle).
- Run ctest serially. Never use
-j; the tests depend on each other. - Format with clang-format 19.1.1 and the repository
.clang-format. - Do not add per-filter documentation sentences about input requirements. The generated parameter table documents them.
- If a fix touches a filter that has a V&V report (
src/Plugins/<Plugin>/vv/<Filter>.md), update every file of that filter's V&V package (report, deviations, provenance, test tables). V&V documents reference DREAM3D 6.5.171 only. - Rebase on develop. Never merge develop into the branch.
Checklist
- CRASH · CreateAMScanPathsFilter · CreateAMScanPaths.cpp:190 —
float lineY += lineSpacing: with coordinates near 1e5 and Hatch Spacing < 0.004,lineYstops advancing and the loop never ends(no cancel check). Otherwise the spacing is quantized.CreateAMScanPathsFilter#6— cancellation handling moved to #1876; floating-point loop progress and spacing precision remain here. - WRONG_DATA · AlignGeometriesFilter · AlignGeometries.cpp:205 — With Alignment Type = Centroid, vertex centroids are summed in
float32(also lines 220-225, 239-244, 258-263), so a mesh with millions of vertices far from the origin gets a slightly wrong translation.AlignGeometriesFilter#5 - WRONG_DATA · AlignSectionsFeatureCentroidFilter · AlignSectionsFeatureCentroid.cpp:105 — xCentroid/yCentroid are
float32sums over the slice (also the out-of-core path near line 322), so on a 2000 x 2000 slice with a nearly full mask the shift can change by one cell.AlignSectionsFeatureCentroidFilter#6 - WRONG_DATA · AlignSectionsMisorientationFilter · AlignSectionsMisorientation.cpp:137 —
disorientationandcountarefloat32and incremented with++(also lines 260-261 and 522-523), so they stop at 2^24 and bias the mismatch fraction on XY slices of more than about 268M cells.AlignSectionsMisorientationFilter#2 - WRONG_DATA · AlignSectionsMutualInformationFilter · AlignSectionsMutualInformation.cpp:371 — The histograms (lines 309-311) and
count(lines 371 and 513) arefloat32and incremented with++, so they stop at 2^24 and skew the mutual information on very large slices.AlignSectionsMutualInformationFilter#4 - WRONG_DATA · ComputeArrayHistogramByFeatureFilter · HistogramUtilities.hpp:23 —
CalculateIncrementreturnsfloat32andFillBinRanges/CalculateBinuse it forint32,int64andfloat64too, so for values over [0, 2^30) the bin edges and the assigned bins disagree near the edges.ComputeArrayHistogramByFeatureFilter#11 - WRONG_DATA · ComputeArrayHistogramFilter · HistogramUtilities.hpp:23 — Same
float32increment and bin index (lines 21-24, 88-89): for anint32DataArray {0, 100000000} with 10 bins, the maximum lands in overflow and the stored upper edges are wrong.ComputeArrayHistogramFilter#3 - WRONG_DATA · ComputeArrayStatisticsFilter · ComputeArrayStatistics.cpp:2036 — Per-group
sumsarefloat32(line 1907; also HistogramUtilities.hpp:395), so with Compute Statistics Per Feature/Ensemble on and a group of more than 2^24 tuples the Mean is wrong (0.62 instead of 1.0).ComputeArrayStatisticsFilter#4 - WRONG_DATA · ComputeBoundaryElementFractionsFilter · ComputeBoundaryElementFractions.cpp:43 — Cell counts are
vector<float32>incremented with++, so for a Feature of more than 16,777,216 cells the count stops and the fraction is overestimated.ComputeBoundaryElementFractionsFilter#3 - WRONG_DATA · ComputeBoundingBoxStatsFilter · ComputeBoundingBoxStatsDirect.cpp:588 —
float32input is summed into afloat32accumulator, so a box of more than about 1e7 cells (for example all 0.1) drifts. Scanline lines 593 and 651 are the same.ComputeBoundingBoxStatsFilter#6 - WRONG_DATA · ComputeLargestCrossSectionsFilter · ComputeLargestCrossSectionsDirect.cpp:199 —
featureCountsisvector<float32>incremented with++, so it stops at 2^24 per plane (for example 4200 x 4200 XY planes). Scanline line 46 is the same.ComputeLargestCrossSectionsFilter#2 - WRONG_DATA · ComputeShapesFilter · ComputeShapes.cpp:312 —
featureVoxelCounts[gnum] += 1.0fon afloat32vector stops at 2^24, which caps Volumes and corrupts Omega3s for large Features. Lines 470 and 515 (second path) are the same.ComputeShapesFilter#3 - WRONG_DATA · ComputeShapesFilter · ComputeShapes.cpp:290 — The per-voxel moment terms are computed in
float32: the eight corner distances andxx,yy,zz,xy,xz,yzarefloat(lines 195 and 212), and Omega3 is formed fromfloat32central moments (lines 390-397). For a filled 4096 × 8 × 4 cuboid Feature (131,072 cells, well below the 2^24 cap ofComputeShapesFilter#3), Omega3s is 0.8024827 but the exact value for the sampled points is 0.8035182 (relative error -1.3e-3). The same sums infloat64give the exact value. Axis Lengths stay correct to about 3e-8, so the eigensolve itself is not the problem. Legacy DREAM3D issue BlueQuartzSoftware/DREAM3D#425 reported this.ComputeShapesFilter#9 - WRONG_DATA · ComputeTriangleGeomVolumesFilter · TriangleUtilities.cpp:922 —
FindTriangleVolumebuilds the matrix relative to the origin infloat32and volumes are accumulated infloat32, so a small closed Feature at coordinates around 1e5-1e6 loses its volume to cancellation.ComputeTriangleGeomVolumesFilter#6 - WRONG_DATA · CreateColorMapFilter · ColorTableUtilities.hpp:151 — The non-integer path divides in
float32, so afloat64range larger than FLT_MAX (for example [0, 1e30, 1e40]) becomes inf, then 0 or NaN, and every value gets the first color.CreateColorMapFilter#4 - WRONG_DATA · RobustAutomaticThresholdFilter · RobustAutomaticThreshold.cpp:47 —
numeratoranddenominatorarefloatand summed per tuple (lines 69-70), so for 20M tuples the denominator stops at 2^24 and the threshold at line 75 is wrong.RobustAutomaticThresholdFilter#1 - MINOR · TriangleCentroidFilter · TriangleCentroid.cpp:40 — Three
float32coordinates are added and divided by3.0Fbefore the value goes into thefloat64output, so the result loses about 1 ulp offloat32precision (for example X = 16777216, 1, 1).TriangleCentroidFilter#1
Already fixed or in progress
- PR #1763 (merged, commit 8bed04ce9):
ComputeKMeansFilter#7,SliceTriangleGeometryFilter#3. - PR #1817 (merged, commit 01809c236):
ComputeSurfaceAreaToVolumeFilter#4.
Out of scope
- Findings in filters that have a V&V report are excluded from this issue: 7 findings, in ComputeAvgCAxesFilter, ComputeAvgOrientationsFilter, ComputeFeatureReferenceCAxisMisorientationsFilter, ComputeFeatureReferenceMisorientationsFilter, ComputeTriangleGeomCentroidsFilter and DBSCANFilter. A separate V&V branch handles them.
- Changes to exemplar archives only to make a test pass. See Rule.
- 主要语言
- C++
- 星标
- 18
- 派生
- 15
- 平均合并
- 4 天 23 小时
- 30 天内合并 PR
- 42
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
BlueQuartzSoftware/simplnx 的其他 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 78/100
BlueQuartzSoftware/simplnx#1987 ·
维护者通常 6 天内回复
-
documentation
难度 2/5 1-3 小时 新手友好度 74/100
BlueQuartzSoftware/simplnx#1980 ·
维护者通常 6 天内回复
-
Testing
难度 2/5 1-3 小时 新手友好度 62/100
BlueQuartzSoftware/simplnx#1979 ·
维护者通常 6 天内回复
-
documentation
难度 2/5 1-3 小时 新手友好度 65/100
BlueQuartzSoftware/simplnx#1978 ·
维护者通常 6 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
BlueQuartzSoftware/simplnx#1945 ·
维护者通常 6 天内回复
查看 BlueQuartzSoftware/simplnx 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
HarperFast/rocksdb-js#1118 ·
维护者通常 1 天内回复
-
bug triage
难度 2/5 1-3 小时 新手友好度 75/100
维护者通常 1 天内回复
-
26.04 bug dark-mode Easy Hack impress symptom: visual
难度 2/5 1-3 小时 新手友好度 68/100
CollaboraOnline/online#16421 · 1 条评论 ·
-
enhancement
难度 2/5 1-3 小时 新手友好度 63/100
telegramdesktop/tdesktop#31653 ·
维护者通常 1 天内回复
-
area/cs bug
难度 2/5 1-3 小时 新手友好度 82/100
ydb-platform/ydb#59881 · 1 条评论 ·
维护者通常 1 天内回复