[C++] FLOAT statistics use values before binary32 narrowing

Leggi c++/src/ColumnWriter.cc, in particolare FloatingColumnWriter::add, dove l’issue indica la differenza tra il valore FLOAT scritto in DATA e quello usato per le statistiche. Controlla i test C++ esistenti per le statistiche delle colonne in virgola mobile e il predicate pushdown, poi aggiungi test per il caso segnalato 0.1. Il lavoro è completato quando le statistiche corrispondono al valore FLOAT serializzato e la ricerca per uguaglianza conserva la riga.

The default C++ writer stores FLOAT statistics from the double input before narrowing it to binary32. A one-row file containing 0.1 returns no rows for an equality search argument whose literal exactly equals the stored FLOAT value.

Version and build

Apache ORC 3.0.0-SNAPSHOT, commit 2069271e06c92197c66def9d4a0214753ac9a3b8. Tested with GCC 13.3.0, CMake 3.28.3 and Python 3.12.3 on Ubuntu, Linux 6.17.0-1020-oracle, x86_64. C++ Release build, default numeric vectors and ZSTD compression. CMake Java and C++ tests were off. As checked on 2026-10-12, upstream main matches this commit.

Reproduction

Save this as repro.cc, then run the Bash commands below from the same directory. Reproducer: CC0-1.0.

#include <orc/OrcFile.hh>
#include <orc/Statistics.hh>
#include <orc/Vector.hh>
#include <orc/sargs/SearchArgument.hh>
#include <iostream>
#include <iomanip>
int main(int argc,char**argv){
  if(argc!=2)return 2;
  auto schema=orc::Type::buildTypeFromString("struct<x:float>");
  {
    auto out=orc::writeLocalFile(argv[1]);
    auto writer=orc::createWriter(*schema,out.get(),orc::WriterOptions());
    auto batch=writer->createRowBatch(1);
    auto& root=dynamic_cast<orc::StructVectorBatch&>(*batch);
    auto& x=dynamic_cast<orc::DoubleVectorBatch&>(*root.fields[0]);
    root.numElements=x.numElements=1;x.data[0]=0.1;
    writer->add(*batch);writer->close();
  }
  auto reader=orc::createReader(orc::readLocalFile(argv[1]),orc::ReaderOptions());
  auto stats=reader->getColumnStatistics(1);
  auto& f=dynamic_cast<orc::DoubleColumnStatistics&>(*stats);
  std::cout<<std::setprecision(17)<<"minimum "<<f.getMinimum()<<"\nmaximum "<<f.getMaximum()<<"\nsum "<<f.getSum()<<"\n";
  for(bool filtered:{false,true}){
    orc::RowReaderOptions opts;
    if(filtered) opts.searchArgument(orc::SearchArgumentFactory::newBuilder()->startAnd()
      .equals("x",orc::PredicateDataType::FLOAT,orc::Literal(0.100000001490116119384765625)).end().build());
    auto rows=reader->createRowReader(opts);auto batch=rows->createRowBatch(1);uint64_t count=0;
    while(rows->next(*batch)){
      auto& root=dynamic_cast<orc::StructVectorBatch&>(*batch);
      auto& x=dynamic_cast<orc::DoubleVectorBatch&>(*root.fields[0]);
      for(uint64_t i=0;i<root.numElements;i++) std::cout<<(filtered?"filtered_value ":"plain_value ")<<x.data[i]<<"\n";
      count+=root.numElements;
    }
    std::cout<<(filtered?"filtered_rows ":"plain_rows ")<<count<<"\n";
  }
}

git clone https://github.com/apache/orc.git orc
git -C orc checkout --detach 2069271e06c92197c66def9d4a0214753ac9a3b8
cmake -S orc -B orc-build \
  -DBUILD_JAVA=OFF -DBUILD_CPP_TESTS=OFF -DBUILD_TOOLS=ON \
  -DCMAKE_BUILD_TYPE=Release -DSTOP_BUILD_ON_WARNING=OFF
cmake --build orc-build --parallel 2
g++ -std=c++17 -O2 -Iorc/c++/include -Iorc-build/c++/include \
  repro.cc orc-build/c++/src/liborc.a \
  orc-build/_deps/protobuf-build/liborc_vendored_protobuf.a \
  orc-build/_deps/zlib-build/liborc_vendored_zlib.a \
  orc-build/_deps/snappy-build/liborc_vendored_snappy.a \
  orc-build/_deps/lz4-build/liborc_vendored_lz4.a \
  orc-build/_deps/zstd-build/lib/liborc_vendored_zstd.a \
  -pthread -o repro
./repro input.orc

Expected and observed

The column statistics specification defines statistics over column values. FLOAT encoding uses IEEE 754 binary32. The C++ vector documentation represents FLOAT input with DoubleVectorBatch.

The stored value has bits 0x3dcccccd. Its exact value is 13421773/134217728, or 0.100000001490116119384765625. With one non-null row, minimum, maximum and sum should all equal that value. The equality search argument should retain the row.

Observed:

minimum 0.10000000000000001
maximum 0.10000000000000001
sum 0.10000000000000001
plain_value 0.10000000149011612
plain_rows 1
filtered_rows 0

An independent byte decoder confirms the DATA value and the mismatching statistics at file, stripe and row-group levels. The literal equals the decoded FLOAT exactly.

Source and scope

In FloatingColumnWriter::add, DATA uses static_cast<float>(doubleData[i]), while statistics use the original double. Statistics should use the value serialized as FLOAT.

Independent C++ controls pass with pre-rounded input, tight float vectors, exact 0.5, and DOUBLE. Negative -0.1 also reproduces. No read-type conversion is used. The independent harness initializes row counts and null flags and closes the writer before reading. Predicate pushdown may retain extra rows, but it must retain this matching row.

Java's FLOAT writer was compared with source at the same commit only. No Java writer runtime comparison or other compiler or architecture test is claimed.

Found by ChipLab's AI verification agents and independently reproduced.

ChipLab ref: CLAB-2026-863