Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

[Bug] Flaky SIGSEGV in VariantParquetTest: Arrow IO thread frees PoolBuffer after the test's memory pool is destroyed

Geschlossen
  • Stimmt etwas nicht?
#385 0 Kommentare 0 Reaktionen 1 zugewiesene Person Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

@SteNicholas arbeitet bereits daran.

Seit 23.9.2026.

Bewertung

Dieses Issue wurde noch nicht bewertet.

Beschreibung

bug
Search before asking
  • I searched in the issues and found nothing similar.
Paimon-cpp version

main, observed on commit dc745fe91932dc2cba919fb13639dfc427280006 (the change in that commit only touches realtime/, unrelated to this failure).

Minimal reproduce step

Flaky; observed in CI job gcc-release-x86_64: https://github.com/apache/paimon-cpp/actions/runs/35845901520/job/107134278444

paimon-parquet-format-test crashes with SIGSEGV right after VariantParquetTest.WriteAndReadRoundTrip passes, while VariantParquetTest.ShreddedWriteAndReadRoundTrip is starting:

[ RUN      ] VariantParquetTest.ShreddedWriteAndReadRoundTrip
build_support/run-test.sh: line 98: 67868 Segmentation fault      (core dumped)
Program terminated with signal SIGSEGV, Segmentation fault.

Thread 1 (Arrow IO thread pool):
#0 arrow::PoolBuffer::~PoolBuffer()
#1 arrow::Future<std::shared_ptr<arrow::Buffer>>::SetResult(...)::{lambda(void*)#1}::_FUN(void*)
#2 arrow::ConcreteFutureImpl::~ConcreteFutureImpl()
#3 std::_Sp_counted_base<...>::_M_release_last_use_cold()
#4 arrow::internal::FnOnce<void ()>::FnImpl<std::_Bind<arrow::detail::ContinueFuture (arrow::Future<std::shared_ptr<arrow::Buffer>>, arrow::io::RandomAccessFile::ReadAsync(arrow::io::IOContext const&, long, long)::{lambda()#1})>>::~FnImpl()
#5 arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1} ...

The main thread is already running the next test (VariantShreddingWritePlan::CreateFromPhysicalSchema) and is unrelated to the crash.

What doesn't meet your expectations?

The test binary should not crash. The crash is a use-after-free of the Arrow memory pool in the test fixture:

  1. VariantParquetTest::SetUp() creates a per-test pool adaptor: arrow_pool_ = GetArrowPool(pool_); (src/paimon/format/parquet/variant_parquet_test.cpp). It is destroyed together with the fixture.
  2. Several tests (WriteAndReadRoundTrip, ShreddedWriteAndReadRoundTrip, and the helper around line 548) open the file for a raw sanity check with the plain Arrow reader: 
    auto file = arrow::io::ReadableFile::Open(file_path_, arrow_pool_.get());
::parquet::arrow::OpenFile(file.ValueOrDie(), arrow_pool_.get(), &raw_reader);
    Only the raw arrow::MemoryPool* is passed, so nothing keeps the adaptor alive.
  3. With Arrow 17, ArrowReaderProperties::pre_buffer() defaults to true, so ReadTable issues RandomAccessFile::ReadAsync. ReadableFile does not override it, so the base implementation submits ReadAt to the IO thread pool and the result PoolBuffer is allocated from arrow_pool_.
  4. The IO worker holds the last reference to the future (and therefore the buffer) until its task object is destroyed, which can happen after the reader has consumed the data and the test has finished. When the fixture is torn down first, ~PoolBuffer() calls Free() on the already destroyed adaptor -> SIGSEGV.

Paimon's own read path (ArrowInputStreamAdapter::ReadAsync) is not affected: it uses a callback-based implementation and already retains the pool with the returned buffer (#182). This issue is limited to tests that use arrow::io::ReadableFile with a raw pointer to a short-lived pool.

Anything else?

Possible fixes (test-only):

  • Use a process-lifetime Arrow pool for these raw sanity-check readers, e.g. arrow::default_memory_pool(), instead of the per-fixture arrow_pool_.get(); or
  • Disable pre-buffering for the raw reader (ArrowReaderProperties::set_pre_buffer(false)), so no async IO task outlives the test; or
  • Read through ArrowInputStreamAdapter, which keeps the pool alive for returned buffers.

Other tests using arrow::io::ReadableFile::Open(..., pool.get()) with a per-test pool may have the same latent issue and should be checked as well.

Vorherrschende Sprache
C++
Sterne
65
Forks
29
Ø Merge
2 T. 5 Std.
Gemergte PRs (30 T.)
77

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus apache/paimon-cpp

Alle Issues in apache/paimon-cpp

Ähnliche Issues

Weitere Issues zu C++

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.