apache/arrow-java

Performance concern: fillHoles() method and read buffer expansion efficiency.

開放

#599 建立於 2025年2月10日

 (9 則留言) (3 個反應) (0 位負責人)Java (132 個分叉)auto 404
Type: enhancementhelp wanted

倉庫指標

星標
 (89 顆星)
PR 合併指標
 (PR 指標待抓取)

描述

Hi everyone,

I have the following use case: I’m benchmarking the read throughput performance when dealing with a large number of non-dictionary string columns (300 columns). Based on the profiler output (see the attached picture), I’ve noticed that a significant amount of time is spent in the fillHoles() method, which is part of the read buffer expansion process.

My question is: why is the buffer filled one element at a time instead of using a bulk operation? Wouldn’t a batch approach be more efficient?

Looking forward to your insights. Thanks!

Image

貢獻者指南