rapidsai/cudf
在 GitHub 查看[FEA] Explicitly guarantee row group ordering in the parquet reader.
Open
#15,697 创建于 2024年5月7日
cuIOfeature requestgood first issueimprovementlibcudf
仓库指标
- Star
- (6,000 star)
- PR 合并指标
- (平均合并 17天 21小时) (30 天内合并 230 个 PR)
描述
From @devavret , the question came up as to whether we guarantee the relative ordering of row groups across multiple input files in the parquet reader. That is, if you have two files [f1, f2] and the row groups within the files (in one column) are specified as [[r0,r3], [r0,r1]], do we guarantee the output ordering would be [f1r0, f1r3, f2r0, f2r1]
The code does in fact do this for both the explicitly specified case and the unspecified (empty user input / all row groups), but we don't make any guarantees about it. Seems like a safe and easy thing to add.