PerformancePythoncuIOfeature requestgood first issuelibcudf
仓库指标
- 星标
- (6,000 个星标)
- PR 合并指标
- (平均合并 17天 21小时) (30 天内合并 230 个 PR)
描述
libcudf's multibyte_split has multiple ways to specify input data source when reading from a file:
-
auto const source = cudf::io::text::make_source_from_file(temp_file_name); // no GDS -
auto const datasource = cudf::io::datasource::create(temp_file_name);auto const source = cudf::io::text::make_source(*datasource); // supports GDS
cudf.read_text uses the first option to create the data_chunk_source object, so it does not benefit from potential direct reads with GDS.
cudf.read_text should create a cudf::io::datasource object to improve performance on GDS-enabled systems.