rapidsai/cudf

Add `peak_memory_usage` to all nvbench benchmarks

已關閉

#10,528 建立於 2022年3月28日

 (12 則留言) (0 個反應) (1 位負責人)C++ (735 個分叉)batch import
0 - BacklogPerformancefeature requestgood first issuelibcudf

倉庫指標

星標
 (6,000 顆星)
PR 合併指標
 (平均合併 17天 21小時) (30 天內合併 230 個 PR)

描述

Update (2023-12-05): Adding peak_memory_usage to nvbenchmarks can be accomplished with this pattern:

auto const mem_stats_logger = cudf::memory_stats_logger(); 

state.exec( ... );

state.add_buffer_size(
    mem_stats_logger.peak_memory_usage(), 
    "peak_memory_usage", 
    "peak_memory_usage"
);

Please consult the cuIO benchmarks for how to add peak memory tracking. If we are tracking peak memory usage as well as bytes per second, then we can estimate memory footprint across the libcudf API.

Original issue: #7770 added support for peak memory usage to cuIO benchmarks using rmm's statistics_resource_adapter. It would be nice to be able to expand that to all of our benchmarks so that we could more easily detect regressions in memory usage. This would be particularly useful for the Dask cuDF team, which is always looking to identify bottlenecks from memory usage. There was already discussion of doing this in #7770, so we should investigate following up now.

貢獻者指南