`avg` over `Duration` panics with "attempt to add with overflow" in debug builds

Open Beginner friendly
#24,894 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust
Domain
data-engineering

Research direction

Start in datafusion/functions-aggregate/src/average.rs and inspect DurationAvgAccumulator::update_batch, merge_batch, and retract_batch alongside add_avg_sum. Run the provided debug-build datafusion-cli SQL reproduction, then verify that duration averages wrap on overflow like sum and the other avg accumulators without panicking.

Written by the indexing model from the issue text.

Description

Describe the bug

DurationAvgAccumulator (datafusion/functions-aggregate/src/average.rs) keeps its running sum with plain += in update_batch / merge_batch and plain - in retract_batch. When the sum of the input durations does not fit in i64 this panics in debug builds and wraps silently in release builds.

The sum aggregate and the other avg accumulators deliberately wrap on overflow (see the comment on add_avg_sum: "Wraps on overflow, matching the sum aggregate and arrow::compute::sum"), so the Duration accumulator is the odd one out.

To Reproduce

Debug build of datafusion-cli:

SELECT avg(x) OVER (ROWS BETWEEN 1 PRECEDING AND CURRENT ROW)
FROM (VALUES (arrow_cast(9223372036854775807, 'Duration(Second)')),
             (arrow_cast(9223372036854775807, 'Duration(Second)'))) t(x);

thread 'main' panicked at datafusion/functions-aggregate/src/average.rs:887:13:
attempt to add with overflow

The plain aggregate SELECT avg(x) FROM ... with the same values hits the same line.

Expected behavior

Consistent with sum and the other avg accumulators: wrap, never panic.

Additional context

Found while running a corpus of extreme-value literals against a debug build of datafusion-cli.

Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 8h
Merged PRs (30d)
354

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datafusion

All issues in apache/datafusion

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.