date_part returning wrong results due to overflows

Open Beginner friendly
#14,738 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust, sql
Domain
databases

Research direction

Start by running the supplied date_part SQL query and inspect datafusion/functions/src/datetime/date_part.rs, especially the calculation around line 290. Trace the integer types involved and add regression coverage for the reproducer; done means the query returns the same result as PostgreSQL without overflow or panic.

Written by the indexing model from the issue text.

Description

bug
Describe the bug

When playing with the date_part function, I see that there's ways of triggering int32 multiplication overflows that either panic on a debug build, or return the wrong number at runtime.

To Reproduce

Executing the following statement shows the behavior.

SELECT date_part('microsecond', timestamp '1970-01-01T00:40:00' - timestamp '1970-01-01T00:00:00')

DataFusion fiddle link <- returns a wrong random number

Postgres fiddle link <- returns 0

Expected behavior

The date_part function should behave the same as Postgres

Additional context

Not 100% sure, but I would say that the changes introduced in https://github.com/apache/datafusion/pull/13466 look suspicious. There, the inner calculations are using int32 types, which are easy to overflow. Special mention to this line of code:
https://github.com/gabotechs/datafusion/blob/763bd681f09d58ce285ab3a677b81291c41adfce/datafusion/functions/src/datetime/date_part.rs#L290-L290

Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 8h
Merged PRs (30d)
354

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datafusion

All issues in apache/datafusion

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.