[Bug] atan2 does not normalize -0.0 the way Spark does

Open Beginner friendly
  • Is something wrong?
#26,333 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@galaxytemple is already working on this.

Since Oct 31, 2026.

  • #26394 by @spicysharkco — closed without merging
  • #26900 by @pujitha24 — closed without merging
  • #26963 by @galaxytemple — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust, sql
Domain
databases

Research direction

The fix is in SparkAtan2 in datafusion/spark/src/function/math/atan2.rs, where the raw arguments go straight to f64::atan2; add 0.0 to both operands first, as the issue describes. The expectation to change is at datafusion/sqllogictest/test_files/spark/math/atan2.slt line 155, from -3.14159 to 3.141592653589793. Run that sqllogictest file, and check that atan2(0.0, -0.0) and atan2(-0.0, -0.0) match the Spark results the issue reports before finishing.

Written by the indexing model from the issue text.

Description

bug spark
Describe the bug

SparkAtan2 passes the raw arguments to Rust's f64::atan2 (atan2.rs#L82). Spark's Atan2 adds 0.0 to both arguments first, which turns a negative zero into a positive zero (mathExpressions.scala#L1229-L1237). The Spark source explains why: "With codegen, the values returned by -0.0 and 0.0 are different. Handled with +0.0".

So atan2 returns a different value whenever y or x is -0.0.

To Reproduce

Spark 4.0.0:

SELECT atan2(CAST('-0.0' AS DOUBLE), -1.0D)
-- 3.141592653589793
SELECT atan2(0.0D, CAST('-0.0' AS DOUBLE))
-- 0.0
SELECT atan2(CAST('-0.0' AS DOUBLE), CAST('-0.0' AS DOUBLE))
-- 0.0

DataFusion: the existing test at atan2.slt#L155 documents the C99 result:

query R
SELECT atan2(-0.0::double, -1.0::double)
----
-3.141592653589793

The other two inputs were not run on DataFusion. By the C99 definition that Rust follows, atan2(0.0, -0.0) is pi and atan2(-0.0, -0.0) is -pi.

Expected behavior

Add 0.0 to both arguments before calling atan2, as Spark does, and change the expectation at atan2.slt#L155 to 3.141592653589793.

Additional context

Found by running the queries in datafusion/sqllogictest/test_files/spark on a local Spark and comparing the results with the expected values in the files, which is the kind of check proposed in #17045. Spark results come from Apple's build of Spark 4.0.0 (4.0.0.65-apple) with a UTC session and spark.sql.ansi.enabled=false unless stated. The Spark source links point at the v4.0.0 tag. DataFusion results are the sqllogictest expectations at 9dec77584, which CI enforces. Anything marked as not run was derived from reading the code.

Dominant language
Rust
Stars
9.6k
Forks
2.6k
Avg merge
3d 22h
Merged PRs (30d)
331

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datafusion

All issues in apache/datafusion

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.