SQL API: aggregates don't skip NULL values (COUNT(col) counts them, SUM and AVG throw NPE)

Abierto
#891 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

@AarushShetty346 ya está trabajando en esto.

Desde el 18/11/2026.

  • #893 de @AarushShetty346 — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
20/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Estancado
Stack tecnológico
java, spark, sql

Línea de trabajo

Start with AggregateAddCols.java, AggregateFunction.java, and AggregateGetResult.java at the linked lines; trace how NULL values are handled through aggregation and how AVG gets its count. Reproduce the listed queries on the Java platform, including a GROUP BY case, and check the behavior on Spark. Done means COUNT(col), SUM, and AVG match the expected results while COUNT(*) still counts every row.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Aggregates in the SQL API treat NULL as a value. COUNT(col) counts NULL rows, and SUM / AVG throw a NullPointerException when a column has a NULL.

In SQL, every aggregate except COUNT(*) ignores NULL values. There are three spots where this goes wrong:

  1. AggregateAddCols.java:53 sets every COUNT column to 1, so COUNT(col) is always the same as COUNT(*).
  2. AggregateFunction.java:118-124 picks the type from whichever side isn't null, then still passes the null to the function. For example, Integer::sum is applied to (1, null) and fails on unboxing.
  3. AggregateGetResult.java:55-56 divides the AVG sum by the row count, not by the number of non-null values. AVG would therefore be wrong even without the NPE.
To reproduce
col1:int,col2:string
1;a
;b
3;c

On the Java platform:

query result expected
SELECT COUNT(*) FROM fs.exampleNull 3 3
SELECT COUNT(col1) FROM fs.exampleNull 3 2
SELECT SUM(col1) FROM fs.exampleNull NPE 4
SELECT AVG(col1) FROM fs.exampleNull NPE 2.0
java.lang.NullPointerException
    at org.apache.wayang.api.sql.calcite.converter.functions.AggregateFunction.castAndMap(AggregateFunction.java:127)
    at org.apache.wayang.api.sql.calcite.converter.functions.AggregateFunction.apply(AggregateFunction.java:58)

The same thing happens with GROUP BY when a group has both NULL and non-NULL values. It fails in JavaReduceBy, and in Spark too.

Tested on main at 713faa7. I have a fix with tests and will open a PR for it.

Lenguaje dominante
Java
Estrellas
284
Forks
147
Merge medio
2 d 13 h
PR fusionados (30 d)
13

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/wayang

Todos los issues de apache/wayang

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.