[Bug] ORCA: QD segfault in CExtendedStatsProcessor when extended statistics (dependencies) do not cover all filtered columns
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 74/100
Línea de trabajo
Reproduce el ejemplo de SQL con ORCA habilitado y luego lee libnaucrates/src/statistics/CExtendedStatsProcessor.cpp alrededor de CExtendedStatsProcessor::ApplyCorrelatedStatsToScaleFactorFilterCalculation, siguiendo los llamadores a través de CFilterStatsProcessor::MakeHistHashMapConjFilter y CLogicalSelect::PstatsDerive. Se considera terminado cuando el caso de estadísticas extendidas parcialmente cubierto ya no desreferencia una asignación de columna inexistente, el backend no produce un segfault y la consulta devuelve 0 filas.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Apache Cloudberry version
main branch (recent build)
What happened
With ORCA enabled, the backend crashes when a query filters on a set of columns that is only partially covered by an extended statistics object (CREATE STATISTICS ... (dependencies)). In the reproducer below the statistics object covers c0, c1 and the outer WHERE also references c2, which is not covered.
The crash is a null-pointer dereference in the QD backend inside ORCA's statistics derivation:
CExtendedStatsProcessor::ApplyCorrelatedStatsToScaleFactorFilterCalculation
(libnaucrates/src/statistics/CExtendedStatsProcessor.cpp, ~line 366)
<- CFilterStatsProcessor::MakeHistHashMapConjFilter
<- CLogicalSelect::PstatsDerive
colid_to_attno_mapping->Find(&colid) returns nullptr for the column that is not part of the statistics object, and the result is dereferenced without a check.
With optimizer = off the query runs fine and returns 0 rows.
Found by SQLancer.
What you think should happen instead
The query should run and return 0 rows (no row has c1 = ''). Cardinality estimation must not assume every filtered column is present in the extended statistics object.
How to reproduce
CREATE TABLE t3 (c0 boolean, c1 text, c2 int) DISTRIBUTED BY (c0);
INSERT INTO t3 SELECT (g%2=0), 'x'||g, g FROM generate_series(1,100) g;
CREATE STATISTICS s0 (dependencies) ON c0, c1 FROM t3; -- covers c0, c1 only
ANALYZE t3;
SET optimizer = on;
SELECT * FROM (SELECT ALL t3.c0 AS t3c0, t3.c1 AS t3c1, t3.c2 AS t3c2
FROM t3
WHERE (t3.c0) IS TRUE
GROUP BY t3.c0, t3.c1, t3.c2
ORDER BY t3.c1) AS result
WHERE result.t3c0 = TRUE AND result.t3c1 = '' AND result.t3c2 > 0; -- c2 not covered by s0
-- QD backend SIGSEGV
Operating System
Linux
Anything else
Found while running SQLancer against Cloudberry main (3-segment demo cluster, --enable-cassert). See the proposal to run SQLancer continuously: https://github.com/apache/cloudberry/discussions/1952
Are you willing to submit PR?
- Yes, I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
- Lenguaje dominante
- C
- Estrellas
- 1.4k
- Forks
- 248
- Merge medio
- 4 d 10 h
- PR fusionados (30 d)
- 40
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de apache/cloudberry
-
type: Bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
apache/cloudberry#1885 · 2 reacciones ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 86/100
apache/cloudberry#1825 ·
-
type: Bug
Dificultad 3/5 1-2 días Aptitud para principiantes 65/100
apache/cloudberry#2048 · 1 reacción ·
-
type: Bug
Dificultad 4/5 3-5 días Aptitud para principiantes 40/100
apache/cloudberry#2047 ·
-
type: Bug
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
apache/cloudberry#2046 · 1 comentario ·
Todos los issues de apache/cloudberry
Issues similares
-
task
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
vsanthanam/JBird#429 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
-
bug documentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
es-ude/OnDeviceTraining#459 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
bilelmoussaoui/gobject-linter#199 · 1 comentario ·
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
bradcypert/plum#53 ·