[Feature][PPL] Production hardening and validation for progressive query execution

Abierto
  • ¿Algo no está bien?
#5,801 4 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
30/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
java
Área
backend, databases, performance

Línea de trabajo

The issue is about hardening the Progressive Query Execution (PPL) feature. Start by reviewing the parent issue #5765 and the existing ProgressiveQueryExecution contract. Examine the prototype code and integration seams. Run existing Calcite integration tests and YAML REST tests to understand the baseline. The goal is to define memory limits, make partial-reduce cadence configurable, and validate cleanup, FGAC, cancellation, and shutdown scenarios.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

async calcite enhancement PPL testing

Parent: #5765

Goal

Turn the validated asynchronous and partial-result prototypes into a production-quality implementation with explicit module boundaries, bounded resource use, and complete regression coverage.

Scope

  • Keep ProgressiveQueryExecution as the only lifecycle-facing execution contract.
  • Remove PoC-only routing and physical-plan hooks in favor of explicit production integration seams.
  • Keep execution callback, Context, StateStore, and aggregation observer details out of the lifecycle layer.
  • Define memory limits and failure behavior for APPEND and REPLACE state.
  • Make partial-reduce cadence configurable and bounded.
  • Validate cleanup, owner-node loss, FGAC, cancellation, expiration, and shutdown.
  • Document supported partial-result shapes and progress semantics.

Acceptance criteria

  • No synchronous behavior or performance regression outside the accepted threshold.
  • All existing Calcite integration tests and YAML REST tests pass.
  • A REST matrix covers non-blocking rows, Composite aggregation, fully pushed aggregation, pushed aggregation with Calcite post-processing, blocking plans, and failure cases.
  • Every final async result is equivalent to the synchronous result.
  • First-data and final latency, memory usage, and aggregation reduce overhead are reported on a multi-shard large dataset.
  • Resource cleanup and lifecycle race tests show no leaked tasks, search contexts, statements, or StateStores.
Lenguaje dominante
Java
Estrellas
175
Forks
233
Merge medio
3 d 36 min
PR fusionados (30 d)
41

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de opensearch-project/sql

Todos los issues de opensearch-project/sql

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.