[BUG] PPL subsearch maxout settings do not apply on the Analytics Engine path (diverges at defaults)

Abierto
  • ¿Algo no está bien?
#5,735 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Error
Claridad
Necesita aclaración
Estado de actividad
Activo
Stack tecnológico
java
Área
api, backend

Línea de trabajo

Comienza leyendo los valores predeterminados documentados en UnifiedQueryContext.Builder y la prueba de clasificación everySeededPlanningSettingIsClassified; después, inspecciona RestUnifiedQueryAction.FORWARDED_CLUSTER_SETTINGS y el contexto de #5611. El issue estará terminado cuando los maintainers decidan si las solicitudes REST reenvían ambas configuraciones o se documente explícitamente la divergencia, y queden registrados la prueba relevante y la decisión sobre las release notes.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

analytic-engine bug design PPL
What is the bug?

On the Analytics Engine (unified query) path, PPL subsearches run unbounded, while the default (non-AE) pipeline caps them. Unlike the other settings-fidelity gaps in this area, this one diverges at defaults — no operator configuration is required to hit it.

Setting Cluster default AE path value
plugins.ppl.subsearch.maxout 10000 0 (unlimited)
plugins.ppl.join.subsearch_maxout 50000 0 (unlimited)

Verified on a default AE context:

SysLimit[querySizeLimit=10000, subsearchLimit=0, joinSubsearchLimit=0]

SysLimit.fromSettings reads both keys, and 0 means "no LogicalSystemLimit injected", so nothing bounds the subsearch.

Root cause — and why this one is a design question, not a plain bug

This is deliberate, and documented in UnifiedQueryContext.Builder:

Settings.Key#PPL_JOIN_SUBSEARCH_MAXOUT defaults to 0 to avoid injecting LogicalSystemLimit into the logical plan, which is an OpenSearch-specific operational concern irrelevant to external consumers of the unified query API. Settings.Key#PPL_SUBSEARCH_MAXOUT is set to 0 for the same reason.

That rationale holds for external consumers of the unified query API as a library (e.g. planning PPL for a non-OpenSearch execution target), where an OpenSearch operational cap is indeed meaningless.

It does not obviously hold for the in-cluster REST path, where the query is being served by an OpenSearch node, consumes that node's memory, and the operator has configured a cap expecting it to apply. Two queries against the same cluster get different memory-safety guarantees purely based on whether the target index happens to use the composite data format.

What is the expected behavior?

Needs a maintainer decision between:

Option A — forward from the REST handler. Add both keys to RestUnifiedQueryAction.FORWARDED_CLUSTER_SETTINGS. This preserves the library default of "unlimited" for external API consumers (who never go through the REST handler) while restoring cluster fidelity in-cluster. Minimal change, and it keeps the documented rationale intact for the case it was written for.

Caveat: this is a behavior change at defaults — AE subsearches would go from unbounded to capped at 10000 / 50000. Queries relying on the current unbounded behavior would start hitting the cap. Needs a release note.

Option B — keep as-is. Accept that AE subsearches are unbounded by design, and document the divergence explicitly so operators know the cap does not apply to composite indices.

Do you have any additional context?

Found while fixing the same class of defect in #5611 (settings the AE path silently ignored). That PR forwards plugins.query.size_limit, plugins.ppl.pattern.*, and plugins.ppl.values.max.limit, and explicitly excludes these two keys pending this decision — the exclusion is pinned by a test (everySeededPlanningSettingIsClassified) so it stays a conscious choice rather than drift.

Related: #5734 (plugins.calcite.all_join_types.allowed guardrail inactive on the AE path).

Lenguaje dominante
Java
Estrellas
175
Forks
233
Merge medio
3 d 3 h
PR fusionados (30 d)
39

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de opensearch-project/sql

Todos los issues de opensearch-project/sql

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.