[BUG] Analytics routing does not resolve aliases and wildcard index expressions

Abierto
  • ¿Algo no está bien?
#5,916 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@KarstenSchnitter ya está trabajando en esto.

Desde el 29/10/2026.

  • #5917 de @KarstenSchnitter — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
25/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
java
Área
backend, databases

Línea de trabajo

Start with RestUnifiedQueryAction.isAnalyticsIndex() and trace how SQL and PPL source expressions are extracted and routed. The issue names no test files, so locate the routing tests and run the relevant SQL and PPL suites. Done means aliases, wildcards, comma-separated sources, and data streams resolve to the intended engine while preserving missing-target behavior, system and REST-source handling, and authorization; the linked open PR indicates work is already underway.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

analytic-engine bug

What is the bug?

RestUnifiedQueryAction.isAnalyticsIndex() determines whether SQL and PPL queries should use the analytics engine by looking up the extracted source as a concrete index name.

Aliases, wildcard expressions, and some multi-index expressions are not concrete index names. Although they may resolve exclusively to composite/Parquet-backed indices, the lookup fails and the query silently falls back to the legacy execution path.

This is particularly visible when columnar storage is enabled per index rather than through the cluster-wide cluster.pluggable.dataformat=composite setting.

SQL PR #5683 documents the same routing gap. It uses the cluster-wide setting in its security test cluster to force all queries through the analytics engine, but does not resolve index expressions for per-index routing.

How can one reproduce the bug?

Run OpenSearch with the SQL plugin and sandbox analytics/composite plugins enabled. Do not set the cluster-wide composite data-format setting.

Create a per-index columnar index and alias:

PUT parquet-logs-000001
{
  "settings": {
    "index.pluggable.dataformat.enabled": true,
    "index.pluggable.dataformat": "composite",
    "index.composite.primary_data_format": "parquet",
    "index.composite.secondary_data_formats": ["lucene"]
  },
  "mappings": {
    "properties": {
      "service": {
        "type": "keyword",
        "index": false
      }
    }
  },
  "aliases": {
    "parquet-logs": {
      "is_write_index": true
    }
  }
}

Index and refresh a document:

POST parquet-logs/_doc
{
  "service": "checkout"
}

POST parquet-logs/_refresh

Run equivalent profiled PPL queries:

POST /_plugins/_ppl
{
  "query": "source = parquet-logs-000001 | fields service",
  "profile": true
}

POST /_plugins/_ppl
{
  "query": "source = parquet-logs | fields service",
  "profile": true
}

POST /_plugins/_ppl
{
  "query": "source = parquet-logs-* | fields service",
  "profile": true
}

The concrete-index query is routed to the analytics engine. The alias and wildcard queries fall back to the legacy path because the source expression is passed to a direct metadata lookup without first being resolved.

Depending on the mapping and query, the fallback can produce an error, omit columnar field values, or return a response whose profile shows that the legacy engine handled the query.

The same problem applies to SQL table expressions.

What is the expected behavior?

Routing should resolve aliases, wildcard expressions, comma-separated sources, and data streams to concrete indices before selecting an execution engine.

Suggested semantics:

  • A non-empty resolution containing only compatible composite indices uses the analytics engine.
  • A resolution containing only ordinary Lucene indices uses the normal SQL/PPL path.
  • Missing and empty expressions retain the normal request path and error behavior.
  • Mixed Lucene/composite targets must not be silently or partially routed. The desired behavior—normal fallback versus an explicit unsupported-target error—should be agreed upon.
  • System catalog and REST-source queries must retain their existing handling.
  • Security authorization must remain enforced for every resolved target.

What is your host/environment?

  • OpenSearch/SQL: current main targeting OpenSearch 3.10
  • Analytics engine and composite/Parquet sandbox plugins enabled
  • Columnar storage enabled per index

Additional context

Origin: opensearch-project/technical-steering#146

Related SQL PR: #5683

A possible implementation is to inject IndexNameExpressionResolver, resolve every extracted expression against the current cluster state, reject empty resolutions, and require every resolved index to have:

index.pluggable.dataformat.enabled = true
index.pluggable.dataformat = composite

Tests should cover:

  • Concrete index
  • Single- and multi-index aliases
  • Wildcards resolving entirely to columnar indices
  • Comma-separated columnar sources
  • Data streams
  • Empty wildcard expansions
  • Missing indices
  • Ordinary Lucene targets
  • Mixed Lucene and columnar targets
  • PPL and SQL query paths
Lenguaje dominante
Java
Estrellas
175
Forks
236
Merge medio
2 d 19 h
PR fusionados (30 d)
40

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de opensearch-project/sql

Todos los issues de opensearch-project/sql

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.