[BUG] sort, stats max, and stats min on an object or nested field return 500

Start with AbstractCalciteIndexScan.pushDownSort and AggregateAnalyzer, then trace how BackgroundSearchScanner and RestPPLQueryAction handle the shard error; reproduce using the supplied mapping and PPL queries. Done means sort and stats max/min on object or nested fields no longer surface as HTTP 500, with behavior matching the expected 400 or pushdown-off results.

Query Information

PPL Command/Query:

source=logs_a | sort f

f is an object field, which PPL reports as struct. The same 500 comes back when f is nested, geo_point or flat_object.

Expected Result: HTTP 400 naming the field and its type. The v2 engine, with plugins.calcite.enabled=false, returns 400 for sort f on all four types.

Actual Result: HTTP 500.

{
  "error": {
    "context": {
      "stage": "executing",
      "stage_description": "Running the query"
    },
    "reason": "java.sql.SQLException: exception while executing query: Failed to fetch data from the index: the background task failed or interrupted.\n  Inner error: Failed to execute phase [query], all shards failed; shardFailures {[sKGeDUTgTzGvZ14nx-1U0g][logs_a][0]: RemoteTransportException[[80a9971e0936][127.0.0.1:9310][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [f] in order to sort on]; }",
    "details": "java.sql.SQLException: exception while executing query: Failed to fetch data from the index: the background task failed or interrupted.\n  Inner error: Failed to execute phase [query], all shards failed; shardFailures {[sKGeDUTgTzGvZ14nx-1U0g][logs_a][0]: RemoteTransportException[[80a9971e0936][127.0.0.1:9310][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [f] in order to sort on]; }",
    "location": ["while running the query"],
    "code": "UNKNOWN",
    "type": "RuntimeException"
  },
  "status": 500
}

Every sort on f returns 500 for all four types.

Query	object	nested	geo_point	flat_object
sort f	500	500	500	500
sort - f	500	500	500	500
sort f | head 1	500	500	500	500
sort f | reverse	500	500	500	500
fields f	200	200	200	200

Each 500 carries a shard error that the same sort, sent to _search directly as {"sort":[{"f":{"order":"asc","missing":"_first"}}]}, returns as HTTP 400.

f	Direct _search
object	400, No mapping found for [f] in order to sort on
nested	400, No mapping found for [f] in order to sort on
geo_point	400, can't sort on geo_point field without using specific sorting feature, like geo_distance
flat_object	400, Field [f] of type [flat_object] does not support doc_value in root field

Over a wildcard sort f also returns 500 when f is object in every index, when it is object in one index and nested in another, and when the other indices do not map it.

Dataset Information

Dataset/Schema Type

• OpenTelemetry (OTEL)
• Simple Schema for Observability (SS4O)
• Open Cybersecurity Schema Framework (OCSF)
• Custom (details below)

Index Mapping

PUT logs_a { "mappings": { "properties": { "host": { "type": "keyword" }, "f": { "properties": { "code": { "type": "keyword" } } } } } }

The other three types use the same index with f mapped as { "type": "nested", "properties": { "code": { "type": "keyword" } } }, { "type": "geo_point" } or { "type": "flat_object" }.

Sample Data

POST logs_a/_doc/1?refresh=true  { "host": "a", "f": { "code": "USD" } }

The geo_point index holds { "lat": 2, "lon": 2 } and the nested index holds [ { "code": "USD" } ]. source=logs_a | fields f returns 200 for all four, so reading the field works.

Bug Description

Issue Summary

OpenSearch cannot sort on any of these four types, and the shard rejects such a sort with a 400. An object field has an ObjectMapper but no MappedFieldType, so FieldSortBuilder.resolveUnmappedType throws QueryShardException, while geo_point and the root of a flat_object throw IllegalArgumentException. PPL plans sort f without checking the field type, and AbstractCalciteIndexScan.pushDownSort pushes it as a field sort. BackgroundSearchScanner.getCurrentResponse, Avatica, OpenSearchExecutionEngine and StageErrorHandler wrap the SearchPhaseExecutionException in turn, so RestPPLQueryAction.getRawErrorCode, which unwraps only the ErrorReport, finds a RuntimeException and returns 500.

Steps to Reproduce

curl -s -XPUT "localhost:9200/logs_a" -H 'Content-Type: application/json' \
  -d '{"mappings":{"properties":{"host":{"type":"keyword"},"f":{"properties":{"code":{"type":"keyword"}}}}}}'
curl -s -XPOST "localhost:9200/logs_a/_doc/1?refresh=true" -H 'Content-Type: application/json' -d '{"host":"a","f":{"code":"USD"}}'

# 1. sort on the object field returns HTTP 500
curl -s -XPOST "localhost:9200/_plugins/_ppl" -H 'Content-Type: application/json' \
  -d '{"query":"source=logs_a | sort f"}' -w '\nHTTP %{http_code}\n'

# 2. The sort that PPL pushes, sent as a raw search, returns HTTP 400
curl -s -XPOST "localhost:9200/logs_a/_search" -H 'Content-Type: application/json' \
  -d '{"sort":[{"f":{"order":"asc","missing":"_first"}}]}' -w '\nHTTP %{http_code}\n'

# 3. A geo_point field returns HTTP 500 the same way
curl -s -XPUT "localhost:9200/logs_g" -H 'Content-Type: application/json' \
  -d '{"mappings":{"properties":{"host":{"type":"keyword"},"f":{"type":"geo_point"}}}}'
curl -s -XPOST "localhost:9200/logs_g/_doc/1?refresh=true" -H 'Content-Type: application/json' -d '{"host":"a","f":{"lat":2,"lon":2}}'
curl -s -XPOST "localhost:9200/_plugins/_ppl" -H 'Content-Type: application/json' \
  -d '{"query":"source=logs_g | sort f"}' -w '\nHTTP %{http_code}\n'

Impact

A sort on an object, nested, geo_point or flat_object field, alone or followed by head or reverse, returns a 500 whose message names an OpenSearch shard failure and a node address. A client cannot tell that request apart from a server fault, and a retry fails the same way.

Environment Information

OpenSearch Version: main at c640cb921, 3.10.0-SNAPSHOT, one node, with the defaults plugins.calcite.enabled=true, plugins.calcite.pushdown.enabled=true and plugins.calcite.fallback.allowed=false.

Additional Details:

No open or closed issue tracks this.