[BUG] Simple `top` queries on high-cardinality fields are very slow

報告されたエントリポイントは PPL の top クエリです。まず ppl --explain の論理プランと物理プラン、およびそれらの集約プッシュダウン、特に生成される composite_buckets リクエストを調べてください。表示されている terms/top_values リクエストと比較してください。完了の条件は、単純な top クエリが coordinator のスキャンを回避し、期待される上位結果を返すことです。

What is the bug?
Simple top queries can take more than 30s if done on fields with high cardinality (>1mil distinct values). This happens even if there's a small top limit such as the default top 10.

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Index data with lots of distinct request_id values (>1mil, keyword-typed)
2. source=idx | top request_id
3. Measure

Explain plan:

> echo 'source=logs | top request_id' | ppl --explain
= Calcite Plan =
== Logical ==
LogicalSystemLimit(fetch=[10000], type=[QUERY_SIZE_LIMIT])
  LogicalProject(request_id=[$0], count=[$1])
    LogicalFilter(condition=[=($2, 10)])
      LogicalProject(request_id=[$0], count=[$1], _row_number_rare_top_=[ROW_NUMBER() OVER (ORDER BY $1 DESC, $0)])
        LogicalAggregate(group=[{0}], count=[COUNT()])
          LogicalProject(request_id=[$60])
            CalciteLogicalIndexScan(table=[[OpenSearch, logs]])


== Physical ==
EnumerableLimit(fetch=[10000])
  EnumerableCalc(expr#0..2=[{inputs}], expr#3=[10], expr#4=[=($t2, $t3)], proj#0..1=[{exprs}], $condition=[$t4])
    EnumerableWindow(window#0=[window(order by [1 DESC, 0] rows between UNBOUNDED PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])])
      CalciteEnumerableIndexScan(table=[[OpenSearch, logs]], PushDownContext=[[AGGREGATION-rel#50:LogicalAggregate.NONE.[](input=RelSubset#49,group={0},count=COUNT())], OpenSearchRequestBuilder(sourceBuilder={
        "aggregations": {
          "composite_buckets": {
            "composite": {
              "size": 10000,
              "sources": [
                {
                  "request_id": {
                    "terms": {
                      "field": "request_id",
                      "missing_bucket": true,
                      "missing_order": "first",
                      "order": "asc"
                    }
                  }
                }
              ]
            }
          }
        },
        "from": 0,
        "size": 0,
        "timeout": "1m"
      }, requestedTotalSize=2147483647, pageSize=null, startFrom=0)])

The inner DSL in this case takes ~300ms (index cache disabled), while the PPL takes ~43800ms on my machine. However, it's also ordering keys lexically instead of by count using OpenSearch's own top_values aggregation.

What is the expected behavior?

At least this case of a trivial top aggregation (without trailing consuming commands) should be fast. OpenSearch itself can do this quickly (133 ms) with straightforward DSL, including returning an accurate size for the "other" bucket (sum_other_doc_count):

{
  "size": 0,
  "aggs": {
    "top_values": {
      "terms": {
        "field": "request_id",
        "size": 10
      }
    }
  }
}

{
    "took": 99,
    "timed_out": false,
    "terminated_early": true,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 10000,
            "relation": "gte"
        },
        "max_score": null,
        "hits": []
    },
    "aggregations": {
        "top_values": {
            "doc_count_error_upper_bound": 4,
            "sum_other_doc_count": 999990,
            "buckets": [
                {
                    "key": "00000067-142d-4d90-8a2c-28409600adbd",
                    "doc_count": 1
                },
                {
                    "key": "00001910-7d05-4e6a-9716-2c74137475e6",
                    "doc_count": 1
                },
                {
                    "key": "00002676-4f8c-428a-a29a-b862ca672cb7",
                    "doc_count": 1
                },
                {
                    "key": "00003943-8463-46f2-a92b-641b4afbafb1",
                    "doc_count": 1
                },
                {
                    "key": "00003f95-b049-41de-b9be-03aae071ad99",
                    "doc_count": 1
                },
                {
                    "key": "00005502-587d-4b00-865c-86f2af5502c2",
                    "doc_count": 1
                },
                {
                    "key": "00005754-ad22-4894-a9b5-dba3007ff19a",
                    "doc_count": 1
                },
                {
                    "key": "00005c50-97d3-4c11-ad74-90a3219ca815",
                    "doc_count": 1
                },
                {
                    "key": "000066c7-0301-45aa-98b1-d4997a590214",
                    "doc_count": 1
                },
                {
                    "key": "00007501-890e-4f0b-a688-2653ade97f8b",
                    "doc_count": 1
                }
            ]
        }
    }
}

What is your host/environment?

• mainline, also tested 3.5
• linux (AL2023), jdk 25

Do you have any screenshots?
N/A

Do you have any additional context?
Seems to be that we don't have any top_values aggregation pushdown at all, so this query turns into a coordinator scan.