ClickHouse/ClickHouse

using sumMap efficiently

開放

#62,535 建立於 2024年4月11日

 (5 則留言) (0 個反應) (0 位負責人)C++ (8,400 個分叉)batch import
help wantedquestion

倉庫指標

星標
 (47,419 顆星)
PR 合併指標
 (平均合併 2天 2小時) (30 天內合併 1,000 個 PR)

描述

I am having trouble understanding and finding the correct way to use sumMap efficiently both ön materialized input and ordered (limited output). Given the following;

create table log (
  timestamp DateTime,
  host String,
  url String,
  referer String
) ENGINE MergeTree order by (host, timestamp)

create table log_hourly (
  hour DateTime,
  host String,
  urlMap Nested(
    url String,
    cnt UInt32
  ),
  refererMap Nested(
    referer String,
    cnt UInt32
  )
) ENGINE SummingMergeTree order by (host, hour)

what would be a better way to write a query to populate log_hourly? My current query is:

    SELECT
        toStartOfHour(timestamp) AS hour,
        host,
        groupArray(url),
        groupArray(1) uc,
        groupArray(referer),
        groupArray(1) AS oc,
    FROM log
    GROUP BY
        host,
        hour

Also, when populated, how would I select the top 10 elements (with cnt) of each (urlMap, refererMap), my try was:

SELECT
    host,
    arrayResize(arrayReverseSort((x, y) -> y, untuple(sumMap(urlMap.url, urlMap.cnt))), 10),
    arrayResize(arrayReverseSort(sumMap(urlMap.url, urlMap.cnt).2), 10)
    arrayResize(arrayReverseSort((x, y) -> y, untuple(sumMap(refererMap.referer, refererMap.cnt))), 10),
    arrayResize(arrayReverseSort(sumMap(refererMap.referer, refererMap.cnt).2), 10)
FROM hourly
GROUP BY host

but I would assume there are far better ways of accomplishing the same. What am I missing?

貢獻者指南