ClickHouse/ClickHouse

using sumMap efficiently

Offen

#62.535 geöffnet am 11.04.2024

 (5 Kommentare) (0 Reaktionen) (0 zugewiesene Personen)C++ (8.400 Forks)batch import
help wantedquestion

Repository-Metriken

Stars
 (47.419 Sterne)
PR-Merge-Metriken
 (Durchschn. Merge 2T 2h) (1.000 gemergte PRs in 30 T)

Beschreibung

I am having trouble understanding and finding the correct way to use sumMap efficiently both ön materialized input and ordered (limited output). Given the following;

create table log (
  timestamp DateTime,
  host String,
  url String,
  referer String
) ENGINE MergeTree order by (host, timestamp)

create table log_hourly (
  hour DateTime,
  host String,
  urlMap Nested(
    url String,
    cnt UInt32
  ),
  refererMap Nested(
    referer String,
    cnt UInt32
  )
) ENGINE SummingMergeTree order by (host, hour)

what would be a better way to write a query to populate log_hourly? My current query is:

    SELECT
        toStartOfHour(timestamp) AS hour,
        host,
        groupArray(url),
        groupArray(1) uc,
        groupArray(referer),
        groupArray(1) AS oc,
    FROM log
    GROUP BY
        host,
        hour

Also, when populated, how would I select the top 10 elements (with cnt) of each (urlMap, refererMap), my try was:

SELECT
    host,
    arrayResize(arrayReverseSort((x, y) -> y, untuple(sumMap(urlMap.url, urlMap.cnt))), 10),
    arrayResize(arrayReverseSort(sumMap(urlMap.url, urlMap.cnt).2), 10)
    arrayResize(arrayReverseSort((x, y) -> y, untuple(sumMap(refererMap.referer, refererMap.cnt))), 10),
    arrayResize(arrayReverseSort(sumMap(refererMap.referer, refererMap.cnt).2), 10)
FROM hourly
GROUP BY host

but I would assume there are far better ways of accomplishing the same. What am I missing?

Contributor Guide