TimeSeries: 256 KiB samples granules make PromQL windows older than the recent table up to 33% faster
Maintainer thường phản hồi trong vòng 1 ngày
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức phù hợp với người mới
- 35/100
- Loại issue
- Tính năng
- Độ rõ ràng
- Khá rõ ràng
- Mức độ hoạt động
- Sôi nổi
- Lĩnh vực
- databases, observability
Hướng nghiên cứu
The default granule size for the TimeSeries samples table is set in StorageTimeSeries.cpp, where samples_index_granularity_bytes is documented. The proposal changes that default from 524288 to 262144 and updates the gtest that checks it, the stateless tests 05140 and 05141, and test_index_granularity in test_prometheus_protocols/test_different_table_engines.py. Done means those references agree with the new default, and the benchmark regressions (up to 13% on some 7- and 14-day scans) are accepted by maintainers.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
Describe the situation
The TimeSeries samples table reads 512 KiB granules, about 135 hours of one 15 s series, even for a 1-hour window. With 256 KiB granules most queries over windows older than the recent table get 12-33% faster. Long scans don't change, except byte counters with random decimal digits: 6-13% slower.
Median client latency (ms) of 15 interleaved pairs on one server; ratio = patched/master, geometric mean with 95% CI.
| Query or cost | Compressible values | Integer byte counters | Byte counters with random decimals |
|---|---|---|---|
O19 rate(node_cpu_seconds_total[5m]), 1 h range 5 days back |
2127 → 1429, 0.683 (0.660-0.706) | 2054 → 1386, 0.678 (0.651-0.707) | 2044 → 1379, 0.679 (0.659-0.699) |
O23b sum(count_over_time(node_cpu_seconds_total[6h])), 5 days back |
2288 → 1527, 0.672 (0.661-0.682) | 2160 → 1468, 0.679 (0.668-0.690) | 2156 → 1462, 0.683 (0.673-0.694) |
O03 sum by (job) (rate(node_cpu_seconds_total{mode!="idle"}[5m])), 6 h range 5 days back |
2573 → 1988, 0.767 (0.753-0.782) | 2395 → 1882, 0.812 (0.760-0.867) | 2391 → 1875, 0.784 (0.776-0.793) † |
O14 sum by (job) (rate(http_requests_total[5m])), 5 days back |
67.1 → 52.6, 0.775 (0.748-0.804) † | 67.3 → 53.1, 0.823 (0.719-0.941) | 66.9 → 54.2, 0.830 (0.778-0.885) |
O17 sum(increase(node_context_switches_total[1h])), 5 days back |
54.1 → 42.6, 0.759 (0.706-0.816) | 62.2 → 49.4, 0.782 (0.748-0.818) | 61.9 → 49.9, 0.824 (0.796-0.851) |
OW sum by (job) (rate(http_requests_total[5m] offset 1w)), last hour |
73.8 → 57.8, 0.773 (0.733-0.816) | 73.9 → 58.4, 0.823 (0.743-0.913) † | 75.3 → 58.9, 0.788 (0.766-0.811) |
O02 rate(node_network_receive_bytes_total{device="eth0"}[5m]), 1 h range 5 days back |
64.6 → 52.7, 0.817 (0.802-0.833) | 62.0 → 54.7, 0.879 (0.853-0.907) | 72.2 → 62.7, 0.872 (0.840-0.907) |
| O01 the same for one series | 20.2 → 20.2, 0.999 (0.954-1.047) | 19.2 → 20.5, 1.050 (1.017-1.083) | 20.6 → 20.4, 0.987 (0.932-1.045) |
W1 sum by (instance) (rate(node_disk_written_bytes_total[5m])), 7 days |
322 → 312, 0.966 (0.873-1.069) | 359 → 344, 0.951 (0.928-0.974) | 367 → 401, 1.091 (1.071-1.112) |
W3 sum by (job) (rate(node_network_receive_bytes_total[5m])), 7 days |
400 → 383, 0.982 (0.948-1.017) | 379 → 368, 0.976 (0.956-0.996) | 465 → 500, 1.082 (1.063-1.101) |
W5 sum by (job) (rate(node_network_receive_bytes_total[1h])), 7 days |
447 → 437, 0.941 (0.881-1.006) | 426 → 424, 0.973 (0.923-1.026) † | 509 → 538, 1.059 (1.000-1.122) |
F1 sum by (job) (rate(node_network_receive_bytes_total[1h])), 14 days |
686 → 650, 0.963 (0.884-1.050) | 650 → 650, 0.979 (0.913-1.050) † | 770 → 837, 1.081 (1.013-1.154) |
F4 sum(increase(node_network_transmit_bytes_total[14d])) |
621 → 603, 0.985 (0.924-1.049) † | 602 → 586, 0.972 (0.916-1.031) | 697 → 767, 1.104 (1.057-1.154) |
F6 sum by (instance) (rate(node_disk_written_bytes_total[5m])), 14 days |
423 → 415, 0.970 (0.946-0.995) | 478 → 464, 0.980 (0.968-0.992) | 482 → 545, 1.125 (1.109-1.141) |
| Other 7-day ranges (W2, W4, W6, W7, W8) | 0.916-1.023 | 0.964-1.021 | 0.952-0.988 |
| Other 14-day queries (F2, F3) | 1.008-1.019 | 0.973-1.024 | 0.976-0.985 |
| Q02 / Q19 / Q14, last hour (recent samples table, not changed) | 0.980 / 1.023 / 1.001 | 1.083 (1.004-1.169) / 1.003 / 1.017 | 1.011 / 1.014 / 1.008 |
| All 24 queries once, sum of medians | 12.3 → 10.0 s, 0.815 | 11.8 → 9.8 s, 0.828 | 12.2 → 10.5 s, 0.862 |
| Cold, restart and dropped page cache before each request (6 pairs): O19 / W3 / F6 | 14.8 / 6.85 / 6.67 → 14.5 / 6.79 / 6.83 s, 0.995 / 0.994 / 1.013 | not measured | 17.9 / 5.28 / 6.04 → 13.5 / 5.52 / 6.01 s, 0.745 (0.721-0.769) / 1.011 / 0.995 |
| 4 concurrent clients x 24 queries, wall time per round (3 rounds each) | 50.9 → 41.1 s, 0.808 (0.778-0.839) | not measured | 49.0 → 41.5 s, 0.844 (0.787-0.905) |
| Samples table on disk | 2,092.6 → 2,088.0 MB, -0.2% | 4,689.1 → 4,724.8 MB, +0.8% | 5,268.7 → 5,322.1 MB, +1.0% |
| Marks / primary key in memory, samples table | 57,346 → 115,546 marks, 2.0x | 49,282 → 99,299 marks / 1.26 → 2.23 MB, 2.0x / 1.8x | 49,282 → 99,299 marks / 1.26 → 2.23 MB, 2.0x / 1.8x |
| Write path: two INSERTs of one day through the engine / OPTIMIZE FINAL (3 reps) | 40.7 / 15.7 s → 41.0 / 16.6 s, +0.8% / +5.3% |
Compressible values: constant and step gauges, counters with flat runs, slow 1-decimal gauges; ZSTD(3) shrinks their ALP output 9.1x. The other two data sets hold the same 19,800 node-exporter and application series with random variation between samples and differ only in the network and disk counters: integers, with disk bytes in 512-byte units, as node_exporter reports them; or the same values with two random decimal digits. The integer counters were measured in both runs; the second gave W1 0.958, W3 0.982, F4 0.972, F6 0.986, O01 1.046 (1.015-1.078) and Q02 1.009 (0.964-1.056), whose first-run ratio comes from two slow outliers. † The results differ only in float summation order (at most 3.3e-15 relative), slightly more than between runs of one table.
Why it happens and what the patch changes
Since #117344 every row of the samples table holds one hour of one series, and the default index_granularity_bytes = 524288 makes a granule about 135 rows: about 135 hours of one 15 s series. A short window older than the recent samples table (4 days) still reads whole granules. With 262144-byte granules, windows 5 days back read about half the rows and decompressed bytes.
Whether long scans get slower depends on how zstd encodes the output of ALP, because compressed blocks follow the granule boundaries: about 260 KB of uncompressed values per block with the default, 130 KB with the patch. zstd stores the literals of a block either raw or Huffman-coded, and Huffman-coded literals decode about 2x slower: 985 vs 1868 MB/s, decoded standalone with contrib/zstd. With compressible values ZSTD(3) Huffman-codes most literals at both block sizes (91% and 87% of the literal bytes). With integer byte counters the mode doesn't change either: the network counters stay raw at both sizes (ZSTD shrinks their ALP output by about 1% or less), and the disk bytes in 512-byte units are Huffman-coded at both sizes (12-13% smaller). So their 7- and 14-day scans don't get slower. Byte counters with two random decimal digits are in between: ZSTD makes their ALP output 6-7% smaller and keeps 94% of their literal bytes raw in the default blocks, but Huffman-codes all of them in the smaller blocks. Their 7- and 14-day scans (W1, W3, W5, F1, F4, F6) get 6-13% slower; the long scans of the other metrics don't.
Two alternatives were measured on the compressible and the random-decimal data and rejected. CODEC(ALP, LZ4) for the values removes the slowdown with random decimal digits, but with compressible values it makes the value column 80-85% and the samples table 19-20% larger. min_compress_block_size = 196608 (two granules per compressed block) keeps the long scans as they are but leaves only 7-9% of the gain for short windows, and 7-day ranges with compressible values get 3-4% slower.
The patch changes the default of samples_index_granularity_bytes from 524288 to 262144, with its documentation in StorageTimeSeries.cpp, and updates the gtest, two stateless test references and one integration test that check the default. Existing tables keep their definitions.
Correctness: the two tables of each data set hold identical rows (count() and sum(cityHash64(*)) of every inner table). For every query the results of the two tables have the same series and timestamps, and they're identical (typed digest of the response) or differ only in float summation order: at most 6e-14 relative for the histogram quantile W2, at most 3.3e-15 for the others. The 73 TimeSeries and PromQL gtests pass. In the 114 TimeSeries stateless tests the only changed results are the new default in 05140 and 05141; tests that need clusters, Keeper or a backup disk could not run in this setup. Integration tests were not run; the patch updates test_index_granularity in test_prometheus_protocols/test_different_table_engines.py, which checks the default.
Which ClickHouse versions are affected?
Tested versions and source commits
Master 9b41283be80e273613e9423e3437e47b7d281942 (26.11.1, 2026-11-12), Release build with clang 21 (no LTO), with the patch below on top. TimeSeries tables with bucketed samples (version 11, since #117344) are affected. Measured on an AWS instance with 4 vCPU (Intel Xeon 8375C) and 30 GiB RAM.
How to reproduce
Data, queries and measurement procedure
Users profile: allow_experimental_time_series_table = 1. Six Prometheus API handlers on one server, one per table:
<http_handlers>
<rule><url_prefix>/cd/api/v1</url_prefix><handler><type>prometheus_api_v1</type><table>default.tsc_d</table></handler></rule>
<rule><url_prefix>/cp/api/v1</url_prefix><handler><type>prometheus_api_v1</type><table>default.tsc_p</table></handler></rule>
<rule><url_prefix>/nd/api/v1</url_prefix><handler><type>prometheus_api_v1</type><table>default.ts_d</table></handler></rule>
<rule><url_prefix>/np/api/v1</url_prefix><handler><type>prometheus_api_v1</type><table>default.ts_p</table></handler></rule>
<rule><url_prefix>/id/api/v1</url_prefix><handler><type>prometheus_api_v1</type><table>default.tsi_d</table></handler></rule>
<rule><url_prefix>/ip/api/v1</url_prefix><handler><type>prometheus_api_v1</type><table>default.tsi_p</table></handler></rule>
</http_handlers>
Tables, with the patch (on master the _p tables get SETTINGS samples_index_granularity_bytes = 262144 instead):
CREATE TABLE tsc_d ENGINE = TimeSeries SETTINGS samples_index_granularity_bytes = 524288;
CREATE TABLE tsc_p ENGINE = TimeSeries;
CREATE TABLE ts_d ENGINE = TimeSeries SETTINGS samples_index_granularity_bytes = 524288;
CREATE TABLE ts_p ENGINE = TimeSeries;
CREATE TABLE tsi_d ENGINE = TimeSeries SETTINGS samples_index_granularity_bytes = 524288;
CREATE TABLE tsi_p ENGINE = TimeSeries;
Byte counters with random decimal digits, 19,800 node-exporter and application series:
DROP DATABASE IF EXISTS gen SYNC;
CREATE DATABASE gen;
CREATE TABLE gen.series (sid UInt32, metric String, tags Map(String, String), kind UInt8, base Float64, rate Float64, amp Float64, phase Float64, grp UInt32, cdf Float64)
ENGINE = MergeTree ORDER BY sid;
-- node_cpu_seconds_total: 200 hosts x 8 cpus x 8 modes = 12800
INSERT INTO gen.series
SELECT n AS sid, 'node_cpu_seconds_total',
map('job', ['node-prod','node-staging','node-dev','node-infra'][(h % 4) + 1], 'instance', concat('host-', leftPad(toString(h), 3, '0'), ':9100'),
'cpu', toString(cpu), 'mode', ['idle','user','system','iowait','irq','softirq','steal','nice'][m + 1]),
1, 2592000 * mr * (0.5 + f), mr * (0.8 + 0.4 * f), 0, 0, 0, 0
FROM (SELECT number AS n, intDiv(number, 64) + 1 AS h, intDiv(number % 64, 8) AS cpu, number % 8 AS m,
[0.85, 0.07, 0.03, 0.015, 0.002, 0.006, 0.001, 0.002][m + 1] AS mr,
(cityHash64('cpu', number) % 1000000) / 1e6 AS f FROM numbers(12800));
-- node_network_{receive,transmit}_{bytes,packets}_total: 200 x 4 metrics x 4 devices = 3200
INSERT INTO gen.series
SELECT 12800 + n, ['node_network_receive_bytes_total','node_network_transmit_bytes_total','node_network_receive_packets_total','node_network_transmit_packets_total'][mt + 1],
map('job', ['node-prod','node-staging','node-dev','node-infra'][(h % 4) + 1], 'instance', concat('host-', leftPad(toString(h), 3, '0'), ':9100'),
'device', ['eth0','eth1','lo','docker0'][d + 1]),
1, 1e9 * (1 + 10 * f), [1.2e6, 1.5e5, 2e4, 6e4][d + 1] * [1.0, 0.7, 1.0/900, 1.0/1100][mt + 1] * (0.5 + f), 0, 0, 0, 0
FROM (SELECT number AS n, intDiv(number, 16) + 1 AS h, intDiv(number % 16, 4) AS mt, number % 4 AS d,
(cityHash64('net', number) % 1000000) / 1e6 AS f FROM numbers(3200));
-- per-host single gauges/counters: load1, uname_info, MemTotal, MemAvailable, context_switches: 200 x 5 = 1000
INSERT INTO gen.series
SELECT 16000 + n, ['node_load1','node_uname_info','node_memory_MemTotal_bytes','node_memory_MemAvailable_bytes','node_context_switches_total'][mt + 1],
if(mt = 1,
map('job', ['node-prod','node-staging','node-dev','node-infra'][(h % 4) + 1], 'instance', concat('host-', leftPad(toString(h), 3, '0'), ':9100'),
'nodename', concat('host-', leftPad(toString(h), 3, '0')), 'release', ['5.15.0-1051-aws','6.1.0-18-amd64','6.8.0-45-generic'][(h % 3) + 1],
'machine', 'x86_64', 'sysname', 'Linux'),
map('job', ['node-prod','node-staging','node-dev','node-infra'][(h % 4) + 1], 'instance', concat('host-', leftPad(toString(h), 3, '0'), ':9100'))),
[3, 6, 6, 7, 1][mt + 1],
multiIf(mt = 0, 1.0 + 3 * f, mt = 1, 1.0, mt IN (2, 3), [17179869184., 34359738368., 68719476736.][(h % 3) + 1], 1e8 * (1 + f)),
if(mt = 4, 5000 + 15000 * f, 0), 0.5 + f, 2 * pi() * f, 0, 0
FROM (SELECT number AS n, intDiv(number, 5) + 1 AS h, number % 5 AS mt, (cityHash64('host1', number) % 1000000) / 1e6 AS f FROM numbers(1000));
-- node_hwmon_temp_celsius: 200 x 4 = 800
INSERT INTO gen.series
SELECT 17000 + n, 'node_hwmon_temp_celsius',
map('job', ['node-prod','node-staging','node-dev','node-infra'][(h % 4) + 1], 'instance', concat('host-', leftPad(toString(h), 3, '0'), ':9100'),
'chip', 'platform_coretemp_0', 'sensor', concat('temp', toString(s + 1))),
4, 45 + 10 * f, 0, 5 + 5 * f, 2 * pi() * f, 0, 0
FROM (SELECT number AS n, intDiv(number, 4) + 1 AS h, number % 4 AS s, (cityHash64('temp', number) % 1000000) / 1e6 AS f FROM numbers(800));
-- node_disk_written_bytes_total: 200 x 3 = 600
INSERT INTO gen.series
SELECT 17800 + n, 'node_disk_written_bytes_total',
map('job', ['node-prod','node-staging','node-dev','node-infra'][(h % 4) + 1], 'instance', concat('host-', leftPad(toString(h), 3, '0'), ':9100'),
'device', ['sda','sdb','nvme0n1'][d + 1]),
1, 1e10 * (1 + f), [2e5, 8e5, 3e6][d + 1] * (0.5 + f), 0, 0, 0, 0
FROM (SELECT number AS n, intDiv(number, 3) + 1 AS h, number % 3 AS d, (cityHash64('disk', number) % 1000000) / 1e6 AS f FROM numbers(600));
-- node_filesystem_avail_bytes: 200 x 3 = 600
INSERT INTO gen.series
SELECT 18400 + n, 'node_filesystem_avail_bytes',
map('job', ['node-prod','node-staging','node-dev','node-infra'][(h % 4) + 1], 'instance', concat('host-', leftPad(toString(h), 3, '0'), ':9100'),
'mountpoint', ['/','/data','/boot'][d + 1], 'device', ['/dev/root','/dev/nvme1n1','/dev/sda1'][d + 1], 'fstype', ['ext4','xfs','vfat'][d + 1]),
5, [1e11, 1e12, 1e9][d + 1] * (0.3 + 0.5 * f), [1e11, 1e12, 1e9][d + 1] * 1e-7 * (0.5 + f), [1e11, 1e12, 1e9][d + 1] * 1e-5, 0, 0, 0
FROM (SELECT number AS n, intDiv(number, 3) + 1 AS h, number % 3 AS d, (cityHash64('fs', number) % 1000000) / 1e6 AS f FROM numbers(600));
-- http_requests_total: 5 jobs x 8 instances x 4 codes x 2 methods = 320
INSERT INTO gen.series
SELECT 19000 + n, 'http_requests_total',
map('job', ['api','web','auth','billing','search'][j + 1], 'instance', concat('app-', ['api','web','auth','billing','search'][j + 1], '-', toString(i + 1), ':8080'),
'code', ['200','404','500','503'][c + 1], 'method', ['GET','POST'][m + 1]),
8, floor(1e6 * f), [40, 2, 0.4, 0.1][c + 1] * [1.0, 0.3][m + 1] * (0.5 + f), 0, 0, 0, 0
FROM (SELECT number AS n, intDiv(number, 64) AS j, intDiv(number % 64, 8) AS i, intDiv(number % 8, 2) AS c, number % 2 AS m,
(cityHash64('http', number) % 1000000) / 1e6 AS f FROM numbers(320));
-- http_request_duration_seconds_bucket: 5 jobs x 8 instances x 12 le = 480
INSERT INTO gen.series
SELECT 19320 + n, 'http_request_duration_seconds_bucket',
map('job', ['api','web','auth','billing','search'][j + 1], 'instance', concat('app-', ['api','web','auth','billing','search'][j + 1], '-', toString(i + 1), ':8080'),
'le', ['0.005','0.01','0.025','0.05','0.1','0.25','0.5','1','2.5','5','10','+Inf'][b + 1]),
2, 0, 30 * (0.5 + fg), 0, 0, g, [0.03,0.08,0.2,0.38,0.6,0.8,0.9,0.95,0.98,0.995,0.999,1.0][b + 1]
FROM (SELECT number AS n, intDiv(number, 96) AS j, intDiv(number % 96, 12) AS i, number % 12 AS b, intDiv(number, 12) AS g,
(cityHash64('hist', intDiv(number, 12)) % 1000000) / 1e6 AS fg FROM numbers(480));
CREATE FUNCTION genVal AS (sid, kind, base, rate, amp, phase, grp, cdf, k, tms) -> multiIf(
kind = 1, round(base + rate * 15 * (k + 0.5 * ((cityHash64(sid, k, 'u') % 1000000) / 1e6)), 2),
kind = 2, floor(rate * 15 * (k + 0.5 * ((cityHash64(grp, k, 'ug') % 1000000) / 1e6)) * cdf),
kind = 3, round(greatest(0, base + amp * sin(2 * pi() * (tms / 1000.0) / 3600 + phase) + 0.5 * ((cityHash64(sid, k, 'u') % 1000000) / 1e6)), 2),
kind = 4, round(base + amp * sin(2 * pi() * (tms / 1000.0) / 7200 + phase) + 3 * ((cityHash64(sid, k, 'u') % 1000000) / 1e6), 1),
kind = 5, floor(base - rate * k + amp * ((cityHash64(sid, k, 'u') % 1000000) / 1e6)),
kind = 6, base,
kind = 7, floor(base * (0.35 + 0.2 * sin(2 * pi() * (tms / 1000.0) / 5400 + phase) + 0.05 * ((cityHash64(sid, k, 'u') % 1000000) / 1e6))),
kind = 8, floor(base + rate * 15 * (k + 0.5 * ((cityHash64(sid, k, 'u') % 1000000) / 1e6))), nan);
-- 14 days of samples (15 s, ending 1794513600), one INSERT per day d = 0..13
INSERT INTO ts_d (metric_name, tags, samples)
SELECT metric, tags, arrayMap(k -> (fromUnixTimestamp64Milli(toInt64(1793304000000 + 15000 * k + (cityHash64(sid, 'phase') % 14950) + (cityHash64(sid, k, 'jit') % 50)), 'UTC'),
genVal(sid, kind, base, rate, amp, phase, grp, cdf, k, toInt64(1793304000000 + 15000 * k + (cityHash64(sid, 'phase') % 14950) + (cityHash64(sid, k, 'jit') % 50)))),
arrayMap(x -> toUInt64(x), range(d * 5760, (d + 1) * 5760)))
FROM gen.series SETTINGS max_threads = 4, max_insert_threads = 1, max_block_size = 1000;
Integer byte counters, the same series with the network counters as integers and the disk bytes in 512-byte units, loaded into tsi_d the same way with genValI instead of genVal, in two INSERTs per day (WHERE sid % 2 = 0, then = 1) to stay within memory:
CREATE FUNCTION genValI AS (sid, kind, base, rate, amp, phase, grp, cdf, k, tms) -> multiIf(
sid >= 12800 AND sid < 16000, floor(base + rate * 15 * (k + 0.5 * ((cityHash64(sid, k, 'u') % 1000000) / 1e6))),
sid >= 17800 AND sid < 18400, 512 * floor((base + rate * 15 * (k + 0.5 * ((cityHash64(sid, k, 'u') % 1000000) / 1e6))) / 512),
genVal(sid, kind, base, rate, amp, phase, grp, cdf, k, tms));
Compressible values, the same series plus 3,240 kube-state-metrics and up series (23,040 series), loaded into tsc_d the same way:
-- Compressible corpus: the 19,800 series above plus 3,240 kube-state-metrics/up series; every series follows one value shape:
-- 1 constant, 2 0/1 gauge per window of p2 samples, 3 enum 0..4 per window, 4 counter with flat runs (one increment p3 per window of
-- p2 = 20/80/240 samples at a random offset, same average rate as above), 5 slow 1-decimal gauge, 6 step-down gauge, 7 8-level gauge.
CREATE TABLE gen.series_c (sid UInt32, metric String, tags Map(String, String), shape UInt8, hkey UInt64, p1 Float64, p2 UInt32, p3 Float64, p4 Float64)
ENGINE = MergeTree ORDER BY sid;
INSERT INTO gen.series_c
SELECT sid, metric, tags,
multiIf(kind IN (1, 2, 8), 4, kind IN (3, 4), 5, kind = 5, 6, kind = 6, 1, kind = 7, 7, 0) AS shape,
if(kind = 2, 1000000 + grp, sid) AS hkey,
multiIf(metric = 'node_cpu_seconds_total', round(base, 2), kind IN (1, 8), floor(base), kind = 2, 0, kind IN (3, 4), base, floor(base)) AS p1,
multiIf(kind IN (1, 2, 8), [20, 80, 240][(cityHash64(hkey, 'L') % 3) + 1], kind IN (3, 4), [21600, 86400][(cityHash64(sid, 'P') % 2) + 1],
kind = 5, [80, 240, 1440][(cityHash64(sid, 'L') % 3) + 1], kind = 7, [240, 1440][(cityHash64(sid, 'L') % 2) + 1], 1) AS p2,
multiIf(metric = 'node_cpu_seconds_total', round(rate * 15 * p2, 2), kind IN (1, 8), greatest(1, floor(rate * 15 * p2)),
kind = 2, greatest(1, floor(rate * 15 * p2 * cdf)), kind IN (3, 4), amp, kind = 5, 4096 * (1 + cityHash64(sid, 'inc') % 256), 0) AS p3,
if(kind IN (3, 4), phase, 1) AS p4
FROM gen.series;
INSERT INTO gen.series_c
SELECT 20000 + n, 'up',
if(n < 200, map('job', ['node-prod','node-staging','node-dev','node-infra'][((n + 1) % 4) + 1], 'instance', concat('host-', leftPad(toString(n + 1), 3, '0'), ':9100')),
map('job', ['api','web','auth','billing','search'][intDiv(n - 200, 8) + 1], 'instance', concat('app-', ['api','web','auth','billing','search'][intDiv(n - 200, 8) + 1], '-', toString((n - 200) % 8 + 1), ':8080'))),
2, 20000 + n, 0, 240, 0, 97
FROM (SELECT number AS n FROM numbers(240));
INSERT INTO gen.series_c
SELECT 20240 + n, 'kube_pod_info', map('job', 'kube-state-metrics', 'namespace', concat('ns-', toString(n % 10)), 'pod', concat('pod-', toString(n)), 'node', concat('host-', leftPad(toString(n % 200 + 1), 3, '0'))),
1, 20240 + n, 1, 1, 0, 1
FROM (SELECT number AS n FROM numbers(1000));
INSERT INTO gen.series_c
SELECT 21240 + n, 'kube_pod_status_ready', map('job', 'kube-state-metrics', 'namespace', concat('ns-', toString(n % 10)), 'pod', concat('pod-', toString(n)), 'condition', 'true'),
2, 21240 + n, 0, [240, 1440, 5760][(cityHash64(n, 'L') % 3) + 1], 0, 80
FROM (SELECT number AS n FROM numbers(1000));
INSERT INTO gen.series_c
SELECT 22240 + n, 'kube_pod_status_phase', map('job', 'kube-state-metrics', 'namespace', concat('ns-', toString(n % 10)), 'pod', concat('pod-', toString(n))),
3, 22240 + n, 0, [240, 1440, 5760][(cityHash64(n, 'L2') % 3) + 1], 0, 5
FROM (SELECT number AS n FROM numbers(1000));
CREATE FUNCTION jstepC AS (h, k, w) -> intDiv(k, w) + if((k % w) >= (cityHash64(h, intDiv(k, w), 'j') % w), 1, 0);
CREATE FUNCTION genValC AS (sid, hkey, shape, p1, p2, p3, p4, k, tms) -> multiIf(
shape = 1, p1,
shape = 2, if((cityHash64(sid, intDiv(k, p2), 'st') % 100) < p4, 1., 0.),
shape = 3, toFloat64(cityHash64(sid, intDiv(k, p2), 'en') % toUInt64(greatest(p4, 1))),
shape = 4, round(p1 + p3 * jstepC(hkey, k, p2), 2),
shape = 5, round(p1 + p3 * sin(2 * pi() * (tms / 1000.0) / p2 + p4), 1),
shape = 6, p1 - p3 * jstepC(sid, k, p2),
shape = 7, floor(p1 * (0.3 + 0.05 * (cityHash64(sid, intDiv(k, p2), 'lv') % 8))),
nan);
INSERT INTO tsc_d (metric_name, tags, samples)
SELECT metric, tags, arrayMap(k -> (fromUnixTimestamp64Milli(toInt64(1793304000000 + 15000 * k + (cityHash64(sid, 'phase') % 14950) + (cityHash64(sid, k, 'jit') % 50)), 'UTC'),
genValC(sid, hkey, shape, p1, p2, p3, p4, k, toInt64(1793304000000 + 15000 * k + (cityHash64(sid, 'phase') % 14950) + (cityHash64(sid, k, 'jit') % 50)))),
arrayMap(x -> toUInt64(x), range(d * 5760, (d + 1) * 5760)))
FROM gen.series_c SETTINGS max_threads = 4, max_insert_threads = 1, max_block_size = 1000;
After loading, OPTIMIZE TABLE ... FINAL of every inner table of ts_d, tsi_d and tsc_d. Then the rows of every inner table are copied into the inner table of the same kind of ts_p, tsi_p and tsc_p (INSERT INTO <inner of ts_p> SELECT * FROM <inner of ts_d>; inner tables are .inner_id.<kind>.<uuid of the table>), followed by OPTIMIZE TABLE ... FINAL of every inner table. The recent samples tables drop expired rows by wall-clock time on every merge, so the last OPTIMIZE of the two recent samples tables of a data set should run within the same 15 minutes; otherwise one of them can hold one 15-minute bucket less.
Queries (kind range = /api/v1/query_range, instant = /api/v1/query):
| ID | Type | Start / time | End | Step | Query |
|---|---|---|---|---|---|
| W1 | range | 1793908800 | 1794513600 | 900s | sum by (instance) (rate(node_disk_written_bytes_total[5m])) |
| W2 | range | 1793908800 | 1794513600 | 900s | histogram_quantile(0.99, sum by (le) (rate(http_request_duration_seconds_bucket[5m]))) |
| W3 | range | 1793908800 | 1794513600 | 900s | sum by (job) (rate(node_network_receive_bytes_total[5m])) |
| W4 | range | 1793908800 | 1794513600 | 3600s | max by (instance) (max_over_time(node_hwmon_temp_celsius[1h])) |
| W5 | range | 1793908800 | 1794513600 | 3600s | sum by (job) (rate(node_network_receive_bytes_total[1h])) |
| W6 | range | 1793908800 | 1794513600 | 300s | rate(node_network_receive_bytes_total{instance="host-001:9100",device="eth0"}[5m]) |
| W7 | range | 1793908800 | 1794513600 | 900s | avg by (job) (rate(node_cpu_seconds_total{mode="user"}[5m])) |
| W8 | range | 1793908800 | 1794513600 | 900s | avg_over_time(node_load1{job="node-prod"}[15m]) |
| O01 | range | 1794078000 | 1794081600 | 15s | rate(node_network_receive_bytes_total{instance="host-001:9100",device="eth0"}[5m]) |
| O02 | range | 1794078000 | 1794081600 | 15s | rate(node_network_receive_bytes_total{device="eth0"}[5m]) |
| O03 | range | 1794060000 | 1794081600 | 30s | sum by (job) (rate(node_cpu_seconds_total{mode!="idle"}[5m])) |
| O14 | instant | 1794081600 | sum by (job) (rate(http_requests_total[5m])) |
||
| O17 | instant | 1794081600 | sum(increase(node_context_switches_total[1h])) |
||
| O19 | range | 1794078000 | 1794081600 | 60s | rate(node_cpu_seconds_total[5m]) |
| O23b | instant | 1794081600 | sum(count_over_time(node_cpu_seconds_total[6h])) |
||
| OW | range | 1794510000 | 1794513600 | 15s | sum by (job) (rate(http_requests_total[5m] offset 1w)) |
| F1 | range | 1793307600 | 1794513600 | 3600s | sum by (job) (rate(node_network_receive_bytes_total[1h])) |
| F2 | range | 1793307600 | 1794513600 | 3600s | quantile(0.95, avg_over_time(node_load1[1h])) |
| F3 | instant | 1794513600 | max_over_time(node_load1{instance="host-007:9100"}[14d]) |
||
| F4 | instant | 1794513600 | sum(increase(node_network_transmit_bytes_total[14d])) |
||
| F6 | range | 1793304300 | 1794513600 | 900s | sum by (instance) (rate(node_disk_written_bytes_total[5m])) |
| Q02 | range | 1794510000 | 1794513600 | 15s | rate(node_network_receive_bytes_total{device="eth0"}[5m]) |
| Q19 | range | 1794510000 | 1794513600 | 60s | rate(node_cpu_seconds_total[5m]) |
| Q14 | instant | 1794513600 | sum by (job) (rate(http_requests_total[5m])) |
Procedure: two runs on one server, the first with ts_d, ts_p, tsi_d and tsi_p, the second with tsc_d, tsc_p, tsi_d and tsi_p. In each run, one warmup of every query on all four tables, then 15 repetitions; in each repetition every query is sent to the four tables back to back, with the order rotating; client wall time per request. The table shows the integer counters from the first run. Ratio = geometric mean of the per-repetition patched/master ratios, Student t 95% interval. Cold: before every request the server is restarted and the page cache dropped (echo 3 > /proc/sys/vm/drop_caches), 6 pairs with alternating order. Concurrent: 4 clients each send all 24 queries once, in a fixed shuffled order, to one table; wall time of the round, 3 rounds per table, alternating. Write path: per repetition a fresh table, two INSERTs of one day of the random-decimal data through the engine, then OPTIMIZE TABLE ... FINAL of the samples table; 3 repetitions, alternating.
Expected performance
Expected behavior
A short window should read about as many rows as it needs. The default granule size shouldn't make long scans slower.
Related issues and pull requests
Related work
- https://github.com/ClickHouse/ClickHouse/pull/117344 introduced the hourly buckets and the current defaults. We reported the same 135-hour granule over-read on it, proposing a time-bucket prefix in the sort key of the main samples table (https://github.com/ClickHouse/ClickHouse/pull/117344#issuecomment-5909661526), and again in the remeasure on its merge commit (https://github.com/ClickHouse/ClickHouse/pull/117344#issuecomment-6467755224). Smaller granules only change a default and keep the sort key, so long single-series ranges stay as they are, but the gain is smaller than with the prefix, which served such windows at close to recent-table speed.
- https://github.com/ClickHouse/ClickHouse/pull/120454 set
CODEC(ALP, ZSTD(3))for the generated value columns and measured that ZSTD remains useful after ALP: "it reduced the encoded values from 87.8 MB to 13.8 MB" on a stratified 120.3-million-row sample. The patch keeps that codec:CODEC(ALP, LZ4)would avoid the slower long scans with noise values, but made the value column 80-85% larger with compressible values. - https://github.com/ClickHouse/ClickHouse/pull/113769 (closed) proposed a byte-based
samples_index_granularity_bytesfor the earlier layout with one sample per row. Our measurements on it (https://github.com/ClickHouse/ClickHouse/pull/113769#issuecomment-5582671209) concerned that layout: a 256 KiB cap on 32768-row granules, and 32768 vs 8192 rows on ClockBench, not the 512 KiB default of version 11.
Additional context
Full patch
diff --git a/src/Storages/StorageTimeSeries.cpp b/src/Storages/StorageTimeSeries.cpp
index 2ca3060cafc..10060d41b55 100644
--- a/src/Storages/StorageTimeSeries.cpp
+++ b/src/Storages/StorageTimeSeries.cpp
@@ -1236,7 +1236,7 @@ SAMPLES INNER COLUMNS
CODEC `samples.timestamp` (Delta, T64, ZSTD(3)),
CODEC `samples.value` (ALP, ZSTD(3))
)
-SAMPLES INNER ENGINE = AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, 'UTC')) ORDER BY (id, bucket) SETTINGS index_granularity = 256, index_granularity_bytes = 524288
+SAMPLES INNER ENGINE = AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, 'UTC')) ORDER BY (id, bucket) SETTINGS index_granularity = 256, index_granularity_bytes = 262144
RECENT SAMPLES INNER COLUMNS
(
`id` Tuple(UInt64, LowCardinality(UUID)),
@@ -1298,7 +1298,7 @@ CREATE TABLE default.`.inner_id.samples.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`
ENGINE = AggregatingMergeTree
PARTITION BY toYYYYMM(toDateTime(bucket, 'UTC'))
ORDER BY (id, bucket)
-SETTINGS index_granularity = 256, index_granularity_bytes = 524288
+SETTINGS index_granularity = 256, index_granularity_bytes = 262144
```
```sql
@@ -1561,7 +1561,7 @@ Here is a list of settings which can be specified while defining a `TimeSeries`
| `tags_deduplication_cache_size_bytes` | UInt64 | 104857600 | Maximum size in bytes of the deduplication cache of the [tags](#tags-table) table. The cache remembers the time series written recently, so their tags aren't written again with every insert. When the cache is full, the entries used only once are evicted first, then the least recently used ones (SLRU). The cache is used only when the [tags](#tags-table) table doesn't store `min_time` and `max_time` (tables of [version](#schema-versioning) 7 or earlier store them unless `store_min_time_and_max_time` is disabled), because otherwise every insert changes these columns: the default value is ignored then, and an explicit non-zero value is rejected. Set to 0 to disable the cache, see also `tags_deduplication_cache_expiration_seconds` |
| `tags_deduplication_cache_expiration_seconds` | UInt64 | 3600 | Time after which an entry of the deduplication cache of the [tags](#tags-table) table expires, counted from the moment the time series was written. The actual lifetime of an entry is uniformly random in the range `[0.8 * expiration_seconds, expiration_seconds]`, so the time series written together aren't all written again at the same moment. So every time series is written again at least once per this period, which limits any difference between the cache and the table. The cache is local to the server and cleared by `TRUNCATE TABLE` executed on it or by `SYSTEM DROP TIME SERIES CACHES`. Used only when the [tags](#tags-table) table doesn't store `min_time` and `max_time`, see `tags_deduplication_cache_size_bytes`. Set to 0 to disable the cache |
| `samples_index_granularity` | UInt64 | 256 | Sets `index_granularity` of the inner [samples](#samples-table) table. When set explicitly, it overrides `index_granularity` from the engine declaration. Ignored for a non-MergeTree engine. The default is 32768 for tables of [version](#schema-versioning) 10 or earlier, which store one sample per row |
-| `samples_index_granularity_bytes` | UInt64 | 524288 | Sets `index_granularity_bytes` of the inner [samples](#samples-table) table. When set explicitly, it overrides `index_granularity_bytes` from the engine declaration. Ignored for a non-MergeTree engine |
+| `samples_index_granularity_bytes` | UInt64 | 262144 | Sets `index_granularity_bytes` of the inner [samples](#samples-table) table. When set explicitly, it overrides `index_granularity_bytes` from the engine declaration. Ignored for a non-MergeTree engine |
| `recent_samples_ttl_seconds` | UInt64 | 345600 | Retention of the additional [recent samples](#recent-samples-table) target table, which every inserted sample is written to as well. An inner recent samples table always gets `TTL toDateTime(bucket) + toIntervalSecond(recent_samples_ttl_seconds + recent_samples_bucket_step_seconds)` derived from this setting (overriding any TTL from the engine declaration): a row expires when all its samples are older than the TTL. Queries whose time range fits in the TTL window prefer the recent samples table to the main samples table (see the query-level setting `time_series_prefer_recent_samples_table`). The default is 4 days; the effective value is pinned into the table definition at CREATE time. Set to 0 to disable the recent samples table |
| `recent_samples_bucket_step_seconds` | UInt64 | 900 | The length in seconds of the time buckets of the [recent samples](#recent-samples-table) table, see `samples_bucket_step_seconds`. The default is 15 minutes; the effective value is pinned into the table definition at CREATE time. Requires `recent_samples_ttl_seconds` to be non-zero |
| `recent_samples_partition_by` | Expression | `toStartOfInterval(toDateTime(bucket, 'UTC'), toIntervalHour(5))` | Partition key of the inner [recent samples](#recent-samples-table) table, for example `toStartOfHour(toDateTime(bucket, 'UTC'))`. When set explicitly, it overrides the partition key from the engine declaration; if neither is set, one partition per 5 hours is used. Requires `recent_samples_ttl_seconds` to be non-zero |
diff --git a/src/Storages/TimeSeries/TimeSeriesSettings.cpp b/src/Storages/TimeSeries/TimeSeriesSettings.cpp
index 685c4f5f394..551568a867e 100644
--- a/src/Storages/TimeSeries/TimeSeriesSettings.cpp
+++ b/src/Storages/TimeSeries/TimeSeriesSettings.cpp
@@ -32,7 +32,7 @@ namespace ErrorCodes
DECLARE(UInt64, samples_bucket_step_seconds, 3600, "The length in seconds of the time buckets of the 'samples' table: a row of the table contains the samples of one time series with timestamps in one bucket, and the 'bucket' column contains the start of the bucket, i.e. the timestamp rounded down to a multiple of this setting. The default is 1 hour; the effective value is pinned into the table definition at CREATE time. Requires 'version' to be at least 11", 0) \
DECLARE(ASTFunction, samples_partition_by, String{}, "Partition key of the inner 'samples' table, for example 'toStartOfWeek(bucket)'. When set explicitly, it overrides the partition key from the engine declaration; if neither is set, 'toYYYYMM(toDateTime(bucket, 'UTC'))' is used, i.e. one partition per month. Requires 'version' to be at least 11", 0) \
DECLARE(UInt64, samples_index_granularity, 256, "Sets 'index_granularity' of the inner 'samples' table. When set explicitly, it overrides 'index_granularity' from the engine declaration. Ignored for a non-MergeTree engine. The default is 32768 for tables of version 10 or earlier, which store one sample per row", 0) \
- DECLARE(UInt64, samples_index_granularity_bytes, 524288, "Sets 'index_granularity_bytes' of the inner 'samples' table. When set explicitly, it overrides 'index_granularity_bytes' from the engine declaration. Ignored for a non-MergeTree engine. Requires 'version' to be at least 11", 0) \
+ DECLARE(UInt64, samples_index_granularity_bytes, 262144, "Sets 'index_granularity_bytes' of the inner 'samples' table. When set explicitly, it overrides 'index_granularity_bytes' from the engine declaration. Ignored for a non-MergeTree engine. Requires 'version' to be at least 11", 0) \
DECLARE(UInt64, recent_samples_ttl_seconds, 345600, "Retention of the additional 'recent samples' target table, which every inserted sample is written to as well. An inner recent samples table always gets 'TTL toDateTime(bucket) + toIntervalSecond(recent_samples_ttl_seconds + recent_samples_bucket_step_seconds)' derived from this setting (overriding any TTL from the engine declaration; tables of version 10 or earlier get 'TTL toDateTime(timestamp) + toIntervalSecond(recent_samples_ttl_seconds)'). Queries whose time range fits in the TTL window prefer the recent samples table to the main samples table (see the query-level setting 'time_series_prefer_recent_samples_table'). The default is 4 days; set to 0 to disable the recent samples table", 0) \
DECLARE(UInt64, recent_samples_bucket_step_seconds, 900, "The length in seconds of the time buckets of the 'recent samples' table, see 'samples_bucket_step_seconds'. The default is 15 minutes; the effective value is pinned into the table definition at CREATE time. Requires 'recent_samples_ttl_seconds' to be non-zero and 'version' to be at least 11", 0) \
DECLARE(ASTFunction, recent_samples_partition_by, String{}, "Partition key of the inner 'recent samples' table, for example 'toStartOfHour(bucket)'. When set explicitly, it overrides the partition key from the engine declaration; if neither is set, 'toStartOfInterval(toDateTime(bucket, 'UTC'), toIntervalHour(5))' is used (with 'timestamp' instead of 'bucket' for tables of version 10 or earlier). Requires 'recent_samples_ttl_seconds' to be non-zero", 0) \
diff --git a/src/Storages/TimeSeries/tests/gtest_normalize_time_series_definition.cpp b/src/Storages/TimeSeries/tests/gtest_normalize_time_series_definition.cpp
index 4b7c52d6638..d636342ee56 100644
--- a/src/Storages/TimeSeries/tests/gtest_normalize_time_series_definition.cpp
+++ b/src/Storages/TimeSeries/tests/gtest_normalize_time_series_definition.cpp
@@ -190,7 +190,7 @@ namespace
+ ", `min_time` SimpleAggregateFunction(min, DateTime64(3))" + DEFAULT_TIMESTAMP_CODEC
+ ", `max_time` SimpleAggregateFunction(max, DateTime64(3))" + DEFAULT_TIMESTAMP_CODEC + DEFAULT_SAMPLES_CODECS;
const String DEFAULT_SAMPLES_ENGINE = "AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, 'UTC')) ORDER BY (id, bucket) "
- "SETTINGS index_granularity = 256, index_granularity_bytes = 524288";
+ "SETTINGS index_granularity = 256, index_granularity_bytes = 262144";
const String DEFAULT_RECENT_SAMPLES_ENGINE = "AggregatingMergeTree PARTITION BY toStartOfInterval(toDateTime(bucket, 'UTC'), toIntervalHour(5)) ORDER BY (id, bucket) "
"TTL toDateTime(bucket) + toIntervalSecond(346500) SETTINGS index_granularity = 256, index_granularity_bytes = 131072, ttl_only_drop_parts = 1";
@@ -772,7 +772,7 @@ TEST_F(NormalizeTimeSeriesDefinitionTest, SamplesEngineSettings)
/// A declared partition key is kept.
definition = normalizeNewTable("CREATE TABLE db.ts ENGINE = TimeSeries SAMPLES ENGINE = AggregatingMergeTree PARTITION BY toYYYYMM(bucket)");
EXPECT_EQ(extractInnerEngine(definition, "SAMPLES"),
- "AggregatingMergeTree PARTITION BY toYYYYMM(bucket) ORDER BY (id, bucket) SETTINGS index_granularity = 256, index_granularity_bytes = 524288");
+ "AggregatingMergeTree PARTITION BY toYYYYMM(bucket) ORDER BY (id, bucket) SETTINGS index_granularity = 256, index_granularity_bytes = 262144");
/// The samples tables require an aggregating engine: the rows of the same series and bucket are merged.
for (const String & engine : {"MergeTree", "ReplacingMergeTree", "Memory", "ReplicatedMergeTree('/clickhouse/tables/{shard}/ts_samples', '{replica}')"})
@@ -1208,7 +1208,7 @@ TEST_F(NormalizeTimeSeriesDefinitionTest, BucketedSamples)
EXPECT_FALSE(extractInnerColumns(definition, "SAMPLES").contains("`timestamp`")) << definition;
EXPECT_EQ(extractInnerColumns(definition, "RECENT SAMPLES"), DEFAULT_SAMPLES_COLUMNS);
EXPECT_EQ(extractInnerEngine(definition, "SAMPLES"),
- "AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, 'UTC')) ORDER BY (id, bucket) SETTINGS index_granularity = 65536, index_granularity_bytes = 524288");
+ "AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, 'UTC')) ORDER BY (id, bucket) SETTINGS index_granularity = 65536, index_granularity_bytes = 262144");
EXPECT_EQ(extractInnerEngine(definition, "RECENT SAMPLES"), DEFAULT_RECENT_SAMPLES_ENGINE);
}
diff --git a/tests/integration/test_prometheus_protocols/test_different_table_engines.py b/tests/integration/test_prometheus_protocols/test_different_table_engines.py
index 1a1eccf872b..4733a7d6d88 100644
--- a/tests/integration/test_prometheus_protocols/test_different_table_engines.py
+++ b/tests/integration/test_prometheus_protocols/test_different_table_engines.py
@@ -323,7 +323,7 @@ def test_inner_engines():
# Checks that the `samples_index_granularity` and `tags_index_granularity` settings
# set `index_granularity` of the samples and tags inner tables.
def test_index_granularity():
- # The default value of `samples_index_granularity` is 256, the default value of `samples_index_granularity_bytes` is 524288,
+ # The default value of `samples_index_granularity` is 256, the default value of `samples_index_granularity_bytes` is 262144,
# the default value of `tags_index_granularity` is 8192.
node.query("CREATE TABLE prometheus ENGINE=TimeSeries")
check()
@@ -333,7 +333,7 @@ def test_index_granularity():
"AND name = (SELECT _table FROM timeSeriesSamples(prometheus) LIMIT 1)"
)
assert "index_granularity = 256" in samples_engine
- assert "index_granularity_bytes = 524288" in samples_engine
+ assert "index_granularity_bytes = 262144" in samples_engine
assert "index_granularity = 8192" in node.query(
"SELECT engine_full FROM system.tables WHERE database = currentDatabase() "
"AND name = (SELECT _table FROM timeSeriesTags(prometheus) LIMIT 1)"
diff --git a/tests/queries/0_stateless/05140_time_series_bucketed_samples_table.reference b/tests/queries/0_stateless/05140_time_series_bucketed_samples_table.reference
index d191d073c0e..a5b70ab5acb 100644
--- a/tests/queries/0_stateless/05140_time_series_bucketed_samples_table.reference
+++ b/tests/queries/0_stateless/05140_time_series_bucketed_samples_table.reference
@@ -56,4 +56,4 @@ Array(Tuple(DateTime64(6), Float32))
-- a samples table of a wrong layout is rejected: the samples column must be SimpleAggregateFunction(timeSeriesGroupArray, Array(Tuple(timestamp ..., value ...)))
-- a table created AS a table of an older version gets the new layout with the same types
`id` Tuple(UInt64, LowCardinality(UUID)), `timestamp` DateTime64(6) CODEC(Delta, ZSTD), `value` Float32 MergeTree ORDER BY (id, timestamp) SETTINGS index_granularity = 32768
-11 `id` Tuple(UInt64, LowCardinality(UUID)), `samples` SimpleAggregateFunction(timeSeriesGroupArray, Array(Tuple(timestamp DateTime64(6), value Float32))), `bucket` DateTime64(6) CODEC(Delta, T64, ZSTD(3)), `min_time` SimpleAggregateFunction(min, DateTime64(6)) CODEC(Delta, T64, ZSTD(3)), `max_time` SimpleAggregateFunction(max, DateTime64(6)) CODEC(Delta, T64, ZSTD(3)), CODEC `samples.timestamp` (Delta, T64, ZSTD(3)), CODEC `samples.value` (ALP, ZSTD(3)) AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, \'UTC\')) ORDER BY (id, bucket) SETTINGS index_granularity = 256, index_granularity_bytes = 524288
+11 `id` Tuple(UInt64, LowCardinality(UUID)), `samples` SimpleAggregateFunction(timeSeriesGroupArray, Array(Tuple(timestamp DateTime64(6), value Float32))), `bucket` DateTime64(6) CODEC(Delta, T64, ZSTD(3)), `min_time` SimpleAggregateFunction(min, DateTime64(6)) CODEC(Delta, T64, ZSTD(3)), `max_time` SimpleAggregateFunction(max, DateTime64(6)) CODEC(Delta, T64, ZSTD(3)), CODEC `samples.timestamp` (Delta, T64, ZSTD(3)), CODEC `samples.value` (ALP, ZSTD(3)) AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, \'UTC\')) ORDER BY (id, bucket) SETTINGS index_granularity = 256, index_granularity_bytes = 262144
diff --git a/tests/queries/0_stateless/05141_time_series_samples_engine_settings.reference b/tests/queries/0_stateless/05141_time_series_samples_engine_settings.reference
index 8347b1fd2a2..de78af9c2f3 100644
--- a/tests/queries/0_stateless/05141_time_series_samples_engine_settings.reference
+++ b/tests/queries/0_stateless/05141_time_series_samples_engine_settings.reference
@@ -1,6 +1,6 @@
-- the defaults: one partition per month for the samples table, 5-hour partitions for the recent samples table
recentsamples AggregatingMergeTree PARTITION BY toStartOfInterval(toDateTime(bucket, \'UTC\'), toIntervalHour(5)) ORDER BY (id, bucket) TTL toDateTime(bucket) + toIntervalSecond(346500) SETTINGS index_granularity = 256, index_granularity_bytes = 131072, ttl_only_drop_parts = 1
-samples AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, \'UTC\')) ORDER BY (id, bucket) SETTINGS index_granularity = 256, index_granularity_bytes = 524288
+samples AggregatingMergeTree PARTITION BY toYYYYMM(toDateTime(bucket, \'UTC\')) ORDER BY (id, bucket) SETTINGS index_granularity = 256, index_granularity_bytes = 262144
202601 1
202602 2
-- the settings define the partition keys and the index granularity
cc @vitlibar (#117344) and @nikitamikhaylov (#113769, #120454). @groeneai could you take a look? If the analysis and patch hold up, feel free to open a PR if it fits.
- Ngôn ngữ chính
- C++
- Star
- 50.9k
- Fork
- 9.3k
- Merge trung bình
- 14 giờ 39 phút
- Pull request đã merge (30 ngày)
- 545
Chuẩn bị môi trường
- Không có Dockerfile hay tệp Docker Compose
- Có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của ClickHouse/ClickHouse
-
comp-cli
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
ClickHouse/ClickHouse#131191 · 2 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Unreadable /etc/ld.so.preload makes non-root ClickHouse refuse to startCó thể đã có người làm @lazerg đã nhận 2 ngày trước. Đang mởai_p2 comp-logging
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 76/100
ClickHouse/ClickHouse#130685 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
ai_p3 comp-foreign-db
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
ClickHouse/ClickHouse#130426 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
ai_p1 comp-aggregate-functions
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
ClickHouse/ClickHouse#130273 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Mutations on bucketed `Map` read all buckets in the per-part affected checkCó thể đã có người làm @Rushikesh1911 đã nhận 4 ngày trước. Đang mởai_p2 comp-mutations
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
ClickHouse/ClickHouse#130085 · 2 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày
Tất cả issue của ClickHouse/ClickHouse
Issue tương tự
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
Maintainer thường phản hồi trong vòng 2 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
mw00/project-maya#983 · 1 bình luận ·
Maintainer thường phản hồi trong vòng 2 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 62/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
StarRocks/starrocks#83246 · 1 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày