Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

ClickHouse: Cold runs preload data into memory before the timer

Abierto
  • ¿Algo no está bien?
#941 1 comentario 5 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
shell, sql
Área
databases, performance

Línea de trabajo

Lee primero la definición de true-cold de README.md y los comentarios de startup de ClickHouse/install. Traza cómo se representan la carga durante el startup y el borrado de la caché en el flujo del benchmark. La tarea estará terminada cuando la política de cold-run esté decidida explícitamente y el comportamiento del benchmark y la documentación coincidan.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Commit 7c1f7a3291521738a7731af38c994debb1ea3b1e enables preloading of the primary key on ClickHouse startup.

Quoting clickhouse/install:

# Force synchronous startup loading so the cold timer doesn't catch
# work that should have been amortized into ./start.
#
# Two independent layers of laziness contribute to the cold-query floor:
#
# 1. async_load_databases (server-level): with the default 1, the server
#    binds its listen port and answers SELECT 1 before user databases
#    have finished loading. ./check passes, then the first query stalls
#    waiting for the part loader.
#
# 2. primary_key_lazy_load and columns_and_secondary_indices_sizes_lazy_calculation
#    (MergeTree-level): even after parts are loaded, the in-memory
#    primary key and the per-column .size streams are populated lazily
#    on first query. With ~25 parts × ~80 columns × multiple metadata
#    files per column that's >1.8k file opens on the first query path,
#    contributing several hundred ms even on local NVMe.
#
# Both are eager-load toggles, not caching shortcuts: the same I/O
# happens either way, just before query timing instead of during it.
# Together they brought Q40 cold from ~3 s to ~1.5 s on c6a.4xlarge.

README.md defines true cold runs differently:

2.a) True cold runs. Before each first run of each query, all [...] database caches (e.g. buffer pools) are cleared.

In the current configuration, ClickHouse pre-warms the primary key data at startup, which defeats the clearing of all database caches. The measured cold runs times aren't truly cold anymore. While size-calculations might be considered metadata, primary key data is definitively data that's also queried by the benchmark queries.

I'm not sure there's a good definition which data is fine to be cached for a true cold run and which not.
Two of my suggestions below:

  1. Include startup times in "true cold" runs. This effectively prevents cheating by preloading data at startup.
  2. Drop the cold run altogether. "Cold" is clearly different for different systems and does not compare performance apples-to-apples
Lenguaje dominante
Shell
Estrellas
1.1k
Forks
321
Merge medio
3 h 57 min
PR fusionados (30 d)
607

Preparar el entorno

  • Sin Dockerfile ni archivo de Docker Compose
  • Tiene una plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de ClickHouse/ClickBench

Todos los issues de ClickHouse/ClickBench

Issues similares

Más issues de Shell/Bash

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.