[BUG] clickhouse: getTimeSeriesData drops dimension columns, collapsing long-format results into one series
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 58/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- typescript
Research direction
Start with clickhouse/src/queries/click-house-time-series-query/get-click-house-data.ts:26-48 and compare its transformation with greptimedb/src/queries/greptimedb-time-series-query/get-greptimedb-data.ts:124-143. Implement the stated partitioning and pivot behavior while preserving wide-result handling, then verify that dimension labels, grouped series, and multi-metric names match the proposed contract without changing the separate time-less case in #841.
Written by the indexing model from the issue text.
Description
Summary
ClickHouseTimeSeriesQuery.getTimeSeriesData() treats every column except time as a numeric metric column. Any string/dimension column is coerced with Number(), becomes null, and is then removed by the "drop all-null series" filter. For long-format (a.k.a. tall) SQL — the shape documented for the Grafana ClickHouse datasource this plugin is a migration target for — the dimension is silently destroyed and every series collapses into a single anonymous series.
Repro
SELECT toStartOfInterval(now(), INTERVAL 300 second) AS time,
machine AS series,
count() AS value
FROM system.query_log
GROUP BY time, series
ORDER BY time, series
Returned rows (abridged):
| time | series | value |
|---|---|---|
| 2026-09-29 12:00:00 | node-a | 41 |
| 2026-09-29 12:00:00 | node-b | 38 |
| 2026-09-29 12:05:00 | node-a | 44 |
| 2026-09-29 12:05:00 | node-b | 36 |
Expected: 2 series — node-a and node-b — each with 2 points.
Actual: 1 series named value with 4 points and duplicated timestamps (a zig-zag), because both machine and the per-node separation are gone.
Cause
clickhouse/src/queries/click-house-time-series-query/get-click-house-data.ts:26-48
const metricNames = Object.keys(data[0] ?? {}).filter((key) => key !== 'time'); // line 32
return metricNames
.map((metricName) => {
const values: Array<[number, number | null]> = data.map((row) => {
const timestamp = new Date(row.time).getTime();
const value = toTimeSeriesValue(row[metricName]); // Number('node-a') -> NaN -> null
return [timestamp, value];
});
return { name: metricName, values };
})
.filter((series) => series.values.some(([, value]) => value !== null)); // line 47: series dropped
There is no pivot step: the plugin can only express one series per column, never one series per (dimension, column).
Knock-on effect:
labelsis never populated, solegend/labels-based features (andTable) have nothing to render. Note the log query does setlabels(get-click-house-log-data.ts:62), so the contract is already understood elsewhere in the plugin, and the GreptimeDB plugin in this repo already pivots non-value columns intolabels(greptimedb/src/queries/greptimedb-time-series-query/get-greptimedb-data.ts:124-143).
(inferStepMs() is not affected: it sorts timestamps and skips non-positive deltas, so [t0, t0, t1, t1] still yields the correct step. Correcting an earlier claim of mine in this issue.)
Scope
We migrated 215 Grafana dashboards (Grafana + grafana-clickhouse-datasource → Perses) and replayed every query through the plugin's real transport:
| queries | |
|---|---|
| lose ≥1 column | 174 / 215 (81%) |
| dimension with ≥2 distinct values collapsed — chart/table is wrong | 109 |
| only the legend label lost (data still correct) | 65 |
| fully clean | 26 |
Per panel kind: TimeSeriesChart 73/141 broken, Table 30/30, GaugeChart 3/3, BarChart/PieChart/LogsTable 4/4, StatChart 1/37.
Real examples of the collapsed dimension:
| dashboard / panel | dropped column | distinct values |
|---|---|---|
baremetal-hosts "Network throughput" |
series |
244 (141 672 rows) |
ci-github-actions "Jobs completed per day" |
repository |
27 |
ceph-health "Stored per pool" |
series |
9 |
k8s-namespace-efficiency (37 namespaces) |
namespace column | 37 |
Compatibility note
grafana-clickhouse-datasource documents this exact shape — README, "Multi-line time series": "field 1: datetime field with an alias of time; field 2: value to group by; field 3+: the metric values" — and Grafana's own Time series dimensions describes building labels from the string columns of a tall result. $__columns(timeColumn, key, value) in the Grafana plugin expands to precisely this shape ("one series per key value").
Proposed behaviour
Detect dimension columns instead of assuming "everything except time is numeric":
- Partition the result columns into
time(a column namedtime— the alias the Grafana plugin itself requires to recognise a time series),numeric(every value isnull | '' | finite number) anddimension(the rest). - No dimension columns → keep the current behaviour exactly (wide results keep working; this is the 26 clean queries).
- ≥1 dimension column → pivot: group rows by the stringified dimension columns and emit one series per group, following the naming already used by the GreptimeDB plugin in this repo (
buildSeriesName:key=value,key=value):labels={ <dimColumn>: <value>, ... };- one numeric column →
name=key=value,...; - several numeric columns → one series per (dimension, numeric column),
name=key=value,... <valueColumn>and the value column also recorded inlabelsso row-oriented consumers (Table) can tell the rows apart.
- Results with no
timecolumn at all are handled in #841.
time-less results (pure table queries, e.g. SELECT app, team, count() FROM ... GROUP BY app, team) currently hit new Date(undefined).getTime() → NaN for every point and end up as a single series holding only the last row — that is filed separately in #841.
Environment
@perses-dev/clickhouse-plugin0.6.0and0.7.0-beta.6— behaviour is identical in both.- Perses
0.54.0.
- Dominant language
- TypeScript
- Stars
- 33
- Forks
- 95
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 40
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from perses/plugins
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 1 day
-
[Trino] Split long-format results into one series per labelPossibly taken @dougkirkley claimed this 2 days ago. Openenhancement
Difficulty 4/5 3-5 days Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
[BUG] splunk: export parsing, job polling and missing values are not handled robustlyPossibly taken @AnKiT-GaRG2 claimed this 22 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
[BUG] clickhouse: FORMAT substring check, envelope fallback and swallowed errorsPossibly taken @Yash121l claimed this 38 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 55/100
perses/plugins#842 · 1 comment ·
Maintainers usually reply within 1 day
Similar issues
-
db-d1-sqlite: upsert is aliased to updateOne, so payload.kv.set never inserts new keysPossibly taken @pcbeingused333 claimed this today. OpenBug db: d1-sqlite Payload OSS
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
payloadcms/payload#19766 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
device-management-toolkit/sample-web-ui#3711 ·
Maintainers usually reply within 1 day
-
bug impact:low Team: SecuritySolution Team:Defend Workflows v9.6.0
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
elastic/kibana#306401 · 3 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
agentes-chat/agentes-chat#927 ·
Maintainers usually reply within 1 day
-
Closing a pinned tab jumps to the Tabs section instead of the next open pinned tabPossibly taken @seenimohamed claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ardia-browser/ardia#2769 ·
Maintainers usually reply within 1 day