MCP: PromQL, dashboard-tile, pattern and `describe_source` tools return no cost statistics

Offen
#3,649 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
35/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
typescript

Rechercherichtung

Start with formatQueryResult in packages/api/src/mcp/tools/query/helpers.ts, then runPromqlTile and summarizeRows in queryTiles.ts, and the pattern and catalog tools. Run the attached repro test with make dev-int FILE=reproToolStatistics. Done when every query tool's response includes a statistics key, and clickstack_query_tiles tiles report elapsedMs, rowsRead and bytesRead.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

external

Summary

clickstack_table, clickstack_timeseries and clickstack_sql return ClickHouse's statistics (elapsed, rows_read, bytes_read) with every result, and agents use it to tell a cheap query from an expensive one. The tools agents use most during an investigation return nothing:

  • clickstack_promql
  • clickstack_query_tile on a PromQL tile (a builder tile does return statistics)
  • clickstack_query_tiles, which returns only status / hasData / rowCount per tile
  • clickstack_event_patterns and clickstack_emerging_signals
  • clickstack_describe_source

The builder tools have statistics only because ClickHouse's JSON output carries it and formatQueryResult (packages/api/src/mcp/tools/query/helpers.ts) passes the response through. The other tools build their own output (runPromqlTile, summarizeRows in queryTiles.ts, the pattern and catalog tools), and the numbers get dropped.

Environment

HyperDX main at c220e164 (2026-11-13), ClickHouse 26.10.3.7 (CI compose). Production-size numbers below are from a ClickHouse 26.10.3 server with about 110M log rows per hour and a TimeSeries table with about 217M series.

Steps to reproduce

reproToolStatistics.int.test.ts (below; copy to packages/api/src/mcp/__tests__/dashboards/) seeds a few log rows and a small TimeSeries table, then calls each tool and looks for a statistics key anywhere in the response:

clickstack_table                                 ok     statistics: {"elapsed":0.040143028,"rows_read":50,"bytes_read":483}
clickstack_timeseries                            ok     statistics: {"elapsed":0.007009402,"rows_read":50,"bytes_read":400}
clickstack_sql                                   ok     statistics: {"elapsed":0.001872216,"rows_read":1,"bytes_read":16}
clickstack_query_tile (builder tile)             ok     statistics: {"elapsed":0.00614494,"rows_read":50,"bytes_read":400}
clickstack_query_tile (promql tile)              ok     statistics: none
clickstack_promql                                ok     statistics: none
clickstack_query_tiles                           ok     statistics: none
clickstack_event_patterns                        ok     statistics: none
clickstack_emerging_signals                      ok     statistics: none
clickstack_describe_source (log source)          ok     statistics: none
clickstack_describe_source (promql source)       ok     statistics: none

Impact

These are the tools whose cost varies most. In 15 minutes of agent investigations on the production-size server, system.query_log showed:

Tool Queries Read in total Slowest Largest read
clickstack_query_tiles 271 217 GiB 25.8 s 10.2 GiB
clickstack_promql 60 73 GiB 25.8 s 20 GiB (the read cap)
clickstack_describe_source 60 62 GiB 5.9 s 5.0 GiB
clickstack_query_tile 10 16.5 GiB 8.0 s 5.8 GiB
clickstack_event_patterns 14 4.7 GiB 2.0 s 1.1 GiB
clickstack_emerging_signals 20 4.5 GiB 2.9 s 1.0 GiB

The agents saw none of this. A PromQL tile that read 10 GiB looked the same as one that read 4 MB, so they couldn't learn which queries or tiles to avoid, or compare a PromQL query with the builder equivalent. The same numbers per tile would also show which dashboard tiles are the expensive ones.

Proposed fix

  1. Return statistics from every query tool, in the same shape as clickstack_table: summed over the ClickHouse queries the call ran, plus a queries count.
  2. In clickstack_query_tiles, add elapsedMs, rowsRead and bytesRead to each tile summary.
  3. For PromQL through ClickHouse's prometheus_api_v1 handler, the response doesn't carry the numbers: on 26.10.3 its X-ClickHouse-Summary header reports read_rows: 0. HyperDX already sets X-ClickHouse-Query-Id (hdx-mcp-…), so the numbers could come from system.query_log for that id, or ClickHouse could report the inner query's progress in the summary header. Until then, return elapsed (wall time) on its own.

Related

  • #3540: MCP query attribution in log_comment
reproToolStatistics.int.test.ts
// Repro: which MCP query tools return ClickHouse cost statistics (elapsed, rows_read, bytes_read).
// Copy to packages/api/src/mcp/__tests__/dashboards/ and run:
//   make dev-int FILE=reproToolStatistics
import { DisplayType, SourceKind } from '@hyperdx/common-utils/dist/types';

import {
  bulkInsertLogs,
  DEFAULT_DATABASE,
  DEFAULT_LOGS_TABLE,
  dropTimeSeriesTable,
  seedTimeSeriesTagsTable,
} from '@/fixtures';
import { callTool, getFirstText } from '@/mcp/__tests__/mcpTestUtils';
import Dashboard from '@/models/dashboard';
import { Source } from '@/models/source';

import { setupDashboardTests } from './setup';

const mockFetch = jest.mocked(global.fetch);
const realFetch: typeof fetch = globalThis.realFetch;

const TABLE = 'repro_tool_statistics';

/** Every value under a key named `statistics`, anywhere in the response. */
const findStatistics = (node: unknown): unknown[] =>
  node == null || typeof node !== 'object'
    ? []
    : Object.entries(node).flatMap(([k, v]) =>
        k === 'statistics' ? [v] : findStatistics(v),
      );

describe('repro: cost statistics per MCP tool', () => {
  const ctx = setupDashboardTests();
  const now = Date.now();
  const start = new Date(now - 30 * 60_000).toISOString();
  const end = new Date(now + 60_000).toISOString();

  beforeAll(async () => {
    const nowSec = Math.floor(now / 1000);
    await seedTimeSeriesTagsTable({
      table: TABLE,
      series: ['a', 'b'].map(pod => ({
        metricName: 'repro_up',
        tags: { job: 'api', pod },
        startSec: nowSec - 1200,
        endSec: nowSec - 60,
      })),
      withSamples: true,
    });
  });
  afterAll(() => dropTimeSeriesTable({ table: TABLE }));

  it('prints which tools return statistics', async () => {
    await ctx.reconnect({ promqlEnabled: true });
    mockFetch.mockImplementation(realFetch);
    await bulkInsertLogs(
      Array.from({ length: 50 }, (_, i) => ({
        Body: `request ${i % 5} failed after ${i} ms`,
        ServiceName: 'api',
        SeverityText: i % 2 ? 'ERROR' : 'INFO',
        Timestamp: new Date(now - (i % 20) * 60_000),
      })),
    );
    const logSource = await Source.create({
      kind: SourceKind.Log,
      team: ctx.team._id,
      from: { databaseName: DEFAULT_DATABASE, tableName: DEFAULT_LOGS_TABLE },
      timestampValueExpression: 'Timestamp',
      connection: ctx.connection._id,
      name: 'Logs',
      bodyExpression: 'Body',
      severityTextExpression: 'SeverityText',
    });
    const promSource = await Source.create({
      kind: SourceKind.Promql,
      team: ctx.team._id,
      connection: ctx.connection._id,
      name: 'Metrics',
      from: { databaseName: DEFAULT_DATABASE, tableName: TABLE },
      timestampValueExpression: 'timestamp',
    });
    const tile = (id: string, config: Record<string, unknown>) => ({ id, x: 0, y: 0, w: 6, h: 3, config });
    const dashboard = await new Dashboard({
      name: 'Repro',
      tiles: [
        tile('promql', {
          configType: 'promql',
          name: 'Up by pod',
          displayType: DisplayType.Line,
          source: promSource._id.toString(),
          granularity: '1 minute',
          promqlExpression: [{ expression: 'sum by (pod) (repro_up)' }],
        }),
        tile('builder', {
          name: 'Log lines',
          displayType: DisplayType.Line,
          source: logSource._id.toString(),
          select: [{ aggFn: 'count', valueExpression: '' }],
          where: '',
          granularity: '1 minute',
        }),
      ],
      filters: [],
      team: ctx.team._id,
    }).save();
    const logId = logSource._id.toString();
    const promId = promSource._id.toString();
    const dashboardId = dashboard._id.toString();

    const calls: [string, Record<string, unknown>][] = [
      ['clickstack_table', { sourceId: logId, select: [{ aggFn: 'count' }], groupBy: 'SeverityText', startTime: start, endTime: end }],
      ['clickstack_timeseries', { sourceId: logId, select: [{ aggFn: 'count' }], startTime: start, endTime: end }],
      ['clickstack_sql', { connectionId: ctx.connection._id.toString(), sql: `SELECT count() FROM ${DEFAULT_DATABASE}.${DEFAULT_LOGS_TABLE}`, startTime: start, endTime: end }],
      ['clickstack_query_tile', { dashboardId, tileId: 'builder', startTime: start, endTime: end }],
      ['clickstack_query_tile', { dashboardId, tileId: 'promql', startTime: start, endTime: end }],
      ['clickstack_promql', { sourceId: promId, expression: 'sum by (pod) (repro_up)', startTime: start, endTime: end }],
      ['clickstack_query_tiles', { dashboardId, startTime: start, endTime: end }],
      ['clickstack_event_patterns', { sourceId: logId, startTime: start, endTime: end }],
      ['clickstack_emerging_signals', {
        sourceId: logId,
        baselineStartTime: new Date(now - 30 * 60_000).toISOString(),
        baselineEndTime: new Date(now - 10 * 60_000).toISOString(),
        currentStartTime: new Date(now - 10 * 60_000).toISOString(),
        currentEndTime: end,
      }],
      ['clickstack_describe_source', { sourceId: logId }],
      ['clickstack_describe_source', { sourceId: promId }],
    ];

    const lines: string[] = [];
    const has: Record<string, boolean> = {};
    for (const [tool, args] of calls) {
      const result = await callTool(ctx.client!, tool, args);
      const text = getFirstText(result);
      let stats: unknown[] = [];
      try {
        stats = findStatistics(JSON.parse(text));
      } catch {
        // not JSON
      }
      const label = `${tool}${args.tileId ? ` (${args.tileId} tile)` : ''}${tool === 'clickstack_describe_source' ? (args.sourceId === promId ? ' (promql source)' : ' (log source)') : ''}`;
      has[label] = stats.length > 0;
      lines.push(
        `${label.padEnd(48)} ${result.isError ? 'ERROR ' : 'ok    '} statistics: ${stats.length ? JSON.stringify(stats[0]) : 'none'}`,
      );
    }
    console.log(`REPRO\n${lines.join('\n')}`);

    expect(has['clickstack_table']).toBe(true);
    expect(has['clickstack_query_tile (builder tile)']).toBe(true);
    for (const label of [
      'clickstack_promql',
      'clickstack_query_tile (promql tile)',
      'clickstack_query_tiles',
      'clickstack_event_patterns',
      'clickstack_emerging_signals',
      'clickstack_describe_source (log source)',
      'clickstack_describe_source (promql source)',
    ]) {
      expect([label, has[label]]).toEqual([label, false]);
    }
  });
});
Vorherrschende Sprache
TypeScript
Sterne
10.1k
Forks
527
Ø Merge
1 T. 20 Std.
Gemergte PRs (30 T.)
186

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus ClickHouse/ClickStack

Alle Issues in ClickHouse/ClickStack

Ähnliche Issues

Weitere Issues zu TypeScript

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.