Token counters left out of #2284 at v8.761.0: AgentTokenRollup, TokenXray, and UsageAggregator's first-wins

Aperta
#2,594 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 5 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
32/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
json, typescript
Ambito
devtools

Direzione di ricerca

Three separate token counters need fixing: TOOLS/AgentTokenRollup.ts (rollupAgent and main's readdirSync of subagents/), TOOLS/TokenXray/TokenXray.ts (computeCostTotals, rateCost, sideLogs), and TOOLS/UsageAggregator.ts (ingestTranscripts, transcriptFiles). Start by reading makeUsageLedger() in TokenXray.ts and pricingFor() in models.ts, then reproduce the undercount on a session with workflow agents. Done means the three tools dedupe per request, price per model, and include subagents/**/*.jsonl, with hand-computed test values.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

#2284 and #1894 fixed the main cost pipeline. Three token counters still have the same defects at tag v8.761.0. I read the code at that tag only, so if your source already covers them, please close this.

Measured on one install: 12,268 subagent replies, 436 workflow agent transcripts.

1. TOOLS/AgentTokenRollup.ts counts lines, and misses workflow agents

Cause. rollupAgent() adds message.usage for every line, with no per-request dedup. main() lists only the top level of subagents/ with readdirSync(found.dir), so subagents/workflows/<id>/agent-*.jsonl is never read.

Effect.

  • On 160 top-level agent transcripts it reports 2.85x the billable tokens and 2.49x the messages.
  • Workflow agents held 7,278 of the 12,268 subagent replies, so this tool cannot see most of them.
  • A session whose agents all ran in workflows reports 0 agents and 0 tokens. On my sample session the real figure was 22 agents and 4.0M billable tokens.

Suggested fix. Reuse the high-water ledger from TokenXray.ts (makeUsageLedger()), and walk with readdirSync(found.dir, { recursive: true }), filtering on the basename.

2. TOOLS/TokenXray/TokenXray.ts prices everything at Opus 4.7, and cost skips subagents

Cause.

  • computeCostTotals() multiplies all tokens by one fixed rates table (uncached: 5.0, output: 25.0), whatever the model. Same for rateCost() in split. On Opus 5.5 at $4/$20 with cache reads at 0.05x, or Fable 5.1 at $10/$50, every figure is off.
  • computeCostTotals() reads only mainLogs(), so subagents add nothing to cost or actual.
  • sideLogs() globs projects/*/*/subagents/*.jsonl, one level short of workflow agents.

Effect. Subagents and workflow agents were 59.5% of list-price cost over the logs on my disk, so cost leaves out more than half of it.

Suggested fix. Price each call with pricingFor(model) from models.ts, which already holds cacheWrite5m, cacheWrite1h and cacheRead per model. Haiku 5.5 needs its 100K prompt tier. Read mainLogs() and sideLogs() in computeCostTotals(), and change sideLogs() to subagents/**/*.jsonl.

3. TOOLS/UsageAggregator.ts: worth checking that the #2284 fix covers it

At the tag, ingestTranscripts() uses first-wins per file:

if (rid) { if (seenRequests.has(rid)) continue; seenRequests.add(rid); }

This is the trade-off your comment on #1894 warned about. output_tokens grows across a reply's lines, so the first line undercounts it: 0.60x of the per-reply maximum on my logs. seenRequests is also per file, so a reply copied into a second file by a resumed session counts twice, and transcriptFiles() stops one level short of workflow agents.

The reply on #2284 describes the ledger, so I can't tell whether this file got the same change.

How I checked

I fixed all three locally with one shared module (one counting rule, one price table), with tests whose expected values are computed by hand from the price page. After the fix, TokenXray's cost and the sum of session-costs.jsonl rows with a live transcript agree to the dollar on the same logs. Happy to share the code if useful.

Lingua principale
TypeScript
Stelle
19.9k
Fork
2.5k
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di danielmiessler/LifeOS

Tutte le issue di danielmiessler/LifeOS

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.