Token counters left out of #2284 at v8.761.0: AgentTokenRollup, TokenXray, and UsageAggregator's first-wins

Open
#2,594 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 5 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
32/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
json, typescript
Domain
devtools

Research direction

Three separate token counters need fixing: TOOLS/AgentTokenRollup.ts (rollupAgent and main's readdirSync of subagents/), TOOLS/TokenXray/TokenXray.ts (computeCostTotals, rateCost, sideLogs), and TOOLS/UsageAggregator.ts (ingestTranscripts, transcriptFiles). Start by reading makeUsageLedger() in TokenXray.ts and pricingFor() in models.ts, then reproduce the undercount on a session with workflow agents. Done means the three tools dedupe per request, price per model, and include subagents/**/*.jsonl, with hand-computed test values.

Written by the indexing model from the issue text.

Description

#2284 and #1894 fixed the main cost pipeline. Three token counters still have the same defects at tag v8.761.0. I read the code at that tag only, so if your source already covers them, please close this.

Measured on one install: 12,268 subagent replies, 436 workflow agent transcripts.

1. TOOLS/AgentTokenRollup.ts counts lines, and misses workflow agents

Cause. rollupAgent() adds message.usage for every line, with no per-request dedup. main() lists only the top level of subagents/ with readdirSync(found.dir), so subagents/workflows/<id>/agent-*.jsonl is never read.

Effect.

  • On 160 top-level agent transcripts it reports 2.85x the billable tokens and 2.49x the messages.
  • Workflow agents held 7,278 of the 12,268 subagent replies, so this tool cannot see most of them.
  • A session whose agents all ran in workflows reports 0 agents and 0 tokens. On my sample session the real figure was 22 agents and 4.0M billable tokens.

Suggested fix. Reuse the high-water ledger from TokenXray.ts (makeUsageLedger()), and walk with readdirSync(found.dir, { recursive: true }), filtering on the basename.

2. TOOLS/TokenXray/TokenXray.ts prices everything at Opus 4.7, and cost skips subagents

Cause.

  • computeCostTotals() multiplies all tokens by one fixed rates table (uncached: 5.0, output: 25.0), whatever the model. Same for rateCost() in split. On Opus 5.5 at $4/$20 with cache reads at 0.05x, or Fable 5.1 at $10/$50, every figure is off.
  • computeCostTotals() reads only mainLogs(), so subagents add nothing to cost or actual.
  • sideLogs() globs projects/*/*/subagents/*.jsonl, one level short of workflow agents.

Effect. Subagents and workflow agents were 59.5% of list-price cost over the logs on my disk, so cost leaves out more than half of it.

Suggested fix. Price each call with pricingFor(model) from models.ts, which already holds cacheWrite5m, cacheWrite1h and cacheRead per model. Haiku 5.5 needs its 100K prompt tier. Read mainLogs() and sideLogs() in computeCostTotals(), and change sideLogs() to subagents/**/*.jsonl.

3. TOOLS/UsageAggregator.ts: worth checking that the #2284 fix covers it

At the tag, ingestTranscripts() uses first-wins per file:

if (rid) { if (seenRequests.has(rid)) continue; seenRequests.add(rid); }

This is the trade-off your comment on #1894 warned about. output_tokens grows across a reply's lines, so the first line undercounts it: 0.60x of the per-reply maximum on my logs. seenRequests is also per file, so a reply copied into a second file by a resumed session counts twice, and transcriptFiles() stops one level short of workflow agents.

The reply on #2284 describes the ledger, so I can't tell whether this file got the same change.

How I checked

I fixed all three locally with one shared module (one counting rule, one price table), with tests whose expected values are computed by hand from the price page. After the fix, TokenXray's cost and the sum of session-costs.jsonl rows with a live transcript agree to the dollar on the same logs. Happy to share the code if useful.

Dominant language
TypeScript
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from danielmiessler/LifeOS

All issues in danielmiessler/LifeOS

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.