Log per-stage input/output tokens in the token tracker (baseline for token reduction)

Open
  • Is something wrong?
#41 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 6 days

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
62/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
ai-infra-agents, javascript
Domain
devtools

Research direction

Start in bundle/hooks/lib/token-tracker.js, at recordPhaseTokens (around L307-343) and printTokenReport (around L622), and check how the handoff payload from buildHandoffEnforcement in bundle/hooks/lib/handoff-enforcer.js is sized against MAX_HANDOFF_CHARS. Add handoff_chars and handoff_tokens_est to each phase entry, then extend the fake 4-stage case in scripts/test-adaptive-loop.js. Done when per-stage input and output estimates appear in the report, the test passes, and bundle/opencode/hooks/lib/token-tracker.js matches byte-for-byte.

Written by the indexing model from the issue text.

Description

enhancement good first issue

Summary

Beginner-friendly first step of "Reduce AI token usage": make the token tracker log input and output tokens per stage so a baseline exists.

Context

  • bundle/hooks/lib/token-tracker.js: estimateTokens (L70-73) is ceil(chars/4) and recordPhaseTokens (L307-343) estimates from extractOutputText(input) — i.e. only the subagent's output, unless the host reports usage (extractReportedUsage).
  • Each phase entry already stores output_chars, estimated_tokens, reported, cache_read_tokens, cache_hit_rate, over_budget, budget.
  • The handoff size (buildHandoffEnforcement in bundle/hooks/lib/handoff-enforcer.js; MAX_HANDOFF_CHARS = 2500) is not recorded.

Task

  • Record handoff_chars / handoff_tokens_est (the payload sent into each stage) in the phase entry.
  • Show input vs output per stage in printTokenReport (token-tracker.js:622).
  • Add a unit test in scripts/test-adaptive-loop.js (it already covers the token tracker) that checks both fields exist for a fake 4-stage run.
  • Mention the new fields in docs/ where token budgets are described.

Acceptance criteria

  • A run produces per-stage {input_est, output_est, reported?}.
  • Tests pass; the opencode mirror (bundle/opencode/hooks/lib/token-tracker.js) is kept byte-identical.

Part of the token-usage reduction issue (linked there).

Raised from the v3.8.0 audit.

Dominant language
JavaScript
Stars
0
Forks
0
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from P-Core-System/pcore-orchestra

All issues in P-Core-System/pcore-orchestra

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.