[optimization:token-optimizer] Optimize .github/workflows/eslint-rules-librarian.md in githubnext/gh-cao

Open
  • Is something wrong?
#20,016 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
35/100
Issue type
Refactor
Clarity
Clearly specified
Activity status
Active
Tech stack
bash, github-actions, javascript
Domain
ci-cd

Research direction

Open .github/workflows/eslint-rules-librarian.md and add the experiments: block as specified, keeping bash and repo-memory. Run gh aw compile --schedule-seed githubnext/gh-cao and npm run compile to validate. Done means the variant omitting tools.github shows >=3% lower median AIC over 10+ runs with no new errors and valid rules-db.mjs build.

Written by the indexing model from the issue text.

Description

optimization optimization:token-optimizer

The ESLint Factory / Librarian workflow (.github/workflows/eslint-rules-librarian.md) is the highest-cost eligible workflow without an open Optimization issue: 45 completed runs in the 7 full UTC days ending 2026-10-31T00:00Z, 45 successes, 0 errors, 847 total AI Credit (AIC), median 18.6 AIC per successful run, on claude-sonnet-5.5. In every audited run it only used bash and noop, yet it still declares the remote GitHub MCP repos toolset, so the proposed change is an A/B test that drops that toolset to shrink per-turn tool context.

Action: Assign this issue to Copilot to implement the recommendation, and accept only when the validation below passes.

Recommendation
  • Workflow: .github/workflows/eslint-rules-librarian.md. Category: tool-surface reduction (unused GitHub MCP toolset), validated with an experiment.
  • Baseline: cost grain = AIC per successful run of this workflow; window = 2026-10-24 to 2026-10-31 UTC; n=45; median 18.6 (range 16.7 to 22.5); cache-read about 11.4M and cache-write about 4.3M tokens in total (about 253k and 96k per run).
  • Change: add an experiments: block with metric: "aic" comparing the current tools.github (mode: remote, toolsets: [repos]) against a variant that omits tools.github. Keep bash and repo-memory. Do not change permissions, safe-outputs, network, or shared/control.md imports. If the shared control import re-adds GitHub tools, record that in the PR and leave the variant unchanged rather than weakening the control.
  • Conservative expected reduction: proposed, not realized. 3 to 6 percent of per-run AIC, about 0.5 to 1.1 AIC per run, or about 25 to 50 AIC per week at the current cadence. The estimate is deliberately low because token class attribution by tool schema is not available.
  • Safeguards: the workflow currently has a 100% success rate with 0 errors, so any increase in failures or errors blocks adoption. Each run must still append valid rule-normalization transactions or a documented "already normalized" noop, the rules-db.mjs build step must succeed, and the library must stay consistent. No max-ai-credits increase is proposed.
  • Validation: run gh aw compile --schedule-seed githubnext/gh-cao and npm run compile. Compare the two variants over the same 7-day UTC window, using successful-run AIC, median and total.
  • Acceptance: the variant's median AIC is at least 3% lower than the control's, with at least 10 completed runs per variant, no extra failures, cancellations, or errors, and the same share of runs producing valid transactions or a justified noop. Otherwise revert.
Evidence
  • Of the 12 most recent successful runs, none called any GitHub MCP tool. Each used only bash (4 to 10 calls) and noop.
  • The success rate is 100% (45 of 45) with 0 errors and a stable per-run cost of 16.7 to 22.5 AIC, so the baseline is steady enough to detect a small effect.
  • There is no open Optimization issue for this workflow. Higher-cost workflows were excluded because they already have open issues, or because they have large share of non-success runs (uk-ai-advisory, cao-evolution, dependabot), which would make a savings estimate unreliable.
Supporting evidence
  • Sample runs: 41116023109 (18.2 AIC), 41107895285 (19.7 AIC), 41041587387 (21.4 AIC).
  • Successful-run token totals over the window: input 1,343,835; output 70,000; cache-read 11,377,788; cache-write 4,333,296; reasoning 0 (not reported). Totals are not synthesized from the token classes.
  • Model: claude-sonnet-5.5. Operational-grader observations: none available, so no accepted-outcome claim is made. Outcome quality is proxied by valid transaction output and rebuild success, kept separate from repository operational value.
  • Tool calls per audited run: bash 4 to 10, noop 3, MCP GitHub 0 (12 runs audited, above the 5-run minimum).
  • Not recommended: inline agents (prompt has fewer than 3 independent major sections), a smaller model (quality evidence insufficient), and a max-ai-credits increase (the current 300 budget is about 15 times the median).
  • Limitations: Activity cache only. The agentic-workflows logs/audit tools were not used (they require a user role). Per-turn counts were not available in the cache.
  • Control envelope: correlation_id 41165930067-1374; central_repo githubnext/gh-cao; control_plane_run_url https://github.com/githubnext/gh-cao/actions/runs/41165930067; batch_label not set. Mode: review; target githubnext/gh-cao.
Agent prompt

Implement the named optimization in the selected workflow only. Preserve its behavior, permissions, safe outputs, and security boundaries. Run gh aw compile for the changed workflow and its focused tests. Report measured validation results and do not claim realized savings until a complete comparison window exists.

cao: githubnext/gh-cao, correlation: 41165930067-1374

Generated by Optimization / Token Optimizer · copilot · auto · ~40 AIC · ⌖ ~1 AIC · ⊞ 7.2K · ◷

  • expires on Nov 14, 2026, 3:59 AM UTC
Dominant language
JavaScript
Stars
8
Forks
5
Avg merge
3h 20m
Merged PRs (30d)
811

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from githubnext/gh-cao

All issues in githubnext/gh-cao

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.