Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[optimization:token-optimizer] Optimize cao-evolution-compiler-security.md in githubnext/gh-cao

Aperta
  • Qualcosa non va?
#19,589 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
github-actions, javascript
Ambito
devops, tooling

Direzione di ricerca

Read .github/workflows/cao-evolution-compiler-security.md and inspect its existing deterministic step, prompt, and experiment configuration. Implement the digest and prompt changes there, preserving the listed safeguards and decision rules. Validate with the specified gh aw compile, npm run compile, and npm test commands; done means validation passes and the experiment records the required AIC, reliability, and decision parity comparisons.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

optimization optimization:token-optimizer

The most expensive eligible workflow is .github/workflows/cao-evolution-compiler-security.md. In the 7-day window it had 8 completed runs (7 success, 1 early failure) and the highest mean cost: 31.5 AI Credit per successful run, median 29.9, total about 225 across the sample. The proposed change is to move report classification into the deterministic pre-step, so the model reads a compact digest instead of the raw compiler and scanner output.

Action: Assign this issue to Copilot, implement the recommendation in the named workflow only, and accept it only when the validation below passes.

Recommendation

  • Workflow / category: .github/workflows/cao-evolution-compiler-security.md, deterministic work assigned to the model (prompt/context reduction).

  • Baseline: AIC per successful run, window 2026-10-15 to 2026-10-22 UTC. n=7, range 28.8–38.0, median 29.9, mean 31.5. The run duration is 20–26 min, almost all in the deterministic compile step.

  • Proposed change:

    • Extend the existing deterministic step to emit a bounded digest.json/digest.txt (per-tool counts, deduplicated finding lines, severity as reported, and the image or workflow named). It should also record whether .grant.yaml is present.
    • Change the prompt to read the digest first and open report.txt only for specific excerpts needed to support a finding.
    • Keep the ownership classification and issue-creation judgment in the model.

  • Expected reduction (proposed, not realized): 3–8 AIC per run (about 10–25%). This is conservative: the cost above the ~21 AIC fixed floor seen in sibling workflows (catalog-advisor, efficiency) is variable. It tracks the extra cache-read and output tokens seen in the costly runs.

  • Safeguards: Do not change permissions, safe outputs, max-ai-credits, network, or tools. Do not remove agentic-workflows or bash, because tool usage was not audited per run. Preserve the target-owned versus upstream-owned rules and the noop and duplicate-search rules.

  • Experiment: Add an experiments: block comparing current against digest with metric: "aic". Use the same grain (AIC per successful run) and a 7-day window. Require at least 5 completed runs per variant.

  • Acceptance:

    • digest median AIC is at least 10% lower than current.
    • Failure and cancellation rate is no worse.
    • Issue-versus-noop decisions and target-owned finding counts match on the same target SHA.
    • No accepted outcome is lost.

  • Validation: gh aw compile --schedule-seed githubnext/gh-cao for the workflow, then npm run compile and npm test.

Evidence

[!NOTE]
Reliability is 7 of 8 successful runs. The one failure (35s, 1 error) is not cost-relevant.

  1. Costly runs have much higher cache-read tokens (407k–843k) and output tokens (2.9k–7.2k). Cheap sibling workflows read about 143k and write under 1k. The pre-step already produces all evidence deterministically.
  2. Cost has no clear relationship to run duration, so wall-clock time is not the lever.
  3. Operational-grader observations are null, so accepted-outcome value could not be measured. The acceptance criteria therefore rely on decision parity.
<b>Supporting evidence</b>

Per-run AIC (compiler-security, successful): 28.79, 28.89, 29.08, 29.88, 34.64, 36.19, 38.02.

Sibling floors: efficiency 21.1 (n=6); catalog-advisor 21.1–22.2 (one outlier at 34.5, cache-write 155k).

Candidates excluded: efficiency and catalog-advisor are close to their floor with fewer than 1k output tokens, so savings are immaterial. Other workflows had fewer than three completed runs or no cached AIC coverage in the Activity database. The Optimization campaign was excluded. The agentic-workflows fallback was not used. I could not search the 18 open optimization issues because they were integrity-filtered, so none are known to cover this workflow.

Runs: https://github.com/githubnext/gh-cao/actions/runs/39725549276, https://github.com/githubnext/gh-cao/actions/runs/39714568326, https://github.com/githubnext/gh-cao/actions/runs/39800741302

Control plane: correlation_id 39860226805-1177, central_repo githubnext/gh-cao, control_plane_run_url https://github.com/githubnext/gh-cao/actions/runs/39860226805, batch_label none.

<b>Agent prompt</b>

Implement the named optimization in the selected workflow only. Preserve its behavior, permissions, safe outputs, and security boundaries. Run gh aw compile for the changed workflow and its focused tests. Report measured validation results and do not claim realized savings until a complete comparison window exists.

Generated by Optimization / Token Optimizer · copilot · auto · 30.3 AIC · ⌖ 0.839 AIC · ⊞ 17.7K · ◷

  • expires on Nov 5, 2026, 9:56 PM UTC
Lingua principale
JavaScript
Stelle
8
Fork
5
Merge medio
2h 16m
PR unite (30g)
857

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di githubnext/gh-cao

Tutte le issue di githubnext/gh-cao

Issue simili

Altre issue su JavaScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.