Measure where every skill's time goes from host transcripts (zero runtime cost), so measurement-gated speedups can be decided
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 42/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Stack tecnológico
- python, typescript
- Área
- developer-experience, tooling
Línea de trabajo
Start with skills/ce-compound/scripts/session-history/extract-metadata.py and extract-skeleton.py to see how host transcripts are detected and parsed; inspect tests/skill-eval-cell/run.ts:429-436 for the optional duration field. Define a read-only profiler under scripts/ that reports the listed timing, token, wait and repeated-read data, and fail softly on unknown formats. Done means the profiler works for the supported host transcripts and has one small fixture per host.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Problem
Only ce-code-review can show where its wall-clock goes. It writes stages.jsonl through skills/ce-code-review/scripts/run-log.py (#1723). No other skill records stage time. That includes ce-plan, ce-doc-review, ce-work, lfg and ce-brainstorm. The skill-eval harness records exitCode, timedOut, stdout_bytes, stderr_bytes, process_outcome per cell, with no duration or tokens (tests/skill-eval-cell/run.ts:429-436).
Several speed decisions are now explicitly waiting on numbers:
- #1815: "Two possible gates wait until they're measured: re-running only the affected
ce-doc-reviewreviewers in round 2, and runningce-simplify-codeinline on small scopes." docs/solutions/skill-design/review-cost-is-in-entering-the-spine-not-the-findings.mdlists per-stage instrumentation beyond code review as an open follow-up. It also says: "Before tuning a threshold, attribute the cost by stage from a real run."- The #1723 plan deferred a cross-run consumer because "nothing consumes it yet".
- The 12-turn Stage 4 re-read in the #1723 eval report was found only by hand-grading a tool trace.
As a user, I tried to answer "where does our CE time go?" from my own transcripts and could not separate agent time from human think-time without writing custom parsing.
Why not add run-log.py to every skill
lfg's step boundaries are skill invocations, not shell calls. The "fold the boundary into a shell call the step already makes" rule (skills/ce-code-review/references/scope.md:168) therefore has nothing to fold into, and each log call would be an extra turn at full orchestrator context. That is the cost the #1723 plan warned against ("The instrumentation must not spend what the split saves").
Proposal
Add a repo-level, read-only profiler under scripts/. It is maintainer tooling and does not ship in any skill. For each CE skill invocation in a host session transcript, it prints:
- wall-clock split into agent time and human think-time (gaps that end in a user turn or a question answer)
- assistant turns and cache-read, cache-write and output tokens, where the host records them
- the longest tool and subagent waits
- files read more than once
skills/ce-compound/scripts/session-history/extract-metadata.py and extract-skeleton.py already detect Claude, Codex, Cursor, Pi and omp transcripts and parse their records. They do not compute timing or usage today, but their platform detection is the natural starting point.
Optional and small: add duration_ms to each skill-eval cell record as a report-only column, with no thresholds.
Quality impact
None to skills: the tool is read-only, runs outside skill runs and adds no turns. Transcript formats drift, so it should fail soft on an unknown format and be pinned with one small fixture per host.
What it unblocks
It gives a direct answer to #1815's two deferred gates. It also gives the evidence needed for or against a ce-doc-review depth gate, overlapping doc-review synthesis with the cross-model peer tail, and starting the code-review peer earlier. And it lets anyone who reports "skill X feels slow" attach a reproducible table instead of an impression.
Drafted with Claude Code (claude-opus-5-5) while reviewing CE for cycle-time improvements. Evidence was checked against 9af474a7.
- Lenguaje dominante
- TypeScript
- Estrellas
- 25.6k
- Forks
- 2.1k
- Merge medio
- 3 h 57 min
- PR fusionados (30 d)
- 170
Preparar el entorno
Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de EveryInc/compound-engineering-plugin
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
EveryInc/compound-engineering-plugin#2085 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 66/100
EveryInc/compound-engineering-plugin#795 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 3/5 1-2 días Aptitud para principiantes 80/100
EveryInc/compound-engineering-plugin#2084 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
EveryInc/compound-engineering-plugin#2083 ·
Los mantenedores suelen responder en 1 día
-
Host support: DeepSeek Harness (dsh)Posiblemente ocupada Un pull request vinculado a esta issue está abierto o ya se fusionó. Abierto
Dificultad 3/5 1-2 días Aptitud para principiantes 48/100
EveryInc/compound-engineering-plugin#2072 ·
Los mantenedores suelen responder en 1 día
Todos los issues de EveryInc/compound-engineering-plugin
Issues similares
-
bug impact:low Team: SecuritySolution Team:Defend Workflows v9.6.0
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
elastic/kibana#306401 · 3 comentarios ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
agentes-chat/agentes-chat#927 ·
Los mantenedores suelen responder en 1 día
-
[Bug]: Webapp workspace fails to render in Safari — "Can't find variable: requestIdleCallback"Abiertocomp/desktop P2 type/bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
NousResearch/hermes-agent#157702 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
Los mantenedores suelen responder en 1 día
-
e2e-failure ready-to-code
Dificultad 1/5 Menos de una hora Aptitud para principiantes 72/100
redhat-developer/rhdh-plugin-export-overlays#4794 · 1 comentario ·
Los mantenedores suelen responder en 1 día