Code review skill produces wildly different results between Claude and Codex
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 47/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Tranquilla
- Stack tecnologico
- typescript
- Ambito
- developer-experience
Direzione di ricerca
The report describes inconsistent findings from /ce-code-review in Codex and Claude and mentions a possible connection to issue #799, but names no implementation files or tests. Start by reading the skill implementation and issue #799, then compare how reviewer outputs are collected and synthesized across the two integrations. Done means the same diff produces a consistent, adequately reconciled report, with a regression check covering the reported behavior.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Hi there,
First of all, thank you for creating such a great plugin. I've been using it for some time now and it's been great.
I've primarily been a Claude user, but lately I've been testing Codex and I've noticed a big difference in how the /ce-code-review skill behaves between the two.
When used against the same task/commit diff, Codex tends to return single-digit findings, while Claude usually produces double-digit findings. The difference between the two is pretty significant.
When I asked Codex why there was such a difference between the results, it said:
I did not spawn the full applicable reviewer set for the final diff.
I did not create/read a fresh run artifact and then synthesize from every reviewer output.
I over-applied the “validate and drop weak findings” mindset before showing you the raw-to-final mapping.
Claude Code has a more direct implementation path for this CE skill; in Codex I had to manually emulate it with spawn_agent, and I under-emulated it.
This isn't a one-off, it consistently happens every time I run the skill. Is this a bug or a limitation of Codex? I'm also not sure if it's related to this https://github.com/EveryInc/compound-engineering-plugin/issues/799 ?
Edit: here's some more comments from Codex:
Why results diverged:
Claude artifacts include raw reviewer output, not just final merged findings. Many items are testing_gaps or residual_risks, not primary findings. My final report only surfaced the main actionable findings and summarized coverage too tightly.
A lot of Claude’s raw findings violate the skill’s strict merge schema: severity values like high/medium/low, confidence values like 90/95/80. Under the skill’s Stage 5 validation, those are supposed to be dropped or normalized before final synthesis. I dropped some instead of salvaging/normalizing useful content.
I demoted or omitted several “coverage-only” items that Claude preserved, ....
Reviewer nondeterminism is real here. Different harness, model tiering, and agent implementations produced different issue discovery. But that does not fully excuse it: several Claude findings are valid and I should have caught or at least surfaced them as testing gaps/residual risks.
I also compressed the final report too much after applying the one safe fix. The skill asks for richer Coverage, Deployment Notes, and residual/actionable tables. My final answer was materially thinner than the Claude artifact set.
So the honest answer: I did spawn the required agents, but my synthesis pass was stricter and less complete than it should have been. The biggest actual miss is not agent coverage; it is that I failed to reconcile valid raw signals into the final report.
- Lingua principale
- TypeScript
- Stelle
- 25.6k
- Fork
- 2.1k
- Merge medio
- 3h 57m
- PR unite (30g)
- 170
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di EveryInc/compound-engineering-plugin
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
EveryInc/compound-engineering-plugin#2085 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
EveryInc/compound-engineering-plugin#795 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 80/100
EveryInc/compound-engineering-plugin#2084 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 48/100
EveryInc/compound-engineering-plugin#2083 ·
I maintainer di solito rispondono entro 1 giorno
-
Host support: DeepSeek Harness (dsh)Forse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 3/5 1-2 giorni Idoneità per principianti 48/100
EveryInc/compound-engineering-plugin#2072 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di EveryInc/compound-engineering-plugin
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
I maintainer di solito rispondono entro 1 giorno
-
e2e-failure ready-to-code
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 72/100
redhat-developer/rhdh-plugin-export-overlays#4794 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 1 giorno
-
area: plugins documentation good first issue
Difficoltà 2/5 1-3 ore Idoneità per principianti 83/100
CatVinci-Studio/Textus#727 ·
I maintainer di solito rispondono entro 1 giorno