Code review skill produces wildly different results between Claude and Codex
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 1〜2日
- 初心者へのやさしさ
- 47/100
- issue の種類
- バグ
- 明瞭さ
- おおむね明確
- 活発さ
- 静か
- 技術スタック
- typescript
調査の方向性
The report describes inconsistent findings from /ce-code-review in Codex and Claude and mentions a possible connection to issue #799, but names no implementation files or tests. Start by reading the skill implementation and issue #799, then compare how reviewer outputs are collected and synthesized across the two integrations. Done means the same diff produces a consistent, adequately reconciled report, with a regression check covering the reported behavior.
索引モデルが issue の本文から書いたものです。
説明
Hi there,
First of all, thank you for creating such a great plugin. I've been using it for some time now and it's been great.
I've primarily been a Claude user, but lately I've been testing Codex and I've noticed a big difference in how the /ce-code-review skill behaves between the two.
When used against the same task/commit diff, Codex tends to return single-digit findings, while Claude usually produces double-digit findings. The difference between the two is pretty significant.
When I asked Codex why there was such a difference between the results, it said:
I did not spawn the full applicable reviewer set for the final diff.
I did not create/read a fresh run artifact and then synthesize from every reviewer output.
I over-applied the “validate and drop weak findings” mindset before showing you the raw-to-final mapping.
Claude Code has a more direct implementation path for this CE skill; in Codex I had to manually emulate it with spawn_agent, and I under-emulated it.
This isn't a one-off, it consistently happens every time I run the skill. Is this a bug or a limitation of Codex? I'm also not sure if it's related to this https://github.com/EveryInc/compound-engineering-plugin/issues/799 ?
Edit: here's some more comments from Codex:
Why results diverged:
Claude artifacts include raw reviewer output, not just final merged findings. Many items are testing_gaps or residual_risks, not primary findings. My final report only surfaced the main actionable findings and summarized coverage too tightly.
A lot of Claude’s raw findings violate the skill’s strict merge schema: severity values like high/medium/low, confidence values like 90/95/80. Under the skill’s Stage 5 validation, those are supposed to be dropped or normalized before final synthesis. I dropped some instead of salvaging/normalizing useful content.
I demoted or omitted several “coverage-only” items that Claude preserved, ....
Reviewer nondeterminism is real here. Different harness, model tiering, and agent implementations produced different issue discovery. But that does not fully excuse it: several Claude findings are valid and I should have caught or at least surfaced them as testing gaps/residual risks.
I also compressed the final report too much after applying the one safe fix. The skill asks for richer Coverage, Deployment Notes, and residual/actionable tables. My final answer was materially thinner than the Claude artifact set.
So the honest answer: I did spawn the required agents, but my synthesis pass was stricter and less complete than it should have been. The biggest actual miss is not agent coverage; it is that I failed to reconcile valid raw signals into the final report.
- 主要言語
- TypeScript
- スター
- 25.6k
- フォーク
- 2.1k
- 平均マージ
- 3時間 57分
- マージ済み PR(30日)
- 170
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
EveryInc/compound-engineering-plugin のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
EveryInc/compound-engineering-plugin#2085 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 66/100
EveryInc/compound-engineering-plugin#795 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 3/5 1〜2日 初心者へのやさしさ 80/100
EveryInc/compound-engineering-plugin#2084 ·
メンテナーはふだん 1 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 48/100
EveryInc/compound-engineering-plugin#2083 ·
メンテナーはふだん 1 日以内に返信
-
Host support: DeepSeek Harness (dsh)対応中かも このイシューにリンクされたプルリクエストがオープン中、またはマージ済みです。 オープン
難易度 3/5 1〜2日 初心者へのやさしさ 48/100
EveryInc/compound-engineering-plugin#2072 ·
メンテナーはふだん 1 日以内に返信
EveryInc/compound-engineering-plugin の issue をすべて見る
似ている issue
-
bug impact:low Team: SecuritySolution Team:Defend Workflows v9.6.0
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
elastic/kibana#306401 · コメント 3 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
agentes-chat/agentes-chat#927 ·
メンテナーはふだん 1 日以内に返信
-
comp/desktop P2 type/bug
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
NousResearch/hermes-agent#157702 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
メンテナーはふだん 1 日以内に返信
-
e2e-failure ready-to-code
難易度 1/5 1時間未満 初心者へのやさしさ 72/100
redhat-developer/rhdh-plugin-export-overlays#4794 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信