Add collector parameter contract assertion convention to scorecard AGENTS.md

オープン 初心者向け
  • 何か問題がありますか？
#4,734 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@fullsend-ai-coder がすでに取り組んでいます。

2026年9月14日 から。

  • #4735 @fullsend-ai-coder による — オープン

評価

難易度
2/5
見積もり時間
1〜3時間
初心者へのやさしさ
78/100
issue の種類
ドキュメント
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
typescript
領域
documentation, testing-qa

調査の方向性

workspaces/scorecard/AGENTS.md から始め、PR #4474 を確認します。特に collector mock と fallback の議論を確認してください。完全な入力に対するアサーション、エラーパスのカバレッジ、レビュー・チェックリストの項目を含む、要求された collector インタラクションテストのサブセクションを追加してください。今後の DORA provider のテスト変更について、ガイダンスが存在し、提示された検証基準に一致していれば完了です。

索引モデルが issue の本文から書いたものです。

説明

documentation ready-for-triage ready-to-code workspace/scorecard

What happened

In PR #4474, the review agent approved the test changes without flagging a test adequacy gap. Human reviewer gustavolira identified that the test mock for predecessor deployment collection only branched on input.from === DORA_PREDECESSOR_COLLECT_FROM, meaning that removing fetchItemsLimit: predecessorRange.fetchItemsLimit from the production code would leave the entire test suite green while production starts requesting 1000 deployments since 1970 on every DORA run.

The reviewer also noted that the catch/fallback path in collectSuccessfulProductionDeploymentsWithPreWindowBoundary had no test coverage — a transient rate limit during predecessor lookup would produce a misleading error message pointing at deployment data rather than the API.

The review agent's test assessment was limited to verifying that tests existed for the new functionality, not that they adequately covered the parameter contracts between callers and collectors.

What could go better

The scorecard workspace AGENTS.md has no guidance on what constitutes adequate test coverage for collector interactions. The existing mock pattern (mockResolvedValueOnce with conditional returns based on a single input field) makes it easy to add tests that pass without actually verifying the full contract between provider and collector.

The human reviewer caught this because they understood that fetchItemsLimit was the key behavioral difference between the normal collect and the predecessor collect — without it, the predecessor query degrades to an unbounded scan. This is a pattern-level issue: any time a provider passes structured input to a collector, tests should assert on the full input shape, not just the fields used for mock branching.

I am moderately confident this would help the review agent. The agent already reads and evaluates test files during review — giving it explicit criteria for what constitutes sufficient collector parameter testing would raise the bar from "tests exist" to "tests verify the contract." However, the agent's ability to reason about which parameters are behaviorally significant (vs. cosmetic) may still be limited without the pipeline context from proposal 1.

This proposal complements the pipeline guidance proposal — together, they give the agent both the domain context to understand why parameters matter and the testing convention to verify they're asserted.

Proposed change

Add a subsection to the proposed "DORA Collector Pipeline" section (or as a standalone section) in workspaces/scorecard/AGENTS.md titled "Testing conventions for collector interactions", covering:

  1. Assert full collector input contracts: When a provider calls collectorsService.collect(), tests must use toHaveBeenCalledWith (or equivalent) to assert on the full input object — including from, to, fetchItemsLimit, and any other parameters — not just the fields used for mock branching. Dropping a parameter from the production call must cause a test failure.

  2. Cover error/fallback paths: When collector calls are wrapped in try/catch with fallback behavior (e.g., falling back to fewer deployments on rate limit), add a test case using mockRejectedValueOnce to verify that (a) the fallback produces the correct output and (b) the warning message distinguishes the failure type.

  3. Review checklist item: When reviewing test changes for DORA providers, verify that mock assertions cover the distinguishing parameters of each collector call. Flag tests where the mock branches on one field but the production code sends additional fields that affect behavior.

Validation criteria

On the next 2 PRs that modify DORA provider tests in the scorecard workspace, the review agent should flag collector mock assertions that only branch on input.from without verifying other behavioral parameters like fetchItemsLimit or to. The review agent should also flag new try/catch blocks around collector calls that lack corresponding test coverage for the error path.

Generated by retro agent from https://github.com/redhat-developer/rhdh-plugins/pull/4474

主要言語
TypeScript
スター
51
フォーク
122
平均マージ
1日 16時間
マージ済み PR（30日）
275

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

redhat-developer/rhdh-plugins のほかの issue

redhat-developer/rhdh-plugins の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。