pull-request-dashboard: replay corpus cannot validate routes end to end

未关闭
#192 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
48/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
冷清

调研方向

首先跟踪 normalize_events、compute_facts 和 route_pr,然后检查 otelbot/pull-request-dashboard-state/ 分支中现有的 harness 和 route 值。使用提交、状态检查和 mergeable 状态重建 441-PR corpus,将 collector 保留在 repository 中,并添加一个 replay,用于对比计算出的 route 与记录的 route。完成的标准是可以在整个 corpus 中比较 route 级别的分类变化。

由索引模型根据 Issue 内容生成。

描述

The 441 pull request replay corpus used to validate dashboard classification changes was collected without commits, so it cannot be used to compare whole-PR routes before and after a change.

normalize_events reads raw["commits"], and the harnesses work around this by passing {"commits": [], "review_comments": []}. That is fine for deriving discussion items and review threads, but it means compute_facts never sees pushes or CI, so route_pr cannot be evaluated. Any classification change can therefore only be measured at the discussion layer.

Concretely, the review-thread rewrite could be measured as "134 of 144 pull requests unchanged in whether any thread demands the author", but not as "N of 441 routes unchanged", which is the number that actually matters.

To close this:

  • re-collect the corpus with commits and whatever else compute_facts reads (status checks, mergeable state)
  • add a replay that computes route_pr per pull request and diffs against the route recorded in the otelbot/pull-request-dashboard-state/<repo> branches
  • keep the collector in the repo rather than as local scratch, so the corpus can be rebuilt

Until then, route-level claims about classification changes are not supported by evidence.

主要语言
JavaScript
星标
7
派生
23
平均合并
15 小时 46 分钟
30 天内合并 PR
41

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

open-telemetry/shared-workflows 的其他 Issue

查看 open-telemetry/shared-workflows 的全部 Issue

相似的 Issue

更多 JavaScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。