Report credits an inaccurate problem restatement as a strength

未关闭
  • 哪里不对？
#613 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

@jserv 已经在做这个了。

开始于 2026年11月7日。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
48/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
rust
领域
ai, backend-api-design

调研方向

Inspect the report generation path and the scoring and communication feedback prompts or rubrics referenced by the report: bundle 44, live prompt 33, report prompt 25, rubric 1, and report schema 4. Compare how the scripted restatement is evaluated with the scenario contract and its stated pitfall, then trace where Repeat and communication strengths are produced. Done means inaccurate or incomplete restatements lower Repeat and are reported as communication improvements.

由索引模型根据 Issue 内容生成。

描述

Observed behavior

In a coding-only Editor interview on Single Resale Window (LeetCode 121), ended after Repeat and Example, the candidate's only restatement misstated the output (a price instead of the profit) and never stated that the sale must come after the purchase. The report still scored Repeat 90/100 in the framework assessment, and communicationFeedback.strengths[0] reads: "You clearly restated the problem requirements, including the input format and the goal of maximizing profit."

The same report's scenario contract requires i < j, and its first common pitfall is "Allowing a sell before the buy", so the report had the rule the restatement omitted.

Impact: a candidate who leaves out the key constraint is told their restatement was clear, so the report reinforces the gap instead of naming it.

Expected behavior

A restatement earns credit only for the inputs, output and rules it actually states. A misstated output, or an omitted contract rule that the brief lists as a pitfall, is a gap: it should lower Repeat and appear under communication improvements, not strengths.

Reproduction
  1. Start a coding-only Editor interview and select Single Resale Window.
  2. When Jim asks for a restatement, say: "The input is an integer array that represents the price every day. I need to find out which day is the best day to buy and which day is the best day to sell. To make the profit maximum. And I need to return the price. The maximum price."
  3. Give one example: decreasing prices 6, 5, 4, 3, 2, 1 return 0.
  4. End the interview and generate the report.
  5. Repeat scores 90/100 and communicationFeedback.strengths says the problem requirements were clearly restated.

Observed twice so far, on bundle 14 and bundle 44. The report is model-generated, so how often it happens is not yet measured.

CodeTrial version

codetrial 0.1.0 (cfcd576)

Environment

OS: Windows 11 with WSL2 (Ubuntu), server built from source
Browser: Google Chrome
Language: C++
Interview loop: Coding only, Editor
Problem: single-resale-window (LeetCode 121)
REACTO or STAR step: report (the claim concerns Repeat)
Reconnect: none

Sanitized evidence

Report header: bundle 44; live prompt 33; report prompt 25; rubric 1; report schema 4.

  • Phase scores: Repeat 90 / 100, Example 85 / 100.
  • communicationFeedback.strengths[0]: "You clearly restated the problem requirements, including the input format and the goal of maximizing profit."
  • Scenario contract in the same report: "bestSingleFlip(prices) returns the maximum of prices[j] - prices[i] over all day pairs with i < j, or 0 if no pair gives a positive difference (including a single day)."

The quoted restatement is the scripted line from a test session run to reproduce this, not a real candidate's interview.

Public data confirmation
  • I removed credentials, session tokens, candidate recordings, and personal interview content from this public report.
主要语言
Rust
星标
223
派生
148
平均合并
1 天 7 小时
30 天内合并 PR
225

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

sysprog21/codetrial 的其他 Issue

查看 sysprog21/codetrial 的全部 Issue

相似的 Issue

更多 Rust Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。