Report credits an inaccurate problem restatement as a strength

Aperta
  • Qualcosa non va?
#613 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@jserv ci sta già lavorando.

Dal 7/11/2026.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
rust
Ambito
ai, backend-api-design

Direzione di ricerca

Inspect the report generation path and the scoring and communication feedback prompts or rubrics referenced by the report: bundle 44, live prompt 33, report prompt 25, rubric 1, and report schema 4. Compare how the scripted restatement is evaluated with the scenario contract and its stated pitfall, then trace where Repeat and communication strengths are produced. Done means inaccurate or incomplete restatements lower Repeat and are reported as communication improvements.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Observed behavior

In a coding-only Editor interview on Single Resale Window (LeetCode 121), ended after Repeat and Example, the candidate's only restatement misstated the output (a price instead of the profit) and never stated that the sale must come after the purchase. The report still scored Repeat 90/100 in the framework assessment, and communicationFeedback.strengths[0] reads: "You clearly restated the problem requirements, including the input format and the goal of maximizing profit."

The same report's scenario contract requires i < j, and its first common pitfall is "Allowing a sell before the buy", so the report had the rule the restatement omitted.

Impact: a candidate who leaves out the key constraint is told their restatement was clear, so the report reinforces the gap instead of naming it.

Expected behavior

A restatement earns credit only for the inputs, output and rules it actually states. A misstated output, or an omitted contract rule that the brief lists as a pitfall, is a gap: it should lower Repeat and appear under communication improvements, not strengths.

Reproduction
  1. Start a coding-only Editor interview and select Single Resale Window.
  2. When Jim asks for a restatement, say: "The input is an integer array that represents the price every day. I need to find out which day is the best day to buy and which day is the best day to sell. To make the profit maximum. And I need to return the price. The maximum price."
  3. Give one example: decreasing prices 6, 5, 4, 3, 2, 1 return 0.
  4. End the interview and generate the report.
  5. Repeat scores 90/100 and communicationFeedback.strengths says the problem requirements were clearly restated.

Observed twice so far, on bundle 14 and bundle 44. The report is model-generated, so how often it happens is not yet measured.

CodeTrial version

codetrial 0.1.0 (cfcd576)

Environment

OS: Windows 11 with WSL2 (Ubuntu), server built from source
Browser: Google Chrome
Language: C++
Interview loop: Coding only, Editor
Problem: single-resale-window (LeetCode 121)
REACTO or STAR step: report (the claim concerns Repeat)
Reconnect: none

Sanitized evidence

Report header: bundle 44; live prompt 33; report prompt 25; rubric 1; report schema 4.

  • Phase scores: Repeat 90 / 100, Example 85 / 100.
  • communicationFeedback.strengths[0]: "You clearly restated the problem requirements, including the input format and the goal of maximizing profit."
  • Scenario contract in the same report: "bestSingleFlip(prices) returns the maximum of prices[j] - prices[i] over all day pairs with i < j, or 0 if no pair gives a positive difference (including a single day)."

The quoted restatement is the scripted line from a test session run to reproduce this, not a real candidate's interview.

Public data confirmation
  • I removed credentials, session tokens, candidate recordings, and personal interview content from this public report.
Lingua principale
Rust
Stelle
223
Fork
148
Merge medio
1g 7h
PR unite (30g)
225

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di sysprog21/codetrial

Tutte le issue di sysprog21/codetrial

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.