Report credits an inaccurate problem restatement as a strength

Abierto
  • ¿Algo no está bien?
#613 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@jserv ya está trabajando en esto.

Desde el 7/11/2026.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
rust
Área
ai, backend-api-design

Línea de trabajo

Inspect the report generation path and the scoring and communication feedback prompts or rubrics referenced by the report: bundle 44, live prompt 33, report prompt 25, rubric 1, and report schema 4. Compare how the scripted restatement is evaluated with the scenario contract and its stated pitfall, then trace where Repeat and communication strengths are produced. Done means inaccurate or incomplete restatements lower Repeat and are reported as communication improvements.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Observed behavior

In a coding-only Editor interview on Single Resale Window (LeetCode 121), ended after Repeat and Example, the candidate's only restatement misstated the output (a price instead of the profit) and never stated that the sale must come after the purchase. The report still scored Repeat 90/100 in the framework assessment, and communicationFeedback.strengths[0] reads: "You clearly restated the problem requirements, including the input format and the goal of maximizing profit."

The same report's scenario contract requires i < j, and its first common pitfall is "Allowing a sell before the buy", so the report had the rule the restatement omitted.

Impact: a candidate who leaves out the key constraint is told their restatement was clear, so the report reinforces the gap instead of naming it.

Expected behavior

A restatement earns credit only for the inputs, output and rules it actually states. A misstated output, or an omitted contract rule that the brief lists as a pitfall, is a gap: it should lower Repeat and appear under communication improvements, not strengths.

Reproduction
  1. Start a coding-only Editor interview and select Single Resale Window.
  2. When Jim asks for a restatement, say: "The input is an integer array that represents the price every day. I need to find out which day is the best day to buy and which day is the best day to sell. To make the profit maximum. And I need to return the price. The maximum price."
  3. Give one example: decreasing prices 6, 5, 4, 3, 2, 1 return 0.
  4. End the interview and generate the report.
  5. Repeat scores 90/100 and communicationFeedback.strengths says the problem requirements were clearly restated.

Observed twice so far, on bundle 14 and bundle 44. The report is model-generated, so how often it happens is not yet measured.

CodeTrial version

codetrial 0.1.0 (cfcd576)

Environment

OS: Windows 11 with WSL2 (Ubuntu), server built from source
Browser: Google Chrome
Language: C++
Interview loop: Coding only, Editor
Problem: single-resale-window (LeetCode 121)
REACTO or STAR step: report (the claim concerns Repeat)
Reconnect: none

Sanitized evidence

Report header: bundle 44; live prompt 33; report prompt 25; rubric 1; report schema 4.

  • Phase scores: Repeat 90 / 100, Example 85 / 100.
  • communicationFeedback.strengths[0]: "You clearly restated the problem requirements, including the input format and the goal of maximizing profit."
  • Scenario contract in the same report: "bestSingleFlip(prices) returns the maximum of prices[j] - prices[i] over all day pairs with i < j, or 0 if no pair gives a positive difference (including a single day)."

The quoted restatement is the scripted line from a test session run to reproduce this, not a real candidate's interview.

Public data confirmation
  • I removed credentials, session tokens, candidate recordings, and personal interview content from this public report.
Lenguaje dominante
Rust
Estrellas
223
Forks
148
Merge medio
1 d 7 h
PR fusionados (30 d)
225

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de sysprog21/codetrial

Todos los issues de sysprog21/codetrial

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.