Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Add same-topic historical comparison and progress to interview reports

Abierto
#393 3 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@Ezechungi7 ya está trabajando en esto.

Desde el 21/10/2026.

  • #466 de @Ezechungi7 — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
javascript, rust

Línea de trabajo

Start by reading web/history.js to understand how completed reports and problem topics are stored, then inspect web/render.js and reportMarkup(...) for the report view. The issue also identifies web/practice-insights.js and tests under tests/browser/ as likely entry points. Done means showing same-topic metric and phase comparisons and resolved weaknesses, while handling first attempts and incomplete reports without changing storage or replay behavior.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Who is affected

A candidate reviewing their interview report after completing a coding interview. Currently, reports are rendered in isolation: even if a candidate deliberately practices multiple problems in the same study-plan category (such as 1D DP or Sliding Window) to work on a recurring gap, the finished report gives no direct comparison against their previous attempt in that category.

Desired outcome

When a candidate finishes a problem and has a prior assessed attempt in the same study-plan category, the report view renders a "Category Progress & Comparison" section. It compares objective session metrics (codingScore, communicationScore, hintsUsed, and same-rubric phase scores) and presents the previous attempt's phase-anchored improvementPlan alongside the current attempt's findings.

Present workaround

None in the report view. The lobby's "Topics to revisit" line (web/app.js:1350) only lists weak tags with more NO_HIRE than HIRE results over the last 8 sessions (web/practice-insights.js:43), and #389 filters lobby charts by a single problemId. Neither compares two reports side-by-side after an interview.

Scope Category

Mapping emitted by scripts/gen-problem-cards.py (from scripts/top-interview-150.json), comparison logic in web/practice-insights.js, report markup in web/render.js, and styles in web/styles.css. Does not alter live interviewer prompts, Gemini report calls, or existing lobby views.

Analysis of Key Design Choices
1. What "Same Category" Means, and Why Category Match Rather Than #389 problemId Match
  • Why category transfer rather than repeating the same problemId: Re-solving the exact same problemId primarily measures recall and naturally inflates scores because the candidate has already seen the solution and edge cases. Comparing against a previous attempt on a different problem in the same category (e.g., finishing Climbing Stairs and next attempting House Robber in 1D DP) measures whether the candidate transferred the underlying skill to an unseen problem.

The Rule: Look up each problem's section in scripts/top-interview-150.json. Finding the previous attempt in the same category is then a simple lookup.

  • Why scripts/top-interview-150.json sections instead of report.topics (web/problem-topics.json): In web/problem-topics.json, problems carry 1 to 5 tags mixing broad containers (Array, String, Matrix) with techniques (Dynamic Programming, Two Pointers, Stack, Monotonic Stack). Because over half the bank carries Array or String, matching on shared topics tags is ambiguous and pairs unrelated problems. By contrast, scripts/top-interview-150.json partitions the imported problem bank into 23 mutually exclusive study-plan categories (1D DP, Multidimensional DP, Two Pointers, Sliding Window, Intervals, Stack, Binary Tree General, Graph General, etc.), and check_plan_slugs in scripts/problem_bank/fetch.py:55 already enforces in CI that every imported slug appears in exactly one section (len(unique) == len(slugs)).
2. Matching improvementPlan Items Across Reports

Both improvementPlan[].weakness and improvementPlan[].drill are free-form prose generated per session by the report model (src/agent/report.rs:145). We compare three approaches for relating a prior report's improvement items to the current report:

  • Approach A: Direct or normalized text matching on weakness (Rejected)
    Compare prev.improvementPlan[].weakness against curr.improvementPlan[].weakness (even after case/punctuation normalization via weaknessKey in web/progress.js:20). Because the model words every report afresh string matching almost always fails and would falsely label nearly every previous weakness as "resolved." web/progress.js:205 explicitly documents this limitation.

  • Approach B: Passing both reports to the LLM to generate an improvement diff (Rejected)
    Send the previous category report and the current session state to Gemini so the model can semantically compare the two reports and write an improvement summary.

    • Why rejected:
      1. Untrusted browser storage & caching boundaries: Anonymous history lives in browser localStorage (web/history.js), so the browser would have to upload a prior report into the model prompt. src/agent/prompts.rs:785 bounds candidate-supplied context to 400 characters and forbids using it as a prior result or grading target, and docs/provider-cost-and-degradation.md:295 keeps report prompts scoped to the current session.
      2. Quota, latency, and schema failure risk: Adding prompt tokens or a second Gemini call works against the token and retry budgets in docs/provider-cost-and-degradation.md and adds validation surface to report_response_schema().
      3. No offline/historical support: An LLM-generated diff would not work when viewing already-saved reports in history.
  • Approach C: Phase-anchored comparison with side-by-side findings
    Match improvementPlan items by their closed phase enum (repeat, example, algorithm, coding, test, optimizations, and the STAR phases), following the exact rule in web/progress.js:205 (// Grouped by phase, not by wording: the model words every report afresh...).

    • Do not claim a prose weakness was "resolved" via text matching.
    • Group both the previous report's and current report's improvementPlan items by phase.
    • For each phase that had an improvementPlan item in the previous same-category attempt, display:
      1. Phase score change: e.g. Algorithm: 55 -> 80 (+25) (when rubricVersion matches).
      2. Phase plan status: whether that phase has new improvementPlan items in the current report or had no items flagged this time.
      3. Side-by-side prose: show the previous attempt's weakness / drill for that phase directly beside the current attempt's improvementPlan item(s) for that phase (or "No improvement item flagged in this phase"), allowing the candidate to read what changed without an unverified automated claim.
3. Comparing Scores When Grading Rules Change (rubricVersion)

Every saved report records a rubricVersion number inside frameworkAssessment (web/lib.js:1248), which marks which version of the grading rules was used to score the 6 REACTO steps. Over time, if CodeTrial updates how individual steps are graded (rubricVersion: 1 -> rubricVersion: 2), a 70 under version 1 and a 75 under version 2 were not graded on the same scale. That is why web/progress.js:154 does not connect chart lines across different rubricVersion numbers.

  • Rule:
    • Same rubricVersion (the normal case): Show the score difference (curr.score - prev.score) for each REACTO step scored in both reports.
    • Different rubricVersion (or missing frameworkAssessment): Still show the overall codingScore, communicationScore, hintsUsed, and the side-by-side feedback by phase, but do not subtract the individual step scores—instead show a short note explaining that the two attempts used different rubric versions ("Step score changes omitted: previous attempt used rubric vX, current uses vY").

Acceptance criteria

  1. Every problem maps to a single category from scripts/top-interview-150.json (plus explicit handling for "origin": "original" problems), verified by the generator gate.
  2. When viewing a completed report (!report.incomplete) that has a prior assessed attempt (HIRE or NO_HIRE) in the same category, the report renders a "Category Progress" comparison section naming the category, the prior problem title, and the date.
  3. The section shows deltas for codingScore, communicationScore, and hintsUsed, and shows per-phase score deltas only when frameworkAssessment.rubricVersion matches on both reports.
  4. improvementPlan items are compared by their closed phase id (Approach C), showing prior and current phase items side-by-side without string-matching prose as "resolved."
  5. When no prior assessed attempt exists in that category, or when the current report is incomplete, the comparison section is omitted cleanly.
  6. Existing lobby features—including "Topics to revisit" (web/app.js:1350) and the progress charts (web/progress.js)—remain unchanged.
Lenguaje dominante
Rust
Estrellas
177
Forks
100
Merge medio
1 d 9 h
PR fusionados (30 d)
209

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de sysprog21/codetrial

Todos los issues de sysprog21/codetrial

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.