Add same-topic historical comparison and progress to interview reports
Los mantenedores suelen responder en 1 día
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 48/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Stack tecnológico
- javascript, rust
Línea de trabajo
Start by reading web/history.js to understand how completed reports and problem topics are stored, then inspect web/render.js and reportMarkup(...) for the report view. The issue also identifies web/practice-insights.js and tests under tests/browser/ as likely entry points. Done means showing same-topic metric and phase comparisons and resolved weaknesses, while handling first attempts and incomplete reports without changing storage or replay behavior.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Who is affected
A candidate reviewing their interview report after completing a coding interview. Currently, reports are rendered in isolation: even if a candidate deliberately practices multiple problems in the same study-plan category (such as 1D DP or Sliding Window) to work on a recurring gap, the finished report gives no direct comparison against their previous attempt in that category.
Desired outcome
When a candidate finishes a problem and has a prior assessed attempt in the same study-plan category, the report view renders a "Category Progress & Comparison" section. It compares objective session metrics (codingScore, communicationScore, hintsUsed, and same-rubric phase scores) and presents the previous attempt's phase-anchored improvementPlan alongside the current attempt's findings.
Present workaround
None in the report view. The lobby's "Topics to revisit" line (web/app.js:1350) only lists weak tags with more NO_HIRE than HIRE results over the last 8 sessions (web/practice-insights.js:43), and #389 filters lobby charts by a single problemId. Neither compares two reports side-by-side after an interview.
Scope Category
Mapping emitted by scripts/gen-problem-cards.py (from scripts/top-interview-150.json), comparison logic in web/practice-insights.js, report markup in web/render.js, and styles in web/styles.css. Does not alter live interviewer prompts, Gemini report calls, or existing lobby views.
Analysis of Key Design Choices
1. What "Same Category" Means, and Why Category Match Rather Than #389 problemId Match
- Why category transfer rather than repeating the same problemId: Re-solving the exact same problemId primarily measures recall and naturally inflates scores because the candidate has already seen the solution and edge cases. Comparing against a previous attempt on a different problem in the same category (e.g., finishing Climbing Stairs and next attempting House Robber in 1D DP) measures whether the candidate transferred the underlying skill to an unseen problem.
The Rule: Look up each problem's section in scripts/top-interview-150.json. Finding the previous attempt in the same category is then a simple lookup.
- Why scripts/top-interview-150.json sections instead of report.topics (web/problem-topics.json): In web/problem-topics.json, problems carry 1 to 5 tags mixing broad containers (Array, String, Matrix) with techniques (Dynamic Programming, Two Pointers, Stack, Monotonic Stack). Because over half the bank carries Array or String, matching on shared topics tags is ambiguous and pairs unrelated problems. By contrast, scripts/top-interview-150.json partitions the imported problem bank into 23 mutually exclusive study-plan categories (1D DP, Multidimensional DP, Two Pointers, Sliding Window, Intervals, Stack, Binary Tree General, Graph General, etc.), and check_plan_slugs in scripts/problem_bank/fetch.py:55 already enforces in CI that every imported slug appears in exactly one section (len(unique) == len(slugs)).
2. Matching improvementPlan Items Across Reports
Both improvementPlan[].weakness and improvementPlan[].drill are free-form prose generated per session by the report model (src/agent/report.rs:145). We compare three approaches for relating a prior report's improvement items to the current report:
-
Approach A: Direct or normalized text matching on
weakness(Rejected)
Compareprev.improvementPlan[].weaknessagainstcurr.improvementPlan[].weakness(even after case/punctuation normalization viaweaknessKeyinweb/progress.js:20). Because the model words every report afresh string matching almost always fails and would falsely label nearly every previous weakness as "resolved."web/progress.js:205explicitly documents this limitation. -
Approach B: Passing both reports to the LLM to generate an improvement diff (Rejected)
Send the previous category report and the current session state to Gemini so the model can semantically compare the two reports and write an improvement summary.- Why rejected:
- Untrusted browser storage & caching boundaries: Anonymous history lives in browser
localStorage(web/history.js), so the browser would have to upload a prior report into the model prompt.src/agent/prompts.rs:785bounds candidate-supplied context to 400 characters and forbids using it as a prior result or grading target, anddocs/provider-cost-and-degradation.md:295keeps report prompts scoped to the current session. - Quota, latency, and schema failure risk: Adding prompt tokens or a second Gemini call works against the token and retry budgets in
docs/provider-cost-and-degradation.mdand adds validation surface toreport_response_schema(). - No offline/historical support: An LLM-generated diff would not work when viewing already-saved reports in history.
- Untrusted browser storage & caching boundaries: Anonymous history lives in browser
- Why rejected:
-
Approach C: Phase-anchored comparison with side-by-side findings
MatchimprovementPlanitems by their closedphaseenum (repeat,example,algorithm,coding,test,optimizations, and the STAR phases), following the exact rule inweb/progress.js:205(// Grouped by phase, not by wording: the model words every report afresh...).- Do not claim a prose weakness was "resolved" via text matching.
- Group both the previous report's and current report's
improvementPlanitems byphase. - For each phase that had an
improvementPlanitem in the previous same-category attempt, display:- Phase score change: e.g.
Algorithm: 55 -> 80 (+25)(whenrubricVersionmatches). - Phase plan status: whether that phase has new
improvementPlanitems in the current report or had no items flagged this time. - Side-by-side prose: show the previous attempt's
weakness/drillfor that phase directly beside the current attempt'simprovementPlanitem(s) for that phase (or"No improvement item flagged in this phase"), allowing the candidate to read what changed without an unverified automated claim.
- Phase score change: e.g.
3. Comparing Scores When Grading Rules Change (rubricVersion)
Every saved report records a rubricVersion number inside frameworkAssessment (web/lib.js:1248), which marks which version of the grading rules was used to score the 6 REACTO steps. Over time, if CodeTrial updates how individual steps are graded (rubricVersion: 1 -> rubricVersion: 2), a 70 under version 1 and a 75 under version 2 were not graded on the same scale. That is why web/progress.js:154 does not connect chart lines across different rubricVersion numbers.
- Rule:
- Same
rubricVersion(the normal case): Show the score difference (curr.score - prev.score) for each REACTO step scored in both reports. - Different
rubricVersion(or missingframeworkAssessment): Still show the overallcodingScore,communicationScore,hintsUsed, and the side-by-side feedback by phase, but do not subtract the individual step scores—instead show a short note explaining that the two attempts used different rubric versions ("Step score changes omitted: previous attempt used rubric vX, current uses vY").
- Same
Acceptance criteria
- Every problem maps to a single category from
scripts/top-interview-150.json(plus explicit handling for"origin": "original"problems), verified by the generator gate. - When viewing a completed report (
!report.incomplete) that has a prior assessed attempt (HIREorNO_HIRE) in the same category, the report renders a "Category Progress" comparison section naming the category, the prior problem title, and the date. - The section shows deltas for
codingScore,communicationScore, andhintsUsed, and shows per-phase score deltas only whenframeworkAssessment.rubricVersionmatches on both reports. improvementPlanitems are compared by their closedphaseid (Approach C), showing prior and current phase items side-by-side without string-matching prose as "resolved."- When no prior assessed attempt exists in that category, or when the current report is
incomplete, the comparison section is omitted cleanly. - Existing lobby features—including "Topics to revisit" (
web/app.js:1350) and the progress charts (web/progress.js)—remain unchanged.
- Lenguaje dominante
- Rust
- Estrellas
- 177
- Forks
- 100
- Merge medio
- 1 d 9 h
- PR fusionados (30 d)
- 209
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de sysprog21/codetrial
-
Allow deselecting a chosen problem card by clicking it againPosiblemente ocupada @woohsin la tomó hace 4 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
sysprog21/codetrial#392 · 4 comentarios · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Audio output can be confirmed without playing a test soundPosiblemente ocupada @ChenYingHan1121 la tomó hace 6 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
sysprog21/codetrial#359 · 5 comentarios · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Add a Copy code button to the final reportPosiblemente ocupada @yansawu la tomó hace 7 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
sysprog21/codetrial#318 · 6 comentarios · 2 reacciones · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Add text objects to whiteboard and example drawingPosiblemente ocupada @Ethan921001 la tomó hoy. Abierto
sysprog21/codetrial#487 · 1 comentario · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Navigate to editor lines from compiler diagnosticsPosiblemente ocupada @yujoulai la tomó hoy. Abierto
sysprog21/codetrial#486 · 1 comentario · 1 asignado ·
Los mantenedores suelen responder en 1 día
Todos los issues de sysprog21/codetrial
Issues similares
-
api bug entity mobai
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
Pumpkin-MC/Pumpkin#4312 ·
Los mantenedores suelen responder en 1 día
-
bug easy good first issue
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
lacs-project/sysknife#659 ·
Los mantenedores suelen responder en 2 días
-
E_FIT_INPUT_MISSING (mixture overrides): the message says ferx-r bundles lack mixture_overridesAbiertoP3
Dificultad 1/5 Menos de una hora Aptitud para principiantes 83/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Document multi-value claims with rowBoundaries operator:in (and their per-record application)Abiertodocumentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
registrystack/registry-stack#2431 ·
Los mantenedores suelen responder en 1 día