Add same-topic historical comparison and progress to interview reports
Maintainer thường phản hồi trong vòng 1 ngày
@Ezechungi7 đang làm issue này rồi.
Từ ngày 21/10/2026.
Đánh giá
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức phù hợp với người mới
- 48/100
- Loại issue
- Tính năng
- Độ rõ ràng
- Khá rõ ràng
- Mức độ hoạt động
- Sôi nổi
- Công nghệ
- javascript, rust
Hướng nghiên cứu
Start by reading web/history.js to understand how completed reports and problem topics are stored, then inspect web/render.js and reportMarkup(...) for the report view. The issue also identifies web/practice-insights.js and tests under tests/browser/ as likely entry points. Done means showing same-topic metric and phase comparisons and resolved weaknesses, while handling first attempts and incomplete reports without changing storage or replay behavior.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
Who is affected
A candidate reviewing their interview report after completing a coding interview. Currently, reports are rendered in isolation: even if a candidate deliberately practices multiple problems in the same study-plan category (such as 1D DP or Sliding Window) to work on a recurring gap, the finished report gives no direct comparison against their previous attempt in that category.
Desired outcome
When a candidate finishes a problem and has a prior assessed attempt in the same study-plan category, the report view renders a "Category Progress & Comparison" section. It compares objective session metrics (codingScore, communicationScore, hintsUsed, and same-rubric phase scores) and presents the previous attempt's phase-anchored improvementPlan alongside the current attempt's findings.
Present workaround
None in the report view. The lobby's "Topics to revisit" line (web/app.js:1350) only lists weak tags with more NO_HIRE than HIRE results over the last 8 sessions (web/practice-insights.js:43), and #389 filters lobby charts by a single problemId. Neither compares two reports side-by-side after an interview.
Scope Category
Mapping emitted by scripts/gen-problem-cards.py (from scripts/top-interview-150.json), comparison logic in web/practice-insights.js, report markup in web/render.js, and styles in web/styles.css. Does not alter live interviewer prompts, Gemini report calls, or existing lobby views.
Analysis of Key Design Choices
1. What "Same Category" Means, and Why Category Match Rather Than #389 problemId Match
- Why category transfer rather than repeating the same problemId: Re-solving the exact same problemId primarily measures recall and naturally inflates scores because the candidate has already seen the solution and edge cases. Comparing against a previous attempt on a different problem in the same category (e.g., finishing Climbing Stairs and next attempting House Robber in 1D DP) measures whether the candidate transferred the underlying skill to an unseen problem.
The Rule: Look up each problem's section in scripts/top-interview-150.json. Finding the previous attempt in the same category is then a simple lookup.
- Why scripts/top-interview-150.json sections instead of report.topics (web/problem-topics.json): In web/problem-topics.json, problems carry 1 to 5 tags mixing broad containers (Array, String, Matrix) with techniques (Dynamic Programming, Two Pointers, Stack, Monotonic Stack). Because over half the bank carries Array or String, matching on shared topics tags is ambiguous and pairs unrelated problems. By contrast, scripts/top-interview-150.json partitions the imported problem bank into 23 mutually exclusive study-plan categories (1D DP, Multidimensional DP, Two Pointers, Sliding Window, Intervals, Stack, Binary Tree General, Graph General, etc.), and check_plan_slugs in scripts/problem_bank/fetch.py:55 already enforces in CI that every imported slug appears in exactly one section (len(unique) == len(slugs)).
2. Matching improvementPlan Items Across Reports
Both improvementPlan[].weakness and improvementPlan[].drill are free-form prose generated per session by the report model (src/agent/report.rs:145). We compare three approaches for relating a prior report's improvement items to the current report:
-
Approach A: Direct or normalized text matching on
weakness(Rejected)
Compareprev.improvementPlan[].weaknessagainstcurr.improvementPlan[].weakness(even after case/punctuation normalization viaweaknessKeyinweb/progress.js:20). Because the model words every report afresh string matching almost always fails and would falsely label nearly every previous weakness as "resolved."web/progress.js:205explicitly documents this limitation. -
Approach B: Passing both reports to the LLM to generate an improvement diff (Rejected)
Send the previous category report and the current session state to Gemini so the model can semantically compare the two reports and write an improvement summary.- Why rejected:
- Untrusted browser storage & caching boundaries: Anonymous history lives in browser
localStorage(web/history.js), so the browser would have to upload a prior report into the model prompt.src/agent/prompts.rs:785bounds candidate-supplied context to 400 characters and forbids using it as a prior result or grading target, anddocs/provider-cost-and-degradation.md:295keeps report prompts scoped to the current session. - Quota, latency, and schema failure risk: Adding prompt tokens or a second Gemini call works against the token and retry budgets in
docs/provider-cost-and-degradation.mdand adds validation surface toreport_response_schema(). - No offline/historical support: An LLM-generated diff would not work when viewing already-saved reports in history.
- Untrusted browser storage & caching boundaries: Anonymous history lives in browser
- Why rejected:
-
Approach C: Phase-anchored comparison with side-by-side findings
MatchimprovementPlanitems by their closedphaseenum (repeat,example,algorithm,coding,test,optimizations, and the STAR phases), following the exact rule inweb/progress.js:205(// Grouped by phase, not by wording: the model words every report afresh...).- Do not claim a prose weakness was "resolved" via text matching.
- Group both the previous report's and current report's
improvementPlanitems byphase. - For each phase that had an
improvementPlanitem in the previous same-category attempt, display:- Phase score change: e.g.
Algorithm: 55 -> 80 (+25)(whenrubricVersionmatches). - Phase plan status: whether that phase has new
improvementPlanitems in the current report or had no items flagged this time. - Side-by-side prose: show the previous attempt's
weakness/drillfor that phase directly beside the current attempt'simprovementPlanitem(s) for that phase (or"No improvement item flagged in this phase"), allowing the candidate to read what changed without an unverified automated claim.
- Phase score change: e.g.
3. Comparing Scores When Grading Rules Change (rubricVersion)
Every saved report records a rubricVersion number inside frameworkAssessment (web/lib.js:1248), which marks which version of the grading rules was used to score the 6 REACTO steps. Over time, if CodeTrial updates how individual steps are graded (rubricVersion: 1 -> rubricVersion: 2), a 70 under version 1 and a 75 under version 2 were not graded on the same scale. That is why web/progress.js:154 does not connect chart lines across different rubricVersion numbers.
- Rule:
- Same
rubricVersion(the normal case): Show the score difference (curr.score - prev.score) for each REACTO step scored in both reports. - Different
rubricVersion(or missingframeworkAssessment): Still show the overallcodingScore,communicationScore,hintsUsed, and the side-by-side feedback by phase, but do not subtract the individual step scores—instead show a short note explaining that the two attempts used different rubric versions ("Step score changes omitted: previous attempt used rubric vX, current uses vY").
- Same
Acceptance criteria
- Every problem maps to a single category from
scripts/top-interview-150.json(plus explicit handling for"origin": "original"problems), verified by the generator gate. - When viewing a completed report (
!report.incomplete) that has a prior assessed attempt (HIREorNO_HIRE) in the same category, the report renders a "Category Progress" comparison section naming the category, the prior problem title, and the date. - The section shows deltas for
codingScore,communicationScore, andhintsUsed, and shows per-phase score deltas only whenframeworkAssessment.rubricVersionmatches on both reports. improvementPlanitems are compared by their closedphaseid (Approach C), showing prior and current phase items side-by-side without string-matching prose as "resolved."- When no prior assessed attempt exists in that category, or when the current report is
incomplete, the comparison section is omitted cleanly. - Existing lobby features—including "Topics to revisit" (
web/app.js:1350) and the progress charts (web/progress.js)—remain unchanged.
- Ngôn ngữ chính
- Rust
- Star
- 177
- Fork
- 100
- Merge trung bình
- 1 ngày 11 giờ
- Pull request đã merge (30 ngày)
- 182
Chuẩn bị môi trường
- Không có Dockerfile hay tệp Docker Compose
- Không có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của sysprog21/codetrial
-
Selecting a pen color leaves Example drawing in eraser modeCó thể đã có người làm @Aa99841 đã nhận hôm nay. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
sysprog21/codetrial#437 · 4 bình luận · 1 người được giao ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Python tests lose every result when the solution returns inf or NaNCó thể đã có người làm @shensyuan đã nhận 2 ngày trước. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 77/100
sysprog21/codetrial#419 · 3 bình luận · 1 người được giao ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Allow deselecting a chosen problem card by clicking it againCó thể đã có người làm @woohsin đã nhận 2 ngày trước. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
sysprog21/codetrial#392 · 4 bình luận · 1 người được giao ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Audio output can be confirmed without playing a test soundCó thể đã có người làm @ChenYingHan1121 đã nhận 4 ngày trước. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 62/100
sysprog21/codetrial#359 · 5 bình luận · 1 người được giao ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Add a Copy code button to the final reportCó thể đã có người làm @yansawu đã nhận 5 ngày trước. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 80/100
sysprog21/codetrial#318 · 6 bình luận · 2 reaction · 1 người được giao ·
Maintainer thường phản hồi trong vòng 1 ngày
Tất cả issue của sysprog21/codetrial
Issue tương tự
-
Docker host resolution missing default rootless Docker pathCó thể đã có người làm @500B đã nhận hôm nay. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 88/100
testcontainers/testcontainers-rs#1000 ·
Maintainer thường phản hồi trong vòng 3 ngày
-
bug needs triage P3
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 76/100
pbakaus/impeccable#1143 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Bundled uv is never updated and trusts the compromised eMudhra certificatesCó thể đã có người làm @FabienDanieau đã nhận hôm nay. Đang mởbug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 76/100
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 66/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 62/100
foxglove/mcap#1907 · 1 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày