Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Add same-topic historical comparison and progress to interview reports

Đang mở
#393 3 bình luận 0 reaction 1 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@Ezechungi7 đang làm issue này rồi.

Từ ngày 21/10/2026.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
48/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
javascript, rust
Lĩnh vực
analytics, frontend

Hướng nghiên cứu

Start by reading web/history.js to understand how completed reports and problem topics are stored, then inspect web/render.js and reportMarkup(...) for the report view. The issue also identifies web/practice-insights.js and tests under tests/browser/ as likely entry points. Done means showing same-topic metric and phase comparisons and resolved weaknesses, while handling first attempts and incomplete reports without changing storage or replay behavior.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Who is affected

A candidate reviewing their interview report after completing a coding interview. Currently, reports are rendered in isolation: even if a candidate deliberately practices multiple problems in the same study-plan category (such as 1D DP or Sliding Window) to work on a recurring gap, the finished report gives no direct comparison against their previous attempt in that category.

Desired outcome

When a candidate finishes a problem and has a prior assessed attempt in the same study-plan category, the report view renders a "Category Progress & Comparison" section. It compares objective session metrics (codingScore, communicationScore, hintsUsed, and same-rubric phase scores) and presents the previous attempt's phase-anchored improvementPlan alongside the current attempt's findings.

Present workaround

None in the report view. The lobby's "Topics to revisit" line (web/app.js:1350) only lists weak tags with more NO_HIRE than HIRE results over the last 8 sessions (web/practice-insights.js:43), and #389 filters lobby charts by a single problemId. Neither compares two reports side-by-side after an interview.

Scope Category

Mapping emitted by scripts/gen-problem-cards.py (from scripts/top-interview-150.json), comparison logic in web/practice-insights.js, report markup in web/render.js, and styles in web/styles.css. Does not alter live interviewer prompts, Gemini report calls, or existing lobby views.

Analysis of Key Design Choices
1. What "Same Category" Means, and Why Category Match Rather Than #389 problemId Match
  • Why category transfer rather than repeating the same problemId: Re-solving the exact same problemId primarily measures recall and naturally inflates scores because the candidate has already seen the solution and edge cases. Comparing against a previous attempt on a different problem in the same category (e.g., finishing Climbing Stairs and next attempting House Robber in 1D DP) measures whether the candidate transferred the underlying skill to an unseen problem.

The Rule: Look up each problem's section in scripts/top-interview-150.json. Finding the previous attempt in the same category is then a simple lookup.

  • Why scripts/top-interview-150.json sections instead of report.topics (web/problem-topics.json): In web/problem-topics.json, problems carry 1 to 5 tags mixing broad containers (Array, String, Matrix) with techniques (Dynamic Programming, Two Pointers, Stack, Monotonic Stack). Because over half the bank carries Array or String, matching on shared topics tags is ambiguous and pairs unrelated problems. By contrast, scripts/top-interview-150.json partitions the imported problem bank into 23 mutually exclusive study-plan categories (1D DP, Multidimensional DP, Two Pointers, Sliding Window, Intervals, Stack, Binary Tree General, Graph General, etc.), and check_plan_slugs in scripts/problem_bank/fetch.py:55 already enforces in CI that every imported slug appears in exactly one section (len(unique) == len(slugs)).
2. Matching improvementPlan Items Across Reports

Both improvementPlan[].weakness and improvementPlan[].drill are free-form prose generated per session by the report model (src/agent/report.rs:145). We compare three approaches for relating a prior report's improvement items to the current report:

  • Approach A: Direct or normalized text matching on weakness (Rejected)
    Compare prev.improvementPlan[].weakness against curr.improvementPlan[].weakness (even after case/punctuation normalization via weaknessKey in web/progress.js:20). Because the model words every report afresh string matching almost always fails and would falsely label nearly every previous weakness as "resolved." web/progress.js:205 explicitly documents this limitation.

  • Approach B: Passing both reports to the LLM to generate an improvement diff (Rejected)
    Send the previous category report and the current session state to Gemini so the model can semantically compare the two reports and write an improvement summary.

    • Why rejected:
      1. Untrusted browser storage & caching boundaries: Anonymous history lives in browser localStorage (web/history.js), so the browser would have to upload a prior report into the model prompt. src/agent/prompts.rs:785 bounds candidate-supplied context to 400 characters and forbids using it as a prior result or grading target, and docs/provider-cost-and-degradation.md:295 keeps report prompts scoped to the current session.
      2. Quota, latency, and schema failure risk: Adding prompt tokens or a second Gemini call works against the token and retry budgets in docs/provider-cost-and-degradation.md and adds validation surface to report_response_schema().
      3. No offline/historical support: An LLM-generated diff would not work when viewing already-saved reports in history.
  • Approach C: Phase-anchored comparison with side-by-side findings
    Match improvementPlan items by their closed phase enum (repeat, example, algorithm, coding, test, optimizations, and the STAR phases), following the exact rule in web/progress.js:205 (// Grouped by phase, not by wording: the model words every report afresh...).

    • Do not claim a prose weakness was "resolved" via text matching.
    • Group both the previous report's and current report's improvementPlan items by phase.
    • For each phase that had an improvementPlan item in the previous same-category attempt, display:
      1. Phase score change: e.g. Algorithm: 55 -> 80 (+25) (when rubricVersion matches).
      2. Phase plan status: whether that phase has new improvementPlan items in the current report or had no items flagged this time.
      3. Side-by-side prose: show the previous attempt's weakness / drill for that phase directly beside the current attempt's improvementPlan item(s) for that phase (or "No improvement item flagged in this phase"), allowing the candidate to read what changed without an unverified automated claim.
3. Comparing Scores When Grading Rules Change (rubricVersion)

Every saved report records a rubricVersion number inside frameworkAssessment (web/lib.js:1248), which marks which version of the grading rules was used to score the 6 REACTO steps. Over time, if CodeTrial updates how individual steps are graded (rubricVersion: 1 -> rubricVersion: 2), a 70 under version 1 and a 75 under version 2 were not graded on the same scale. That is why web/progress.js:154 does not connect chart lines across different rubricVersion numbers.

  • Rule:
    • Same rubricVersion (the normal case): Show the score difference (curr.score - prev.score) for each REACTO step scored in both reports.
    • Different rubricVersion (or missing frameworkAssessment): Still show the overall codingScore, communicationScore, hintsUsed, and the side-by-side feedback by phase, but do not subtract the individual step scores—instead show a short note explaining that the two attempts used different rubric versions ("Step score changes omitted: previous attempt used rubric vX, current uses vY").

Acceptance criteria

  1. Every problem maps to a single category from scripts/top-interview-150.json (plus explicit handling for "origin": "original" problems), verified by the generator gate.
  2. When viewing a completed report (!report.incomplete) that has a prior assessed attempt (HIRE or NO_HIRE) in the same category, the report renders a "Category Progress" comparison section naming the category, the prior problem title, and the date.
  3. The section shows deltas for codingScore, communicationScore, and hintsUsed, and shows per-phase score deltas only when frameworkAssessment.rubricVersion matches on both reports.
  4. improvementPlan items are compared by their closed phase id (Approach C), showing prior and current phase items side-by-side without string-matching prose as "resolved."
  5. When no prior assessed attempt exists in that category, or when the current report is incomplete, the comparison section is omitted cleanly.
  6. Existing lobby features—including "Topics to revisit" (web/app.js:1350) and the progress charts (web/progress.js)—remain unchanged.
Ngôn ngữ chính
Rust
Star
177
Fork
100
Merge trung bình
1 ngày 11 giờ
Pull request đã merge (30 ngày)
182

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của sysprog21/codetrial

Tất cả issue của sysprog21/codetrial

Issue tương tự

Thêm issue về Rust

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.