feat(report): visualize evaluation history and comparison trends
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- go
- Domain
- data-visualization
Research direction
Start by reading the existing report artifact format and the comparison contract from #175; check how #317 represents named metrics and how reports are rendered. Then identify the current indexing, comparison, and report entry points in the repository. Done means retained runs can be indexed and compared locally with coverage, missing or failed states, configuration differences, and a rendered example, without rerunning evaluations.
Written by the indexing model from the issue text.
Description
Goal
Let Skill authors browse previous evaluation runs, track changes over time, and investigate improvements and regressions without manually opening individual result files.
Scope
- Index completed and interrupted run artifacts using stable run IDs, timestamps, suite/case identity, Skill source revision or fingerprint, and resolved engine/model/runtime metadata.
- Provide a local-first history view with filtering, baseline selection, and trend visualization for verdict/pass rate, available named metrics, token usage, and duration.
- Show case-level changes and link each point or comparison to its original report and supporting artifacts.
- Display missing data, failed runs, and metric coverage explicitly; distinguish execution errors from genuine zero scores.
- Check comparability of datasets, judge/rubric definitions, and metric definitions. Surface configuration differences and incompatible comparisons.
- Keep original results authoritative. A history index should be rebuildable from retained artifacts and must not require rerunning evaluations.
- Define opt-in indexing, storage location, and retention behavior without exposing credentials or raw sensitive configuration.
The presentation may be an extended HTML report or another local viewer; a hosted dashboard/service is not required.
Acceptance criteria
- Import/index at least three existing runs and browse them without calling an Agent or Judge.
- Select a baseline and visualize run-level trends and case-level fixed/regressed/added/removed transitions.
- Charts expose sample counts/coverage and retain explicit ERROR/SKIP/missing states.
- Named metric trends work when #317 data is available, while legacy boolean-only reports remain usable.
- Configuration changes and incompatible metric/rubric comparisons are visible.
- Rebuilding an index preserves run identity and handles duplicates, missing artifacts, and malformed files clearly.
- Regression tests cover indexing and comparison semantics, and a rendered example demonstrates the history view.
Related work
- #175 owns offline comparison of two result files and CI gates; reuse that comparison contract where available.
- #232 tracks automated evaluation of old versus new versions.
- #317 defines continuous scores and named metrics.
This issue presents retained results; scheduling concurrent multi-version evaluations is tracked separately.
- Dominant language
- Go
- Stars
- 1.5k
- Forks
- 134
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 61
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from alibaba/skill-up
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
OpenSandbox runtime: expect.files_exist and file_contains fail because WorkspacePath is remotePossibly taken @bababoom721 claimed this 1 day ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 25/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
alibaba/skill-up#408 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
alibaba/skill-up#407 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
alibaba/skill-up#406 · 1 comment ·
Maintainers usually reply within 1 day
All issues in alibaba/skill-up
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
OneBusAway/maglev#2145 ·
Maintainers usually reply within 2 days
-
priority/medium status/ready type/chore
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
discobox-ai/discobox#1357 ·
Maintainers usually reply within 1 day
-
TestPseudoLocaleRemote fails now and then: it types the code before the laptop asks for itPossibly taken @fzn0x claimed this today. Openbug priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
edit_file diff rendering panics on invalid UTF-8 textPossibly taken @TDnorthgarden claimed this today. Openarea/tui
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
docker/docker-agent#4955 ·
Maintainers usually reply within 1 day
-
enhancement ui/ux
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
jbigler/railswitch#243 ·
Maintainers usually reply within 1 day