Evaluation UI: Model Config
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- typescript
- Domain
- frontend
Research direction
No files or tests are named. Start by locating the evaluation configuration UI and the code that builds model configuration, then trace how the current model name, vector store ID, and system prompt are handled. Done means model-specific reasoning, verbosity, temperature, top_p, and max_tokens options can be configured for evaluations, including different combinations.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem?
The current evaluation config only supports basic options: model name, vector store ID, and system prompt. However, different models have different configuration parameters—newer models support reasoning and verbosity, while older models use temperature and top_p.
This limits our ability to properly evaluate models with their optimal settings.
Describe the solution you'd like
Add support for model-specific configuration options in the evaluation config:
- Newer models: reasoning, verbosity
- Older models: temperature, top_p, max_tokens
This would:
- Allow evaluations with model-appropriate parameters
- Enable testing different config combinations (e.g., high vs low temperature)
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 0
- Avg merge
- 10h 40m
- Merged PRs (30d)
- 4
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ProjectTech4DevAI/kaapi-frontend
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
ProjectTech4DevAI/kaapi-frontend#266 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ProjectTech4DevAI/kaapi-frontend#258 · 3 comments ·
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 55/100
All issues in ProjectTech4DevAI/kaapi-frontend
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
copse-dev/agent-pane#2953 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Eynzof/Hermes-CN-Desktop#610 ·
-
bug clawsweeper:linked-pr-open clawsweeper:needs-live-repro clawsweeper:no-new-fix-pr impact:message-loss issue-rating: 🐚 platinum hermit P2 regression
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
calcite-components needs triage refactor
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Esri/calcite-design-system#15203 ·