Idea: Use modules instead of YAML for evals
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start by reviewing the existing YAML definitions and the assertion code referenced in the issue, then read the comment discussion for the unresolved design concerns. Done means there is an agreed approach for representing evals with Elixir modules and a helper, but the issue does not name files, tests, or more specific acceptance criteria.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
So, I don't really understand why you'd want YAML definitions for this, especially given the awkward assertion code.
Describe the solution you'd like
It would make more sense to me to use normal Elixir modules with perhaps a helper use. Like, why complicate things?
Describe alternatives you've considered
No response
Additional context
No response
- Dominant language
- Elixir
- Stars
- 51
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ash-project/evals
-
Feat: Add new eval type 'match_output' for stdout gradingPossibly taken @hoanle0126 claimed this 7 days ago. Openenhancement
Difficulty 3/5 1-2 days Newbie friendliness 55/100
ash-project/evals#3 · 1 reaction ·
-
Feat: Implement `:tagged_summary` report formatMay be free again @SubWal claimed this 30 days ago, and no pull request is open. Openenhancement
Difficulty 3/5 1-2 days Newbie friendliness 35/100
ash-project/evals#2 · 3 comments · 2 reactions ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 50/100
ash-project/evals#1 · 1 reaction ·
All issues in ash-project/evals
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
intellij-elixir/intellij-elixir#4645 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 69/100
ash-project/ash_typescript#114 ·
-
Unused require Logger warning in lib/absinthe/plug.ex (Elixir 1.20)Possibly taken @obutuz claimed this 1 day ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
whatyouhide/redix#358 · 2 reactions ·