[kbn-evals] Examples should not ship in the installed wheel
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- python
- Domain
- build-system, cli, documentation
Research direction
Start with pyproject.toml lines 37-47 to inspect the wheel package list and elastic_evals.suites entry points. Verify a clean pip or uv install contains no examples, confirm elastic-evals list shows no pre-registered suites, and document the plugin entry-point contract with a short example.
Written by the indexing model from the issue text.
Description
Summary
pyproject.toml includes examples/ in the wheel and pre-registers all three demo suites as entry-point plugins. Every pip install elastic-evals ships demo code to the user and shows three suites they didn't create in elastic-evals list.
Problem
pyproject.toml:46-47:exampleslisted in wheel packagespyproject.toml:37-40: three example suites registered aselastic_evals.suitesentry pointsexamplesas a top-level module name can collide with other packages- The three entry-point IDs use inconsistent naming patterns within kebab-case (
agent-builder,claude-code-eval,chatbot-rag-eval-example)
Fix
- Remove
examplesfrom the wheel packages list - Remove example entry-point registrations
- Document the plugin entry-point contract so users can register their own suites
Done when
-
piporuvinstalls no example code -
elastic-evals liston a clean install shows no pre-registered suites - The plugin contract is documented with a short example
- Dominant language
- Python
- Stars
- 2
- Forks
- 0
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 17
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from elastic/elastic-evals-sdk-python
-
kbn-evals Team:nightshift-context-and-research
Difficulty 4/5 3-5 days Newbie friendliness 64/100
-
kbn-evals Team:nightshift-context-and-research
Difficulty 3/5 1-2 days Newbie friendliness 78/100
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
kbn-evals Team:nightshift-context-and-research
Difficulty 4/5 3-5 days Newbie friendliness 50/100
-
kbn-evals Team:nightshift-context-and-research
Difficulty 5/5 Over a week Newbie friendliness 32/100
All issues in elastic/elastic-evals-sdk-python
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100