[CI/CD] Introduce testing for the clients and agents
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python
- Domain
- ci-cd, testing-qa
Research direction
Start by reading the CI pipeline introduced in #19 and the client and agent code that the issue asks to cover. Consider the API-cost constraint while examining how tests could protect the benchmark during the refactoring planned in #146 and future support for #49. Done means a defined test approach for all clients and agents that runs in CI without API costs and catches regressions.
Written by the indexing model from the issue text.
Description
We need a way to establish tests for all of the clients and agents within AIOpsLab. We also need a way to include these tests with the CI pipeline running that was introduced in #19
How do we test the clients and agents without introducing API costs per run is the BIG question. We primarily need these tests to make sure that, when we are refactoring the code as we planned to in #146 and to support #49 in the future, we don't introduce regressions into the benchmark and break functionality for everyone.
- Dominant language
- Python
- Stars
- 989
- Forks
- 176
- Avg merge
- 7d 15h
- Merged PRs (30d)
- 3
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/AIOpsLab
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
All issues in microsoft/AIOpsLab
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
PedestrianDynamics/pyFDS-Evac#199 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
521xueweihan/HelloGitHub#3790 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
sandialabs/atlas-ui-3#978 ·
Maintainers usually reply within 1 day
-
area: tests perceived difficulty: 2
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Nitjsefnie-Harness-Commons/daedalus#1255 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
EleutherAI/lm-evaluation-harness#4256 ·
Maintainers usually reply within 1 day