Detect flaky integration tests

Open
  • Is something wrong?
#163 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
66/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
docker, github-actions, python
Domain
ci-cd, testing

Research direction

Start with .github/workflows/ and tests/integration/, then inspect how make integration runs and how the repository creates GitHub issues from automation. Add a scheduled workflow that repeats the integration run and reports tests whose results vary. Done means the schedule runs successfully and automatically opens an issue listing detected flaky tests.

Written by the indexing model from the issue text.

Description

area: ci size: S tests
Context

Docker-backed tests can be timing-sensitive. A test that fails one run in twenty erodes trust in CI.

What to do

Add a scheduled workflow that runs make integration several times and reports any test whose result changed between runs.

Done when

Flaky tests are found and listed in an issue automatically.

Where to look

.github/workflows/, tests/integration/

New here? Read CONTRIBUTING.md and comment on this issue to say you are taking it.

Generated by Claude Code

Dominant language
Python
Stars
0
Forks
0
Avg merge
1h 54m
Merged PRs (30d)
14

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from patsypppe/meridian

All issues in patsypppe/meridian

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.