alteryx/evalml

Extract Parallel Tests into Separate Github CI Action

Open

#3,647 opened on Aug 8, 2022

View on GitHubContributor guide
 (2 comments) (0 reactions) (0 assignees)Python (93 forks)auto 404
good first issuetesting

Repository metrics

Stars
 (852 stars)
PR merge metrics
 (PR metrics pending)

Description

As an EvalML dev, I would like parallel CI tests to be put into a separate Github Action CI check so that development can carry on if they prove flaky.

Parallel tests with DaskEngine and CFEngine have proven less than resilient to dependency changes. Until the specific cause of this flakiness can be determined, the tests for parallel behavior should be extracted and put into a separate CI check that is allowed to flake and fail periodically.

Here are the locations of "git-test-parallel"

Contributor guide

Research direction
Identify the existing parallel test configuration in the repository (e.g., in Makefile or CI configuration) and extract it into a separate GitHub Actions workflow file that can run independently and be allowed to fail without blocking the main CI.
Tech stack
python
Domain
devopsci cd
Issue type
Chore
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
GitHub Actions
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60