Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Optuna grid-search tests doubled PR CI time since #1252 (~23 → ~47 min); cap their search budget in tests

Closed Beginner friendly
  • Is something wrong?
#1,321 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Refactor
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
machine-learning, testing

Research direction

Locate the eight test_eval_grid_search_multimetric_scorer[*-True] cases and test_eval_grid_search_optuna, then read how they configure OptunaSearchCV and its evaluation call. Pass time_out=1 to those evaluations and add a whats_new.rst entry under Code health. Run the affected tests and confirm they still cover the Optuna path and primary-metric selection.

Written by the indexing model from the issue text.

Description

Problem. Since 2026-10-14 the Test workflow takes ~47 min per PR, up from ~23 min. Most of the increase is the Optuna-backed evaluation tests added in #1252: each OptunaSearchCV uses the default 10 trials over a 2-point grid (csp__metric: euclid/riemann), so the 8 test_eval_grid_search_multimetric_scorer[*-True] cases and test_eval_grid_search_optuna take ~110 s each. On the ubuntu py3.12 leg that is ~1,200 of ~2,150 s of pytest (job 117553166134, run 39086137050), against ~230 s on 2026-10-09 (job 113726805695).

Proposed change (test-only, ~4 lines). Pass time_out=1 to the evaluation in those Optuna test cases, so Optuna stops after the trial in flight. The tests still exercise the Optuna path and the primary-metric selection; only the search budget changes. No library behaviour changes. Plus a whats_new.rst line under Code health.

Measured on a fork (agarg21/moabb, PR-triggered Test workflow on develop, 3 runs per arm, all green, identical 5,269 passed / 1,255 skipped):

  • PR wall median 46.9 → 23.2 min (−50%)
  • ubuntu py3.12 job 40.8 → 23.2 min, windows py3.12 46.8 → 17.4 min, macOS py3.12 34.2 → 22.2 min
  • runner time ~197 → ~114 job-min per run (−42%)
  • Runs before: 39220244267, 39231604955, 39264518650; after: 39226327039, 39237394932, 39281666313

Happy to open the PR against develop if this approach sounds good.

Prepared with AI assistance (Claude Code); measured on a fork and reviewed before posting.

Dominant language
Python
Stars
1.1k
Forks
269
Avg merge
1d 4h
Merged PRs (30d)
124

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NeuroTechX/moabb

All issues in NeuroTechX/moabb

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.