[Feature] Allow users to specify custom spawners

Open
#942 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
distributed-systems, hpc

Research direction

Start by reading src/executorlib/standalone/interactive/spawner.py and the existing subprocess, mpiexec, srun, and flux run interfaces. Determine how a custom spawner should be supplied and what its contract must be; done means users can provide custom process-launch commands alongside the existing interfaces.

Written by the indexing model from the issue text.

Description

enhancement

At the moment the spawner module provides four different interfaces to start a new process, using a subprocess, mpiexec, srun or flux run. But different clusters might have different commands to trigger MPI parallel runs or in principle it would also be possible to add additional computing resources from a cloud provider to speed up an application running on your laptop. Either way it would be great to add the option to provide custom spawners.

Dominant language
Python
Stars
77
Forks
7
Avg merge
10h 32m
Merged PRs (30d)
12

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pyiron/executorlib

All issues in pyiron/executorlib

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.