[Documentation] Explain the differences between the TestClusterExecutor and the SlurmClusterExecutor and the FluxClusterExecutor
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 50/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- documentation, hpc
Research direction
Start by reading the linked spawner_subprocess.py and spawner_pysqa.py sections to verify how dependency handling differs between TestClusterExecutor and the scheduler-backed executors. Add documentation that clearly explains when dependent tasks are waited on versus delegated to the scheduler, and ensure it covers TestClusterExecutor, SlurmClusterExecutor, and FluxClusterExecutor.
Written by the indexing model from the issue text.
Description
For the TestClusterExecutor this is the case, because the subprocess interface does not provide any dependency management. It waits until all dependent tasks are completed before the next step is submitted:
https://github.com/pyiron/executorlib/blob/main/src/executorlib/task_scheduler/file/spawner_subprocess.py#L49
In contrast to this the FluxClusterExecutor and the SlurmClusterExecutor use the job scheduler for dependency management, so for those there is no need to wait:
https://github.com/pyiron/executorlib/blob/main/src/executorlib/task_scheduler/file/spawner_pysqa.py#L70
- Dominant language
- Python
- Stars
- 77
- Forks
- 7
- Avg merge
- 10h 32m
- Merged PRs (30d)
- 12
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from pyiron/executorlib
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
pyiron/executorlib#1054 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
pyiron/executorlib#1032 ·
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
pyiron/executorlib#1005 ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 35/100
pyiron/executorlib#1049 · 1 comment ·
-
documentation
Difficulty 3/5 1-2 days Newbie friendliness 35/100
pyiron/executorlib#999 · 1 comment ·
All issues in pyiron/executorlib
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
anthropics/skills#1811 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
speaches-ai/speaches#678 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
datalayer/mcp-compose#42 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
conda-forge/spacy-feedstock#177 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
UKGovernmentBEIS/inspect_evals#2523 ·