[Feature]: Scheduler protocol + Slurm scheduler + generic BatchBackend
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- backend-api-design, ci-cd, testing
Research direction
Start with IC-ADR-001 §3 and the Scheduler protocol requirements, then trace the planned _slurm module, Slurm registration via intercede.schedulers, and the generic BatchBackend interfaces. Run the integration contract suite against the ssh-slurm and local-slurm stacks first. Done means both stacks pass submit, status, fetch, kill, fetch-twice, and partial-failure checks, with array submission and the listed output and load-reporting behavior verified.
Written by the indexing model from the issue text.
Description
User Story
As the WMS consumer,
I want BatchBackend(SSHTransport, Slurm) and BatchBackend(LocalTransport, Slurm) working
end-to-end against the containerised Slurm stacks,
So that the N+M composition promise of IC-ADR-001 §3 is proven on the first real scheduler.
Feature Description
Schedulerprotocol finalised:submit_cmd(spec),parse_status(raw),kill_cmd(ids),
stages_own_filesflag (+stage_inputs/collect_outputshooks where needed). Evaluate the
ADR's open question — promote staging to aStagercollaborator — and record the outcome._slurminternal module (Tier C): sbatch script/option generation fromSubmissionSpec
(incl.count→--arrayfor identical fan-out),squeue/sacctparsing, native-state →
JobStatusmap (port DIRAC's mapping: PENDING/SUSPENDED/CONFIGURING→waiting, COMPLETED→done,
CANCELLED/PREEMPTED→aborted…).Slurmscheduler consuming_slurm; registered viaintercede.schedulers.- Generic
BatchBackend(Tier B): satisfiesJobBackend+OutputRetriever
(destructive=False) +Cancellable+Purgeable+LoadReporter; sandbox staging via
Transport.put/getwhenstages_own_filesis false; workdir layout per job;JobHandle
carries host/workdir routing (replaces DIRAC'sssh<batch>://ref encoding). - Phase 2 wiring: the integration contract suite (submit → status → fetch → kill, fetch-twice,
partial-failure maps) runs againstssh-slurmandlocal-slurmstacks via markers.
Definition of Done
- Contract suite green on
ssh-slurmandlocal-slurmstacks in CI -
count > 1submits via a single--array(asserted, not N submits) -
fetch_outputstreams an arbitrary output sandbox (not just stdout/stderr) with bounded
materialisation (size/count/timeout/path-containment) -
LoadReporter.counts()fromsqueue; unknown ids →JobStatus.UNKNOWN
Alternatives Considered
- Per-combination classes (
SSHSlurmBackend,LocalSlurmBackend…) — N×M explosion; rejected. - Porting DIRAC's remote-executed
SLURM.pydriver — requires remote python; dropped (see
issue-10).
Additional Context
This issue is the template for every further scheduler.
- Dominant language
- Python
- Stars
- 3
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DIRACGrid/intercede
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 28/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
All issues in DIRACGrid/intercede
Similar issues
-
triage/confirmed
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
agentscope-ai/agentscope#2775 ·
-
comp/desktop P3 type/bug
Difficulty 1/5 Under an hour Newbie friendliness 92/100
NousResearch/hermes-agent#118866 ·
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 90/100
apache/cloudstack#14222 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100