[Feature]: HTCondor scheduler (shared `_htcondor` core) for BatchBackend

Open
#21 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
python
Domain
backend

Research direction

Start with the HTCondorScheduler and the shared _htcondor module described in the issue, then read IC-ADR-001 §3 and the existing DIRAC STATES_MAP/HOLD_REASON_SUBCODE test fixtures. Run the contract suite marked scheduler and htcondor against a plain schedd. Done means queue N fan-out, native output transfer, state parsing, and an importable _htcondor core without scheduler-class dependencies.

Written by the indexing model from the issue text.

Description

User Story

As the WMS consumer,
I want BatchBackend(SSHTransport|LocalTransport, HTCondorScheduler) working against a real
schedd,
So that the second scheduler proves the Scheduler protocol generalises, and the _htcondor core
is ready for reuse by the HTCondor-CE backend.

Feature Description

  • _htcondor internal module (Tier C, shared with #): submit-description builder
    from SubmissionSpec (incl. queue N for count fan-out, should_transfer_files/
    transfer_output_files from OutputSpec), condor_q/condor_history ClassAd/JSON parsing,
    native state map incl. hold codes/subcodes (port DIRAC's STATES_MAP/HOLD_REASON_SUBCODE).
  • HTCondorScheduler: drives a schedd via the transport (CLI condor_submit/condor_q/
    condor_rm over Transport.run); stages_own_files=True — HTCondor declares file transfer in
    the submit description (the cross-scheduler staging axis from IC-ADR-001 §3).
  • Phase 2 wiring: contract suite against the htcondor stack's plain-schedd path
    (markers scheduler and htcondor); ssh-condor stack optional here, may land later.

Definition of Done

  • Contract suite green against the htcondor stack (plain schedd)
  • count > 1 submits as one queue N (asserted)
  • Output sandbox via native file transfer, honouring OutputSpec at submit time
  • State mapping unit-tested against canned condor_q/condor_history output (port DIRAC's
    test fixtures)
  • _htcondor has no scheduler-class dependencies (importable by the CE backend, #)

Alternatives Considered

  • Python htcondor bindings instead of CLI-over-transport — bindings can't run over SSH to a
    remote schedd; CLI-over-transport keeps the transport axis orthogonal. Bindings remain an option
    for the monolithic CE backend (issue-15), where the schedd is reached natively.

Related Issues

Additional Context

IC-ADR-001 §3 "Shared batch-system knowledge": scheduler adapter and monolithic CE share the
_htcondor functions, never a subtype.

Dominant language
Python
Stars
3
Forks
3
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from DIRACGrid/intercede

All issues in DIRACGrid/intercede

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.