[Feature]: HTCondorCEBackend — grid CE over the shared `_htcondor` core
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- backend, distributed-systems, testing-qa
Research direction
Start by reading the shared _htcondor core and the contract suite for the htcondor stack, then run the remote and htcondor tests. Focus on the fetch-twice destructive_fetch regression, capability narrowing, and kill-path coverage. Done means the CE path is green with IDTOKENS, records its leave_in_queue decision, and exercises partial-batch outcomes.
Written by the indexing model from the issue text.
Description
User Story
As the WMS consumer,
I want an HTCondorCEBackend submitting to an HTCondor-CE with IDTOKENS and spooled file
transfer,
So that the second major grid-CE path exists on the new contract, including the destructive-fetch
semantics that motivated IC-ADR-002.
Feature Description
HTCondorCEBackend:JobBackend+OutputRetriever+Cancellable.
Deliberately noLoadReporter(DIRAC'sgetCEStatusis an explicit "not supported") and no
live diagnostics — capability absence is the point, and the integration suite must show the
narrowing works.- Reuses the
_htcondorcore (submit description, ClassAd parsing, state
map); differs in transport/auth: remote schedd (-pool/-remote) or CE submission, grid
universe routing,-spool. - Token + proxy together per IC-ADR-003 (its Motivation 7 / all-of
kinds): SCITOKENS/IDTOKENS
channel auth (token file +_CONDOR_*env injection, audience<ce>:9619) plus the X.509
proxy materialised into the submit description (use_x509userproxy = true) — consumed
site-side (VOMS attributes → APEL accounting), including the
_CONDOR_DELEGATE_JOB_GSI_CREDENTIALS=falseworkaround (HTCONDOR-2904). The pure-IDTOKENS
basicstack config exercises the token path; a proxy-bearing config is the ADR-002 auth axis. - Destructive retrieval:
condor_transfer_datathen declareddestructive=True
(HTCondor ≥ 25.8: completed spooled job leaves the queue); evaluateleave_in_queueas the
re-fetchability override and record the decision. Coarse bounded materialisation (submit-time
transfer_output_filesallow-list, transfer timeout, post-hoc size/count check). - Phase 2 wiring: contract suite against the
htcondorstack's CE path (markers
remote and htcondor); the fetch-twicedestructive_fetchtest is the flagship regression
(versions axis[lts, latest]makes 25.8-class changes red on the Renovate PR).
Definition of Done
- Contract suite green against the
htcondorstack CE path with IDTOKENS - Fetch-twice test asserts the declared one-shot semantics;
leave_in_queuedecision recorded -
isinstance(be, LoadReporter)is False and the suite's capability-skip path exercises it - Kill path green; per-id outcomes on partial batches
Alternatives Considered
- Expressing it as
BatchBackend(HTCondorCETransport, HTCondorScheduler)— open ADR question;
blocked on a transport-level "destructive get" notion. Start monolithic, revisit (share only
_htcondorfunctions, never a subtype). - GSI/VOMS auth — out of scope; IDTOKENS-first per ADR-002 §4.
Related Issues
Blocked by: # (_htcondor), #. Validated by: #5 / htcondor stack.
Additional Context
- Dominant language
- Python
- Stars
- 3
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DIRACGrid/intercede
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Difficulty 5/5 Over a week Newbie friendliness 28/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
All issues in DIRACGrid/intercede
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100