[Feature]: ARCBackend — submit + status over ARC 7 REST, token auth
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 28/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- api, backend, distributed-systems
Research direction
Read DIRAC's AREXComputingElement.py and the existing contract interfaces first, then inspect the arc-slurm stack and its manifest. The issue does not name implementation or test files beyond config/proxy/ and the remote arc marker, so map those integration points before coding. Done means the listed token/proxy contract, bulk, state, retrieval, delegation, and registration checks pass.
Written by the indexing model from the issue text.
Description
User Story
As the WMS consumer,
I want an ARCBackend that submits and monitors jobs on an ARC 7 CE via REST with token auth,
So that the primary grid-CE path (DIRAC's AREX CE) exists on the new contract, validated against
the arc-slurm stack.
Feature Description
Essential half of the AREX port (IC-ADR-001 migration map):
ARCBackend: implementsSubmitter+Monitoragainst the ARC REST
API (/arex/rest/1.0) usingpyarcrest(evaluate fitness; fall back to httpx against the REST
spec if it can't do async/bulk cleanly — record the choice).- Job description generation from
SubmissionSpec(XRSL or ADL — pick one, document why;
DIRAC builds XRSL), incl. input-sandbox declaration and queue targeting. - Bulk semantics: ARC REST supports multi-job operations — submit
countcopies and poll many
ids per call; per-id failures ride the outcome maps. - State normalisation: ARC state map →
JobStatus(port DIRAC'sSTATES_MAP, incl. the
auto-kill-on-Holdquestion — surface as policy, don't hide it in the backend). - Token auth per IC-ADR-003: bearer header, audience exposure. Proxy/delegation
is issue-14. OutputRetriever(destructive=False): list the ARC session dir, stream every member into
destfile-by-file, return theJobOutputmanifest (.stdout/.stderr/.files/.log).
Bounded materialisation enforced inline: byte ceiling mid-stream, file-count ceiling,
per-transfer timeout, path containment (../absolute members rejected) — IC-ADR-001 §4.Cancellable.kill/Purgeable.purge(DIRACkillJob/cleanJob): bulk, per-id outcomes.LoadReporter.counts()from the CE info endpoint.SupportsLiveDiagnostics.diagnostics(): thediagnose/errorsendpoint — the one genuinely
independent, non-destructive pre-completion fetch.- Proxy/delegation auth: create delegation (CSR → sign with the
proxy chain → upload), renew,AlwaysIncludeProxyparity where tokens+proxy coexist. Add the
config/proxy/configuration axis to thearc-slurmstack (ADR-002 §7) to test it. - Fetch-twice regression test proving re-fetchability (
destructive=False) — the counterpart
of HTCondor's one-shot test. - Phase 2 wiring: contract suite (submit/status subset) against the
arc-slurmstack
(markersremote and arc).
Definition of Done
- Full contract suite (all capabilities) green against
arc-slurm, both token and proxy
configs -
count > 1submission bundles oneSubmissionwith per-copy failure reporting - Unknown/expired ids →
JobStatus.UNKNOWN(never dropped) - State map unit-tested; Hold handling decided and documented
- Fetch-twice test green; containment/ceiling tests green (hostile filenames, oversized
output) - Delegation renewal covered by a test (short-lived test-CA proxy)
-
arc-slurmstack gainsconfig/proxy/; manifest updated - Registered via
intercede.backendsasarc
Alternatives Considered
- Wrapping the
arcCLI tools — spawns processes, brittle parsing; REST is first-class in ARC 7
and DIRAC's AREX already proves it.
Related Issues
Additional Context
ARC floor is 7.x (IC-ADR-002 §8). DIRAC reference: `AREXComputingElement.py
- Dominant language
- Python
- Stars
- 3
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DIRACGrid/intercede
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
All issues in DIRACGrid/intercede
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
PedestrianDynamics/pyFDS-Evac#343 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
theskumar/python-dotenv#708 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 2 days
-
Docs Timedelta
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
pandas-dev/pandas#69919 ·
Maintainers usually reply within 1 day
-
API documentation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
zephyrproject-rtos/west#1009 · 2 comments ·
Maintainers usually reply within 3 days