batchtools_slurm futures block (Was: stdout/stderr to log file only when using slurm?)
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Start by reproducing the shown future() loop with future.batchtools on a Slurm cluster and investigate the mentioned split option for output handling. Done means jobs can be submitted without the sending session waiting for captured output, while warnings are written only to the Slurm log files.
Written by the indexing model from the issue text.
Description
I have a loop like this:
for (i in 14:11) {
f <- future({
warning("Sleeping ", i, " seconds")
Sys.sleep(i)
saveRDS(i^2, paste0("tmp-data/res", i))
})
}
and I would to output the warning in the log files only.
I've seen the new option split in a blog post to do both (but I didn't find it in the documentation).
The problem is that capturing this output and returning it to the main session usually blocks my session for a long time (sometimes minutes), and I would like to avoid this.
Basically, is there any way to use future/future.batchtools just to submit jobs on a slurm cluster without returning anything to the session sending them?
- Dominant language
- R
- Stars
- 87
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from futureverse/future.batchtools
-
feature/resources scheduler/lsf
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
futureverse/future.batchtools#105 ·
-
feature/resources scheduler/sge
Difficulty 4/5 3-5 days Newbie friendliness 50/100
futureverse/future.batchtools#104 · 1 comment ·
-
feature/resources scheduler/slurm
Difficulty 3/5 1-2 days Newbie friendliness 66/100
futureverse/future.batchtools#103 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
futureverse/future.batchtools#102 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 15/100
All issues in futureverse/future.batchtools
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
briandconnelly/airnow#9 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
OHDSI/CohortConstructor#774 ·
-
pre-review R TeX Track: 5 (DSAIS)
Difficulty 1/5 Under an hour Newbie friendliness 60/100
openjournals/joss-reviews#11330 · 7 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100