retrieve future expression on local machine
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- r
- Domain
- distributed-systems, hpc
Research direction
Review the future.batchtools behavior for the shown plan using login, batchtools_lsf, and multiprocess. Start with the documented lifecycle of a submitted future and determine whether results can be retrieved after the local R session closes or from the remote machine; document the supported behavior and any required persistence mechanism.
Written by the indexing model from the issue text.
Description
Hi Henrik,
maybe that's utterly basic and the wrong place to ask, but I thought it might be of interest for more people:
Is there a way you can retrieve a future expressions in the following setup:
login <- tweak(remote, workers = "remote@remote")
bsub <- tweak(batchtools_lsf, ...)
plan(list(
login,
bsub,
multiprocess
))
and I submit the following job (just an example):
y %<-% furrr::future_map_dfr(1:5, function(x){
cores <- availableCores()
furrr::future_map_dfr(1:5, function(x){
tibble::tibble(x = x,
nodename = Sys.info()['nodename'],
Sys.getpid = Sys.getpid(),
availableCores = cores)
})
})
... let's imagine this takes an considerable amount of time and I would close my local r session in between submitting the job and its being finished.
After a login then (and rerunning the same plan again) I can't access y, neither can I access y on the remote machine in an R session. Is that possible in general?
If not, we wondered if you can save R objects somehow in an future_apply, submitted from a local machine and evaluated an a remote one?
Sorry for the very basic questions, just started to fiddle around with future.batchtools.
Cheers
Marco
- Dominant language
- R
- Stars
- 87
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from futureverse/future.batchtools
-
feature/resources scheduler/lsf
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
futureverse/future.batchtools#105 ·
-
feature/resources scheduler/sge
Difficulty 4/5 3-5 days Newbie friendliness 50/100
futureverse/future.batchtools#104 · 1 comment ·
-
feature/resources scheduler/slurm
Difficulty 3/5 1-2 days Newbie friendliness 66/100
futureverse/future.batchtools#103 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
futureverse/future.batchtools#102 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 15/100
All issues in futureverse/future.batchtools
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
robjhyndman/forecast#1220 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
JamesHWade/deputy#192 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
bug triage_needed
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
pharmaverse/rtables#1123 · 1 comment · 1 reaction ·