JuliaParallel/Dagger.jl

`@spawn` has confusing move semantics

Open

#357 opened on Sep 9, 2022

View on GitHub
 (8 comments) (0 reactions) (1 assignee)Julia (86 forks)batch import
documentationeager apienhancementhelp wanted

Repository metrics

Stars
 (706 stars)
PR merge metrics
 (Avg merge 12d 5h) (4 merged PRs in 30d)

Description

Note that

◖◗ Dagger.@spawn single=2 println(myid())
EagerThunk (running)

◖◗       From worker 2:        1
◖◗

◖◗ Dagger.spawn(() -> println(myid()), single=2)
EagerThunk (running)

◖◗       From worker 2:        2

I think what's happening here is that in the former case it is evaluating myid() on process 1 and the function executed on worker 2 is just println(1).

This seems like a problem, it's hard to guess that these examples would be different. I suggest that @spawn should match spawn so that its arguments can be read as if it were the function body passed to spawn. Otherwise I think it will need a lot of documentation.

A stronger argument might be that Distributed.@spawnat already works as I'm suggesting:

◖◗ @spawnat 2 println(myid());

◖◗       From worker 2:        2

(My suggestion is clearly badly breaking at least in principle, so this is tricky.)

Contributor guide

Research direction
Compare the macro expansions of `@spawn` and `spawn` calls, trace how `myid()` is evaluated in each case, and modify the macro to capture the expression for remote evaluation instead of evaluating it locally, matching the behavior of `Distributed.@spawnat`.
Tech stack
None
Domain
backend
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Julia
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60