JuliaParallel/Dagger.jl

`@spawn` has confusing move semantics

Open

#357 opened on Sep 9, 2022

View on GitHub
 (8 comments) (0 reactions) (1 assignee)Julia (706 stars) (86 forks)batch import
documentationeager apienhancementhelp wanted

Description

Note that

◖◗ Dagger.@spawn single=2 println(myid())
EagerThunk (running)

◖◗       From worker 2:        1
◖◗

◖◗ Dagger.spawn(() -> println(myid()), single=2)
EagerThunk (running)

◖◗       From worker 2:        2

I think what's happening here is that in the former case it is evaluating myid() on process 1 and the function executed on worker 2 is just println(1).

This seems like a problem, it's hard to guess that these examples would be different. I suggest that @spawn should match spawn so that its arguments can be read as if it were the function body passed to spawn. Otherwise I think it will need a lot of documentation.

A stronger argument might be that Distributed.@spawnat already works as I'm suggesting:

◖◗ @spawnat 2 println(myid());

◖◗       From worker 2:        2

(My suggestion is clearly badly breaking at least in principle, so this is tricky.)

Contributor guide

Tech stack
None
Domain
developer experience
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Basic JuliaUnderstanding of Dagger's macros
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
Investigate the macro expansion of `@spawn` in Dagger.jl. Compare with the `spawn` function and `@spawnat` from Distributed.jl. The goal is to modify `@spawn` so that its arguments are captured as the function body, similar to `spawn` and `@spawnat`. Reference the linked issue (https://github.com/JuliaParallel/Dagger.jl/issues/357) and any existing PRs. Note that there is an assignee; check if they have made progress.