JuliaDynamics/DrWatson.jl
Ver no GitHubUsage together with Distributed can cause precompile issues
Open
#270 aberto em 23 de jul. de 2021
help wantedproject-setup
Métricas do repositório
- Stars
- (915 stars)
- Métricas de merge de PR
- (Nenhuma PRs mesclada em 30d)
Description
As mentioned already in https://github.com/JuliaDynamics/DrWatson.jl/issues/269, using DrWatson together with the Distributed package can cause errors where many remote worker processes try to precompile packages at the same time. This either causes the machine to run out of memory or raise issues with the compilecache.
MWE for the error (now using a pure Julia package):
using Distributed
addprocs(8)
@everywhere using DrWatson
@everywhere @quickactivate "TestEnv"
@everywhere begin
using Distributions
end
Slightly altering the order of imports solves the issue. This works:
using Distributed
addprocs(8)
@everywhere using DrWatson
@everywhere begin
@quickactivate "TestEnv"
using Distributions
end
The issue seems indeed to be related to DrWatson, because running
using Distributed
addprocs(8)
@everywhere import Pkg
@everywhere Pkg.activate(".")
@everywhere begin
using Distributions
end
also poses no problem.
I am really interested to know the reasons for this.
EDIT: Last example corrected to better match the issue case.