haskell-nix/hnix

Allow concurrent evaluation

Aperta

#170 aperta il 12 apr 2018

 (1 commento) (0 reazioni) (0 assegnatari)Nix (116 fork)github user discovery
challengingenhancementhelp wanted

Metriche repository

Star
 (835 stelle)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

Right now parsing is our slowest task, and yet parsing occurs during evaluation all the time due to import expressions. It would be nice if we could assign a pool of N workers (using the async-pool library), and perform these slow evaluations concurrently since none of them can have interfering side-effects. This should speed up the evaluation of nixpkgs considerably if there are many capabilities available.

To do this will need a few steps:

  • Make Thunk.hs thread-safe by introducing a variable that causes other threads to wait for pending evaluations to complete. Right now there is a boolean flag in an IORef; it might be sufficient to just turn this into a trinary flag in a TVar.

  • Create MonadConcurrent to abstract when concurrent evaluation is possible. It wouldn't be for the symbolic evaluator, for example, but would for an evaluator that runs in IO, such as Lazy.

  • Replace the calls to traverse that happen in Exec.hs for NList and NSet to instead use a new concurrentTraverse method defined in MonadConcurrent, which for IO would be implemented by calling Control.Concurrent.Async.Pool.mapTasks <group> and for pure evaluation would just call traverse as before.

Pinging @ryantrinkle, who requested this feature.

Guida contributor