Feature: Configuration to set maximum parallelization of :parallel runner
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
Start by reading the SSHKit implementations of the :parallel and :groups runners and how default_runner_config is applied. Compare the proposed limit and threads configurations, then verify that a bounded default runner works for deployment operations without requiring SCM-specific rate limiting.
Written by the indexing model from the issue text.
Description
There have been multiple requests to set an upper limit on the number of git operations that are executed in parallel in the default Capistrano git strategy. Similarly, users are also asking for a limit on the number of parallel bundle install executions in capistrano-bundler.
What these tasks have in common is that they all use the default :parallel runner provided by SSHKit. When using Capistrano to deploy to a large number of servers, firing off these operations to all servers in parallel can overload shared resources like a git server or private gem repository.
Rather than implement rate limiting for each SCM, capistrano-bundler, etc., I feel like a more general solution should be provided by SSHKit itself.
My proposal would be to change the implementation of the :parallel runner to essentially be a subclass of the :groups runner, except with defaults of wait: 0 and limit: INFINITY. Then, if a user wants to limit the amount of parallelization, they could simply do this:
# Limit the default :parallel runner to 10 threads
SSHKit.config.default_runner_config = { limit: 10 }
If sharing implementation and configuration keys between :parallel and :groups is too confusing, then perhaps the :parallel runner could use a different configuration key (but to the same effect):
# Limit the default :parallel runner to 10 threads
SSHKit.config.default_runner_config = { threads: 10 }
Thoughts?
See also:
- Dominant language
- Ruby
- Stars
- 1.2k
- Forks
- 257
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from capistrano/sshkit
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
capistrano/sshkit#563 · 6 reactions ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
capistrano/sshkit#562 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 48/100
capistrano/sshkit#554 · 3 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
capistrano/sshkit#543 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
capistrano/sshkit#518 · 1 comment ·
All issues in capistrano/sshkit
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
simp/pupmod-simp-simp#395 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
simp/pupmod-simp-rsyslog#219 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
simp/pupmod-simp-pupmod#256 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
simp/pupmod-simp-sudo#150 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100