feature: Support make jobserver protocol
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- build-system
Research direction
Start with the GNU Make Job-Slots specification and the linked Ninja client PR, then inspect BuildStream's handling of --max-jobs and build sandboxes. Done means implementing the jobserver server, mounting its FIFO or pipe into sandboxes, and setting MAKEFLAGS so nested builds share the job limit.
Written by the indexing model from the issue text.
Description
Make has already for some time used jobserver protocol such that when a sub-make is run, top-make decides whether more processes can be spawned. This is a requirement for -jN option to work with nested makes. Recently, there's a PR to add jobserver client support to ninja as well. I propose buildstream to implement the actual jobserver part (at least fifo, can use also pipe) and mounts relevant fifo into build sandbox. There is also the part of setting MAKEFLAGS and declaring where the job server is. When this setup is fully in place and user passes --max-jobs to BuildStream, it can actually take effect over all sandboxes allowing high -jN option to safely be used without worrying about overloading the system.
The specification is in https://www.gnu.org/software/make/manual/html_node/Job-Slots.html and client implementation PR for ninja is in https://github.com/ninja-build/ninja/pull/2506. The PR also contains Apache-licensed Python code that implements the server that can be used as inspiration.
- Dominant language
- Python
- Stars
- 144
- Forks
- 45
- Avg merge
- 20h 38m
- Merged PRs (30d)
- 6
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/buildstream
-
logging
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/buildstream#2150 · 3 comments ·
-
question
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
apache/buildstream#2143 · 2 comments · 1 reaction ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
apache/buildstream#2194 ·
-
Modification time of an element with an open workspace propagates into consumer element's sandbox Openbug
Difficulty 3/5 1-2 days Newbie friendliness 58/100
apache/buildstream#2186 · 2 comments · 2 reactions ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
apache/buildstream#2184 · 1 comment ·
All issues in apache/buildstream
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
anthropics/skills#1811 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
speaches-ai/speaches#678 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
datalayer/mcp-compose#42 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
conda-forge/spacy-feedstock#177 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
UKGovernmentBEIS/inspect_evals#2523 ·