Initializing the remote caches as needed
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Refactor
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- build-system
Research direction
Start by tracing the checkout startup sequence and the point where remote caches are initialized. Compare the listed loading, resolving, cache-query, fetch, and checkout steps; done means remote caches are initialized only when a missing or incomplete artifact must be fetched, not before target validation or local-cache use.
Written by the indexing model from the issue text.
Description
Currently, remote caches are initialized pretty early in the buildstream start sequence. At least for checkout operations, this is often too early to know whether the caches are actually needed.
There are a few situations:
- target directory is not empty: Loading elements -> Resolving elements -> Initializing remote caches -> target directory is not empty
- this wastes time before reporting the error
- artifacts are cached locally: Loading elements -> Resolving elements -> Initializing remote caches -> Query cache -> Staging dependencies -> Integrating sandbox -> Checking out files
- We don't need remote caches here, so it also wastes time
- artifacts aren't cached: Loading elements -> Resolving elements -> Initializing remote caches -> Query cache -> Attempting to fetch missing or incomplete artifact(s) ...
- This is the case that needs initializing remote caches. However it makes more sense to do it after the "Attempting to fetch", not before.
- Dominant language
- Python
- Stars
- 144
- Forks
- 45
- Avg merge
- 20h 38m
- Merged PRs (30d)
- 6
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/buildstream
-
logging
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/buildstream#2150 · 3 comments ·
-
question
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
apache/buildstream#2143 · 2 comments · 1 reaction ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
apache/buildstream#2194 ·
-
Modification time of an element with an open workspace propagates into consumer element's sandbox Openbug
Difficulty 3/5 1-2 days Newbie friendliness 58/100
apache/buildstream#2186 · 2 comments · 2 reactions ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
apache/buildstream#2184 · 1 comment ·
All issues in apache/buildstream
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
syfoud/Simulated_Scepter#172 ·
-
A cancelled tests run makes the coverage comment workflow fail and reports it as a red check on main Openarea: ci bug perceived difficulty: 3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Nitjsefnie-Harness-Commons/daedalus#921 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
EleutherAI/lm-evaluation-harness#4207 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ClickHouse/clickhouse-connect#1057 ·