Revisit how we determine files-changed
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 25/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Da chiarire
- Stato di attività
- Ferma
- Stack tecnologico
- git, github, python
- Ambito
- build-system, ci-cd, devops
Direzione di ricerca
Start in src/taskgraph/decision.py around the linked logic that determines the base revision and files-changed, then review how shallow clones affect git merge-base. Compare the proposed diff-based and GitHub API approaches for force pushes, new branches, and pull requests; done means choosing and implementing a solution with defined files-changed behavior for these cases.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Problem
Taskgraph currently does some extra logic to figure out a proper base revision that we then use to compute the files changed by the graph:
https://github.com/taskcluster/taskgraph/blob/99840c4b0405ebeb7777b466df779b79b5612078/src/taskgraph/decision.py#L202
This logic is solving some deficiencies with Github events:
- For force pushes, the
event.beforeis the newly orphaned head rather than the actual base revision. - For pushes to new branches, the
event.beforeis the null revision - For pull requests, the
pull_request.base.shaproperty contains the revision that the base reference currently points to, not the revision that the PR is based on.
So instead, we essentially use git merge-base <default branch> <head rev> to find the ancestor commit of both, and then set that to the base revision, which then gets used to compute files-changed.
The problem is that now we're starting to use shallow clones, which means that merge-base doesn't work without progressively deepening the repository until we find the ancestor (negating the benefit of shallow clones).
So a new solution is needed!
Possible Solutions
Simply stop using git merge-base
In this solution, we would simply run git diff <base> <head> to get the files modified between the two trees. Here we would be consciously making a tradeoff of sometimes having inaccurate files-changed for simplicity and clone performance. The above scenarios would roughly shake out like:
files-changedwould include files that were modified by orphaned commits under<base>, even if<head>didn't touch them. This would cause us to run more tasks than necessary.- We'd probably have to special case this, maybe only looking at files touched by
<head>. - Pull requests would have files-changed that are derived from all commits on the base branch that the PR hasn't rebased on top of yet, potentially running many more tasks than expected
Use the Github API
It should be possible to get the files modified from a PR or force push using the Github API. This would be fast and simple to implement, though we'd need to start worrying about rate limits, tokens and it isn't portable if we ever want to support non-Github repos.
In this case, we'd likely still want a fallback to the merge-base solution for non-Github repos or when hitting rate limits.
- Lingua principale
- Python
- Stelle
- 26
- Fork
- 55
- Merge medio
- 2g 6m
- PR unite (30g)
- 15
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di taskcluster/taskgraph
-
Dependency Dashboard Aperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 20/100
taskcluster/taskgraph#952 ·
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
taskcluster/taskgraph#915 ·
-
feature
Difficoltà 3/5 1-2 giorni Idoneità per principianti 35/100
taskcluster/taskgraph#911 · 1 commento ·
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 45/100
taskcluster/taskgraph#882 ·
-
Further optimize Git clones Aperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
taskcluster/taskgraph#865 · 1 commento ·
Tutte le issue di taskcluster/taskgraph
Issue simili
-
enhancement
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
canonical/paas-charm#368 · 1 commento ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
-
tech debt
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
-
addition to tracking list Aperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
StevenBlack/hosts#3256 ·
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
qualcomm/qai-appbuilder#275 ·