GitRepository `.spec.ref.commit` + `.spec.ref.branch` does not shallow clone, contrary to the docs
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 42/100
Direzione di ricerca
Inizia da pkg/git/gogit/clone.go, in particolare cloneCommit, e confrontalo con cloneBranch, cloneTag e cloneSemVer; poi esamina gitrepository_controller.go e docs/spec/v1/gitrepositories.md. Conferma il comportamento attuale e i test pertinenti prima di decidere se l'ambito si limita alla documentazione o include il cloning a profondità limitata e il commit fast path. Il lavoro è concluso quando il comportamento selezionato è implementato e documentato accuratamente, con copertura di regressione per il percorso interessato.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
The v1 GitRepository docs state that combining .spec.ref.commit with .spec.ref.branch performs a shallow clone. It does not — the commit clone path never sets a depth. The combination gives you --single-branch, not --depth.
Either the docs should be corrected, or a real bounded-depth mechanism should be added. I'd argue for both, plus a third fix that is probably the biggest win (see "Skipped fast path" below).
What the docs say
docs/spec/v1/gitrepositories.md, under Commit example:
This field takes precedence over all other fields. It can be combined with
.spec.ref.branchto perform a shallow clone of the branch, in which the commit must exist:spec: ref: branch: <branch> commit: "<commit SHA within branch>"
The wording appears to date back to the v1beta2 API field comment from the gitImplementation: go-git | libgit2 era (quoted in #1017):
// Commit SHA to check out, takes precedence over all reference fields.
//
// When GitRepositorySpec.GitImplementation is set to 'go-git', this can be
// combined with Branch to shallow clone the branch, in which the commit is
// expected to exist.
What the code does
In fluxcd/pkg/git/gogit, cloneBranch, cloneTag and cloneSemVer all honour opts.ShallowClone:
var depth int
if opts.ShallowClone {
depth = 1
}
cloneOpts := &extgogit.CloneOptions{ /* ... */ Depth: depth /* ... */ }
cloneCommit has no depth variable and no Depth field in its CloneOptions at all. The only effect of also setting a branch is:
if opts.Branch != "" {
cloneOpts.SingleBranch = g.singleBranch // true by default
cloneOpts.ReferenceName = plumbing.NewBranchReferenceName(opts.Branch)
}
singleBranch defaults to true in NewClient ("Default to single branch as it is the most performant option"), and Client.clone dispatches on Commit before any other ref field, so .spec.ref.commit always routes here.
So branch + commit narrows the fetch from all branches, full history to one branch, full history. That is a real improvement over commit alone, but it is --single-branch, not a shallow clone. source-controller sets ShallowClone: true unconditionally in gitrepository_controller.go, and cloneCommit silently ignores it.
Skipped fast path
cloneCommit is also the only clone path without the LastObservedCommit short-circuit that the others open with:
if lastObserved := git.TransformRevision(opts.LastObservedCommit); lastObserved != "" {
head, err := g.getRemoteHEAD(ctx, url, ref, authMethod) // ls-remote, no clone
// ...
if head != "" && shortRef == lastObserved {
return c, nil // nothing fetched
}
}
This is visible in the controller logs. A branch- or tag-pinned GitRepository logs no changes since last reconciliation; a commit-pinned one logs artifact up-to-date with remote revision — i.e. it cloned first and only then discovered the artifact was already current.
This seems like the most valuable thing to fix: a commit pin is immutable, so polling can never discover a new revision, yet every interval pays for a full-history fetch.
Impact
Observed on a production cluster, source-controller v1.9.4:
- Three commit-pinned GitRepositories (
branch+commit+sparseCheckout) against one large monorepo,interval: 1h. - Each reconcile takes 3–4 minutes and drives the container's Go heap to its 1Gi limit. Since
GOMEMLIMITis wired tolimits.memoryin the stock manifest, Go enters continuous GC at that ceiling,/healthzmisses the Kubernetes defaulttimeoutSeconds: 1, and the kubelet restarts the container. - 49 restarts in ~6 days, every one of them
Killing: Container manager failed liveness probe. The 10sterminationGracePeriodSecondsis then too short for a clean shutdown, so the container is SIGKILLed (exit 137) without releasing its leader lease, costing another ~45s of lease expiry per restart, after which everything reconciles at once and can spike again. - A tag-pinned GitRepository against the same monorepo in the same cluster returns instantly via the fast path — useful as a control.
Not filing this as "source-controller crashes"; the resource limits are ours to tune. The point is that the documented mitigation (add branch to get a shallow clone) does not exist, so there is no way to reduce the fetch cost of a commit pin.
Proposals
- Docs: correct the Commit example —
branch+commitrestricts the fetch to a single branch, it does not make the clone shallow. - Bounded, incremental deepening: give
cloneCommita real depth-limited fetch. Start at a modest depth, and deepen incrementally (git fetch --deepen-style) until the commit resolves, falling back to full history if it is never found. In a GitOps flow the pinned commit is usually close to the branch tip, so a first attempt at a small depth would hit the overwhelming majority of the time. A user-facing knob (e.g..spec.ref.depth, or a controller flag for the initial depth) would let people tune it. - Fast path for commit pins: move the artifact-up-to-date check ahead of the clone when
.spec.ref.commitis set. If the pinned commit equals the last observed revision and the artifact is present in storage, no fetch is needed at all.
(2) and (3) are independent; (3) alone would eliminate the recurring cost for anyone pinning commits, while (2) helps the first fetch and any genuine revision change.
Versions
- source-controller
v1.9.4 github.com/fluxcd/pkg/git v0.52.0(per source-controller'sgo.mod)git/gogit/clone.gois blobb59744db03f544e46e090a1fdbd7cdd0737a731con bothgit/v0.52.0andmainat time of writing, so this is current onmaintoo.
Happy to open a PR for the docs fix, and to help with (2)/(3) if the approach sounds right.
- Lingua principale
- Go
- Stelle
- 283
- Fork
- 252
- Merge medio
- 1h 6m
- PR unite (30g)
- 12
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di fluxcd/source-controller
-
area/docs good first issue help wanted
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
fluxcd/source-controller#666 · 2 commenti ·
-
area/git bug
Difficoltà 4/5 3-5 giorni Idoneità per principianti 55/100
fluxcd/source-controller#2165 · 3 commenti ·
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 45/100
fluxcd/source-controller#2150 ·
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 55/100
fluxcd/source-controller#2136 · 1 commento ·
-
HelmRepository: use conditional HTTP requests (ETag / If-Modified-Since) when fetching index.yaml Aperta
Difficoltà 4/5 3-5 giorni Idoneità per principianti 48/100
fluxcd/source-controller#2113 · 1 commento ·
Tutte le issue di fluxcd/source-controller
Issue simili
-
bug github_actions
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
registrystack/registry-stack#1393 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
JakeChampion/lang#10213 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
oasisprotocol/oasis-sdk#2523 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100