perf(trace): post-commit starts three git processes for each committed file

Aperta
  • Qualcosa non va?
#3,756 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
62/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
git, go
Ambito
cli, performance

Direzione di ricerca

Start with settleCommittedLines in app/cli/pkg/action/trace_hook_handler.go and FileAt in app/cli/internal/trace/git/lines.go; check how both ExecClient and GoGitClient read committed files. Then inspect recordRootChanges in app/cli/pkg/action/trace_agent_hook.go and the relevant existing tests. Done when both callers can read multiple paths without resolving the same commit separately for every file, and paths missing from the commit still produce no content.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

chainloop component/CLI performance trace-provider
Problem

The post-commit hook of Chainloop Trace starts 3 git processes for each committed file that has pending AI lines. A large commit makes the hook slow.

settleCommittedLines (app/cli/pkg/action/trace_hook_handler.go) calls client.FileAt(repoRoot, sha, path) one time for each pending path in each session. ExecClient.FileAt (app/cli/internal/trace/git/lines.go) runs these commands for each call:

  1. git rev-parse --verify --quiet <sha>^{commit}
  2. git cat-file -e <sha>:<path>
  3. git cat-file blob <sha>:<path>

The commit SHA is the same for all the files. Thus step 1 runs again for each file without a reason.

Measurement

Local clone of this repository (3.3k files). One commit changes 301 files, and all the changes are AI lines that are pending.

Item Value
Git processes that post-commit starts More than 900
Time of post-commit Approximately 5.2 s

This is on main. It is not caused by #3631.

Suggested fix

Read all the files with one git cat-file --batch process. Send one <sha>:<path> line for each path on stdin. For a path that is not in the commit, Git prints <sha>:<path> missing. Use this result as "no content", as FileAt does now. Resolve the commit one time before the loop.

For example, add a Client method that takes a revision and a list of paths and returns the content by path. The GoGitClient version can read the commit tree one time.

recordRootChanges (app/cli/pkg/action/trace_agent_hook.go) calls FileAt for each changed path in the same way, during PostToolUse for Bash calls. The same method can fix that caller too.

Lingua principale
Go
Stelle
582
Fork
61
Merge medio
12h 47m
PR unite (30g)
137

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di chainloop-dev/chainloop

Tutte le issue di chainloop-dev/chainloop

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.