Make fewer git calls per operation

オープン
  • 何か問題がありますか？
#325 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
45/100
issue の種類
リファクタリング
明瞭さ
おおむね明確
活発さ
活発
技術スタック
git, rust
領域
developer-experience, performance

調査の方向性

Start with the helpers in crates/newgit-core/src/source.rs and crates/newgit-core/src/compose.rs, then inspect crates/newgit-core/tests/restore_sparse_paths.rs and measure Git process counts with GIT_TRACE2_EVENT. The issue lists batching and disabling auto-maintenance as options, but choosing an approach requires understanding the repo’s operations; done means measuring call counts before and after and restoring the macOS suite to pull request CI.

索引モデルが issue の本文から書いたものです。

説明

newgit starts a new git process for nearly every question it asks the repo. On Linux that costs little. On macOS a process start is expensive, and on GitHub's macOS runners it's the whole cost of CI: the suite there is CPU-bound, at ~30ms per git spawn. That's why macOS left PR CI in #321. A developer's Mac pays the same cost on every spawn, checkpoint, and undo.

How much

crates/newgit-core/tests/restore_sparse_paths.rs starts 816 git processes for 4 tests (counted with GIT_TRACE2_EVENT):

subcommand count
rev-parse 116
rev-list 97
maintenance (auto, after fetch/commit) 62
for-each-ref 57
fetch 53
update-ref 51
cat-file 37
config 29
diff-tree 23
unpack-objects 22

These are the read and ref plumbing calls that git can batch.

Where to look

Most calls go through a few helpers: git() in crates/newgit-core/src/source.rs, and git(), git_env(), git_lines() and command() in crates/newgit-core/src/compose.rs. compose.rs already uses cat-file --batch in places, so the pattern exists.

Options, cheapest first

  1. Turn off auto-maintenance in newgit's own repos. Pass -c maintenance.auto=false -c gc.auto=0 (or set it in the store's config). That removes ~8% of spawns at once. newgit should probably decide when its internal repos get gc'd anyway, rather than have git do it after every fetch.
  2. Batch what git already batches:
    • rev-parse A B C in one call instead of three.
    • update-ref --stdin for groups of ref updates. This is also atomic, which is a correctness win.
    • One for-each-ref with a format string instead of repeated lookups.
    • A long-lived cat-file --batch / --batch-check per operation.
  3. Cache within an operation. Many rev-parse and for-each-ref calls probably ask the same question more than once within a single command.
  4. Use a library for reads (gix/libgit2), which avoids spawning anything for read-only queries. This is in tension with "never reimplement git's object model": it isn't reimplementing it, but it does add a second implementation that newgit depends on. Needs a decision before anyone starts.

Done when

The git call count for a typical spawn / checkpoint / undo is measured before and after, and the macOS suite is fast enough to put back on pull requests.

主要言語
Rust
スター
0
フォーク
0
平均マージ
7時間 27分
マージ済み PR（30日）
140

環境構築

このプロジェクトの環境構築ファイルはまだ確認していません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

Spheroman/newgit のほかの issue

Spheroman/newgit の issue をすべて見る

似ている issue

Rust の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。