C# build targets duplicate the Copilot CLI bundle per source path: no shared cache, archive kept, payload copied twice

Offen
  • Stimmt etwas nicht?
#2,903 0 Kommentare 1 Reaktion 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
45/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
csharp
Bereich
backend, build-system

Rechercherichtung

Start with build/GitHub.Copilot.SDK.targets in the NuGet package; inspect how _CopilotCacheDir, the archive download and extraction, and native payload staging are defined. Reproduce with the two source paths in the issue and compare archive and output contents. Done when repeated builds avoid the reported duplicate storage while still providing the required native files to each app.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Summary

The GitHub.Copilot.SDK MSBuild targets cache the downloaded Copilot CLI bundle under
$(IntermediateOutputPath)copilot-cli/<version>/<platform>/. Because the intermediate output path of a
.NET 10 file-based app (dotnet run app.cs) is a per-source-path directory in the user's runfile cache,
every distinct source path re-downloads and re-extracts the CLI bundle. Nothing is shared between
entries, so identical bytes are duplicated once per path.

On a machine where a test harness ran the same app from many temporary directories, this produced
~1,400 copies of the same 84 MB archive and its extracted payload — 755 GB.

This is the SDK-package half of a larger report; the file-based-app cache-keying half is filed at
dotnet/sdk#56940.

Environment

GitHub.Copilot.SDK 1.0.26   (CopilotCliVersion 1.0.103)
.NET SDK 10.0.402
macOS 26 (arm64), APFS

~/.nuget/packages/github.copilot.sdk/1.0.26/build/GitHub.Copilot.SDK.targets:

<_CopilotCacheDir>$(IntermediateOutputPath)copilot-cli\$(CopilotCliVersion)\$(_CopilotPlatform)</_CopilotCacheDir>
<_CopilotArchivePath>$(_CopilotCacheDir)\copilot.tgz</_CopilotArchivePath>
...
<DownloadFile SourceUrl="$(_CopilotChecksumsUrl)" DestinationFolder="$(_CopilotCacheDir)" ... />
<DownloadFile SourceUrl="$(_CopilotDownloadUrl)" DestinationFolder="$(_CopilotCacheDir)" DestinationFileName="copilot.tgz" ... />
<Exec Command="&quot;$(_TarCommand)&quot; -xzf &quot;$(_CopilotArchivePath)&quot; --strip-components=1 -C &quot;$(_CopilotCacheDir)&quot;" ... />

The cache is scoped to IntermediateOutputPath, which is scoped to the project — and for a file-based
app the project is per source path.

Reproduction

// repro.cs
#:package [email protected]
Console.WriteLine("repro");

CACHE="$HOME/Library/Application Support/dotnet/runfile"

mkdir -p /tmp/rfr/a /tmp/rfr/b
cp repro.cs /tmp/rfr/a/ && cp repro.cs /tmp/rfr/b/   # byte-identical

(cd /tmp/rfr/a && dotnet run repro.cs)   # cold: 4.4s, creates repro-<hash A>
(cd /tmp/rfr/a && dotnet run repro.cs)   # warm: 0.14s, no new directory
(cd /tmp/rfr/b && dotnet run repro.cs)   # cold again: 4.7s, creates repro-<hash B>

Both cache directories are 546 MB. The archive is byte-identical in both, and physically duplicated:

$ stat -f 'inode=%i links=%l size=%z' \
    $CACHE/repro-552f7c…/obj/debug/copilot-cli/1.0.103/darwin-arm64/copilot.tgz \
    $CACHE/repro-16b020…/obj/debug/copilot-cli/1.0.103/darwin-arm64/copilot.tgz
inode=96355505 links=1 size=83944573 …/repro-552f7c…/copilot.tgz
inode=96356143 links=1 size=83944573 …/repro-16b020…/copilot.tgz

$ shasum -a 256 …/repro-552f7c…/copilot.tgz …/repro-16b020…/copilot.tgz
4cba81686f2d4a56e2ea2e1bd1cdde303d07f28de11593e1db1786f1219b8f74  …/repro-552f7c…/copilot.tgz
4cba81686f2d4a56e2ea2e1bd1cdde303d07f28de11593e1db1786f1219b8f74  …/repro-16b020…/copilot.tgz

Per-artifact cost breakdown (one cold build, 546 MB)

284 MB  obj/debug/copilot-cli/1.0.103/darwin-arm64/
         81 MB   copilot.tgz                       (the downloaded archive, kept after extraction)
         98 MB   prebuilds/darwin-arm64/runtime.node
         15 MB   foundry-local-sdk/…/libfoundry_local.dylib
         12 MB   prebuilds/darwin-arm64/libOneAuthInterop.dylib
261 MB  bin/debug/runtimes/osx-arm64/native/
         98 MB   runtime.node
         98 MB   libcopilot_runtime.dylib
         12 MB   libOneAuthInterop.dylib
 18 MB  bin/debug/GitHub.Copilot.SDK.dll
0.3 MB  the app's own dll + pdb

Two further observations from the same measurement:

  1. bin/debug/runtimes/osx-arm64/native/runtime.node and libcopilot_runtime.dylib are the same
    file.     Both are 98 MB with sha256 70280452e26d03c3b6bd77b47510460b70416e0e421eac336c134a397a964fc8
    and links=1 — the same payload is copied twice into one output directory. The targets comment says
    the cdylib is "renamed from the tarball's napi-style prebuilds//runtime.node", so this looks
    like both the original name and the renamed copy surviving.
  2. copilot.tgz is kept after extraction (81 MB). The extraction target never deletes it, so the
    archive is retained in every cache directory alongside the extracted tree it produced.

So even a single, correctly reused artifact directory holds the payload roughly three times over
(obj/…/prebuilds/runtime.node, bin/…/runtime.node, bin/…/libcopilot_runtime.dylib), plus the
archive.

Workaround

Setting ArtifactsPath keeps the build output out of the shared cache:

dotnet run repro.cs --artifacts-path /tmp/scratch

This does work — build output lands under /tmp/scratch — but the runfile cache still gains an ~84 KB
stub directory per source path (csc.rsp, build-start.cache, build-success.cache), so it is a
mitigation rather than a fix.

Possible directions

  1. Cache the download under a location keyed by CopilotCliVersion + platform only (e.g. under
    $(NuGetPackageRoot) or a per-user CLI cache), and have each project's intermediate output link or
    copy from there. The archive and extracted payload are identical for every consumer of the same
    version.
  2. Delete copilot.tgz after successful extraction.
  3. Avoid emitting the same 98 MB payload under both runtime.node and libcopilot_runtime.dylib in the
    same output directory.
  4. Prefer hard links (or a copy-if-newer that preserves them) when staging the bundle into the output
    directory.

Impact

A five-line app referencing this package costs 546 MB of build output per distinct source path. That is
survivable for a normal project, but a test harness, CI matrix, or agent sandbox that generates a fresh
directory per case turns it into unbounded disk growth: ~1,400 directories, 755 GB on the machine this
was measured on.

Vorherrschende Sprache
Java
Sterne
10.7k
Forks
1.5k
Keine gemergten PRs in 30 T.

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus github/copilot-sdk

Alle Issues in github/copilot-sdk

Ähnliche Issues

Weitere Issues zu Java

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.