C# build targets duplicate the Copilot CLI bundle per source path: no shared cache, archive kept, payload copied twice

Ouverte
  • Quelque chose ne va pas ?
#2,903 0 commentaires 1 réaction 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
45/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
csharp
Domaine
backend, build-system

Piste de recherche

Start with build/GitHub.Copilot.SDK.targets in the NuGet package; inspect how _CopilotCacheDir, the archive download and extraction, and native payload staging are defined. Reproduce with the two source paths in the issue and compare archive and output contents. Done when repeated builds avoid the reported duplicate storage while still providing the required native files to each app.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Summary

The GitHub.Copilot.SDK MSBuild targets cache the downloaded Copilot CLI bundle under
$(IntermediateOutputPath)copilot-cli/<version>/<platform>/. Because the intermediate output path of a
.NET 10 file-based app (dotnet run app.cs) is a per-source-path directory in the user's runfile cache,
every distinct source path re-downloads and re-extracts the CLI bundle. Nothing is shared between
entries, so identical bytes are duplicated once per path.

On a machine where a test harness ran the same app from many temporary directories, this produced
~1,400 copies of the same 84 MB archive and its extracted payload — 755 GB.

This is the SDK-package half of a larger report; the file-based-app cache-keying half is filed at
dotnet/sdk#56940.

Environment

GitHub.Copilot.SDK 1.0.26   (CopilotCliVersion 1.0.103)
.NET SDK 10.0.402
macOS 26 (arm64), APFS

~/.nuget/packages/github.copilot.sdk/1.0.26/build/GitHub.Copilot.SDK.targets:

<_CopilotCacheDir>$(IntermediateOutputPath)copilot-cli\$(CopilotCliVersion)\$(_CopilotPlatform)</_CopilotCacheDir>
<_CopilotArchivePath>$(_CopilotCacheDir)\copilot.tgz</_CopilotArchivePath>
...
<DownloadFile SourceUrl="$(_CopilotChecksumsUrl)" DestinationFolder="$(_CopilotCacheDir)" ... />
<DownloadFile SourceUrl="$(_CopilotDownloadUrl)" DestinationFolder="$(_CopilotCacheDir)" DestinationFileName="copilot.tgz" ... />
<Exec Command="&quot;$(_TarCommand)&quot; -xzf &quot;$(_CopilotArchivePath)&quot; --strip-components=1 -C &quot;$(_CopilotCacheDir)&quot;" ... />

The cache is scoped to IntermediateOutputPath, which is scoped to the project — and for a file-based
app the project is per source path.

Reproduction

// repro.cs
#:package [email protected]
Console.WriteLine("repro");

CACHE="$HOME/Library/Application Support/dotnet/runfile"

mkdir -p /tmp/rfr/a /tmp/rfr/b
cp repro.cs /tmp/rfr/a/ && cp repro.cs /tmp/rfr/b/   # byte-identical

(cd /tmp/rfr/a && dotnet run repro.cs)   # cold: 4.4s, creates repro-<hash A>
(cd /tmp/rfr/a && dotnet run repro.cs)   # warm: 0.14s, no new directory
(cd /tmp/rfr/b && dotnet run repro.cs)   # cold again: 4.7s, creates repro-<hash B>

Both cache directories are 546 MB. The archive is byte-identical in both, and physically duplicated:

$ stat -f 'inode=%i links=%l size=%z' \
    $CACHE/repro-552f7c…/obj/debug/copilot-cli/1.0.103/darwin-arm64/copilot.tgz \
    $CACHE/repro-16b020…/obj/debug/copilot-cli/1.0.103/darwin-arm64/copilot.tgz
inode=96355505 links=1 size=83944573 …/repro-552f7c…/copilot.tgz
inode=96356143 links=1 size=83944573 …/repro-16b020…/copilot.tgz

$ shasum -a 256 …/repro-552f7c…/copilot.tgz …/repro-16b020…/copilot.tgz
4cba81686f2d4a56e2ea2e1bd1cdde303d07f28de11593e1db1786f1219b8f74  …/repro-552f7c…/copilot.tgz
4cba81686f2d4a56e2ea2e1bd1cdde303d07f28de11593e1db1786f1219b8f74  …/repro-16b020…/copilot.tgz

Per-artifact cost breakdown (one cold build, 546 MB)

284 MB  obj/debug/copilot-cli/1.0.103/darwin-arm64/
         81 MB   copilot.tgz                       (the downloaded archive, kept after extraction)
         98 MB   prebuilds/darwin-arm64/runtime.node
         15 MB   foundry-local-sdk/…/libfoundry_local.dylib
         12 MB   prebuilds/darwin-arm64/libOneAuthInterop.dylib
261 MB  bin/debug/runtimes/osx-arm64/native/
         98 MB   runtime.node
         98 MB   libcopilot_runtime.dylib
         12 MB   libOneAuthInterop.dylib
 18 MB  bin/debug/GitHub.Copilot.SDK.dll
0.3 MB  the app's own dll + pdb

Two further observations from the same measurement:

  1. bin/debug/runtimes/osx-arm64/native/runtime.node and libcopilot_runtime.dylib are the same
    file.     Both are 98 MB with sha256 70280452e26d03c3b6bd77b47510460b70416e0e421eac336c134a397a964fc8
    and links=1 — the same payload is copied twice into one output directory. The targets comment says
    the cdylib is "renamed from the tarball's napi-style prebuilds//runtime.node", so this looks
    like both the original name and the renamed copy surviving.
  2. copilot.tgz is kept after extraction (81 MB). The extraction target never deletes it, so the
    archive is retained in every cache directory alongside the extracted tree it produced.

So even a single, correctly reused artifact directory holds the payload roughly three times over
(obj/…/prebuilds/runtime.node, bin/…/runtime.node, bin/…/libcopilot_runtime.dylib), plus the
archive.

Workaround

Setting ArtifactsPath keeps the build output out of the shared cache:

dotnet run repro.cs --artifacts-path /tmp/scratch

This does work — build output lands under /tmp/scratch — but the runfile cache still gains an ~84 KB
stub directory per source path (csc.rsp, build-start.cache, build-success.cache), so it is a
mitigation rather than a fix.

Possible directions

  1. Cache the download under a location keyed by CopilotCliVersion + platform only (e.g. under
    $(NuGetPackageRoot) or a per-user CLI cache), and have each project's intermediate output link or
    copy from there. The archive and extracted payload are identical for every consumer of the same
    version.
  2. Delete copilot.tgz after successful extraction.
  3. Avoid emitting the same 98 MB payload under both runtime.node and libcopilot_runtime.dylib in the
    same output directory.
  4. Prefer hard links (or a copy-if-newer that preserves them) when staging the bundle into the output
    directory.

Impact

A five-line app referencing this package costs 546 MB of build output per distinct source path. That is
survivable for a normal project, but a test harness, CI matrix, or agent sandbox that generates a fresh
directory per case turns it into unbounded disk growth: ~1,400 directories, 755 GB on the machine this
was measured on.

Langage dominant
Java
Étoiles
10.7k
Forks
1.5k
Aucune PR mergée en 30 j

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de github/copilot-sdk

Toutes les issues de github/copilot-sdk

Issues similaires

Plus d'issues Java

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.