Add configurable maximum number of attempts and wait time to download retries
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 42/100
Research direction
Start in src/LibraryManager/Cache/CacheService.cs, especially the download and cache-refresh paths linked in the issue, and trace how attempts and the 200ms backoff are supplied. Identify the existing configuration surface before deciding where the two settings belong. Done means configurable attempt and wait-time values with defaults of 5 and a longer backoff, covered by the relevant retry tests.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
When downloading a package fails, LibMan can retry it, but it doesn't seem to do this when simply downloading a package anew. This causes sporadic build failures for us with the kenwheeler/slick@ecb6ea2 library references (and it seems only this one), LibMan failing with the following exception:
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : Microsoft.Web.LibraryManager.Contracts.ResourceDownloadException: Failed to download resource from "https://cdn.jsdelivr.net/gh/kenwheeler/slick@ecb6ea2/slick/slick.min.css" [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (Not Found). [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at Microsoft.Web.LibraryManager.Cache.WebRequestHandler.GetStreamAsync(String url, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : --- End of inner exception stack trace --- [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at Microsoft.Web.LibraryManager.Cache.WebRequestHandler.GetStreamAsync(String url, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at Microsoft.Web.LibraryManager.Cache.CacheService.DownloadToFileAsync(String url, String fileName, Int32 attempts, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at Microsoft.Web.LibraryManager.Cache.CacheService.<>c__DisplayClass9_0.<<RefreshCacheAsync>g__DownloadFileIfNecessaryAsync|0>d.MoveNext() [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : --- End of stack trace from previous location --- [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at Microsoft.Web.LibraryManager.Utilities.ParallelUtility.DoActionAndRelease[T](Func`2 act, T input, SemaphoreSlim s) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at Microsoft.Web.LibraryManager.Utilities.ParallelUtility.ForEachAsync[T](Func`2 action, Int32 degreeOfParallelism, IEnumerable`1 items, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at Microsoft.Web.LibraryManager.Cache.CacheService.RefreshCacheAsync(IEnumerable`1 librariesCacheMetadata, ILogger logger, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : at Microsoft.Web.LibraryManager.Providers.BaseProvider.RefreshCacheAsync(ILibraryInstallationState state, ILibrary library, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
libman.json : error LIB010: Failed to download resource from "https://cdn.jsdelivr.net/gh/kenwheeler/slick@ecb6ea2/slick/slick.min.css" [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
Describe the solution you'd like
Ideally, we would not get such download failures in the first place. However, this seems to be noticeably frequent for kenwheeler/slick@ecb6ea2 for some reason. Only jsdelivr supports such digest expressions, so we can't try the others.
However, if this can't be solved somehow for good, it'd be great to have better retries:
- Make the number of attempts configurable, and use a default of 5, like it is for cache refreshes.
- The current backoff time between attempts is 200ms. This is too low for transient errors to reliably clear (even with 5 attempts it would be just 1s). Instead, make this configurable (we'd use something like 5s).
Describe alternatives you've considered
Having fallback providers, but that brings its own challenges.
Additional context
Nothing else to add.
- Dominant language
- C#
- Stars
- 486
- Forks
- 91
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from aspnet/LibraryManager
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
aspnet/LibraryManager#824 · 1 comment · 3 reactions ·
-
Difficulty 1/5 Under an hour Newbie friendliness 68/100
aspnet/LibraryManager#804 · 2 reactions ·
-
Difficulty 1/5 Under an hour Newbie friendliness 48/100
aspnet/LibraryManager#829 · 1 comment · 3 reactions ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
aspnet/LibraryManager#808 · 5 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 38/100
aspnet/LibraryManager#807 · 3 comments ·
All issues in aspnet/LibraryManager
Similar issues
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Type: enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
apache/arrow-adbc#4809 ·
-
type/automation type/tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
microsoft/vscode-azurefunctions#5197 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
microsoft/microsoft-ui-reactor#1274 ·