Unstable results from the API

Open
#126 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
java
Domain
api, backend

Research direction

Start by reproducing the /major_versions and /packages requests from the report, using the cited Maven Wrapper workflow and recording response status and bodies across repeated attempts. Compare the intermittent 503, empty-result, and missing-package cases to determine whether the cause is in the API or its infrastructure; done means the cause is documented and a concrete stabilization path is identified.

Written by the indexing model from the issue text.

Description

I've been playing with the Disco API since a few days as part of https://github.com/apache/maven-wrapper/pull/337.

I did setup some IT that do use the Disco API to actually perform a few requests and tests the real behaviours. However, the results are really unstable and the same requests may work or return empty results, no idea why.

For example, in this workflow, most ITs failed:
https://github.com/apache/maven-wrapper/actions/runs/16498393081/job/46650180851?pr=337

There are mainly two different errors:
the first one is about LTS versions:

Warning:  DEBUG: Known LTS version 17 detected as non-LTS!
Warning:  DEBUG: This indicates a Disco API issue. Re-fetching with debug logging...
Warning:  DEBUG: Fetching LTS versions from Disco API due to incorrect LTS detection
Warning:  DEBUG: Disco API request details:
Warning:    URL: https://api.foojay.io/disco/v3.0/major_versions?ea=false&ga=false&maintained=true&include_build=false&include_versions=false
Warning:    Max attempts: 3
Warning:    Base delay: 2000ms
Warning:    Attempt 1/3: HTTP 200
Warning:    Response length: 32 characters
Warning:    Response body: {  "result":[  ],  "message":""}
Warning:  DEBUG: Parsed LTS versions from API response: []
Warning:  DEBUG: Fallback LTS versions: [17, 33, 21, 8, 25, 11, 29]
Warning:  DEBUG: LTS version 17 still not found after debug re-fetch!
Warning:  DEBUG: This is a confirmed Disco API issue. Using fallback LTS detection.

the second one is about a JDK 17 temurin which is not found

Querying Disco API for JDK versions: https://api.foojay.io/disco/v3.0/packages?distro=temurin&package_type=jdk&jdk_version=17&operating_system=macos&architecture=aarch64&archive_type=tar.gz&latest=available&release_status=ga&directly_downloadable=true&include_versions=false
No matching JDK version found for 17 with distribution temurin
DEBUG: JDK 17 with Temurin should be available but was not found!
DEBUG: This indicates a Disco API issue. Logging request details:
DEBUG: API URL: https://api.foojay.io/disco/v3.0/packages?distro=temurin&package_type=jdk&jdk_version=17&operating_system=macos&architecture=aarch64&archive_type=tar.gz&latest=available&release_status=ga&directly_downloadable=true&include_versions=false
DEBUG: HTTP Status: 
DEBUG: Response length: 45 characters
DEBUG: Full API response: {"result":[],"message":"No package(s) found"}
DEBUG: Expected pattern: "java_version":"17.x.x"
DEBUG: Actual matches: 

I often had 503 errors in the past days, so I implemented a retry strategy.
We also had the failure where most packages were not returned anymore.

Is there anything we can do to help stabilise the infrastructure ? What could cause such instability ?

If there's no way to improve it, I'll have to find another solution for Maven.

Dominant language
Java
Stars
154
Forks
18
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from foojayio/discoapi

All issues in foojayio/discoapi

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.