Bazel maintains HTTP connection after 401 with failed credential helper

Description of the bug:

When the credential helper fails in an async manner as documented, the HTTP downloader continues to attempt a download anyway. When this inevitably results in a 401 response, the HTTP downloader is cached in the Bazel server and re-used for all subsequent attempts to download from that server even when the credentials have been updated in the credential helper.

This requires that Bazel is shut down after re-authenticating with the credential helper before it will re-attempt running the credential helper and resolving the 401 error.

The ideal solution would be for the HTTP downloader not to plough on regardless if the credential helper fails. However, if it is desired to do so, then the connection to the upstream should not be cached and therefore allow the credential helper to be invoked once again when a new build command comes in.

Furthermore, this caching of connection completely ignores the expiry provided to Bazel in the credential helper response.

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Use a http_archive to download from a server that requires authentication. Provide the authentication through a credential_helper script. Provide invalid credentials on the first execution of Bazel (exit code 1). Update the script to provide the correct credentials (exit 0 with valid JSON). Note that Bazel still gets 401 from the download.

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 8.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response