storage 9.0.0: retries never stop and run about 1 ms apart (maxRetries ignored, delays read as milliseconds)
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 52/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- nodejs, typescript
- Ambito
- api
Direzione di ricerca
Start in the StorageTransport makeRequest method, where the retryConfig object is built from maxRetries and retryOptions (src/storage-transport.ts, compiled to build/cjs/src/storage-transport.js). Check how the shouldRetry wrapper ignores the attempt count, and whether maxRetryDelay and totalTimeout are passed to gaxios in seconds when gaxios expects milliseconds. Run the reproduction script with a local server that returns 503. Done when the call rejects after about 4 requests and roughly 15 seconds, and no TimeoutNegativeWarning appears.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Package: @google-cloud/storage (handwritten/storage)
Environment
- @google-cloud/storage 9.0.0 (gaxios 8.3.3, google-auth-library 11.3.4)
- Node 26.5.1 (same under Bun 1.4)
- 8.0.0 behaves as documented with the same script
Steps to reproduce
A local server that always answers 503, so no bucket or credentials are needed:
import http from "node:http";
import { Storage } from "@google-cloud/storage";
import { JWT } from "google-auth-library";
let hits = 0;
const server = http.createServer((req, res) => {
hits++;
res.writeHead(503, { "content-type": "application/json" });
res.end('{"error":{"code":503,"message":"down"}}');
});
await new Promise((r) => server.listen(0, "127.0.0.1", r));
const authClient = new JWT({ email: "[email protected]", key: "fake" });
authClient.credentials = { access_token: "t", expiry_date: Date.now() + 3600e3 };
const storage = new Storage({ projectId: "p", authClient, apiEndpoint: `http://127.0.0.1:${server.address().port}` });
const t0 = Date.now();
const outcome = await Promise.race([
storage.bucket("b").file("o").getMetadata().then(() => "resolved", (e) => `rejected code=${e.code}`),
new Promise((r) => setTimeout(() => r("still pending"), 8000)),
]);
console.log({ outcome, ms: Date.now() - t0, requests: hits });
process.exit(0);
Actual (9.0.0)
TimeoutNegativeWarning: -2 is a negative number.
Timeout duration was set to 1.
{ outcome: 'still pending', ms: 8001, requests: 5039 }
The call never settles and sends about 600 requests per second. The same happens for a 429 on an idempotent upload (file.save(buffer, { resumable: false, preconditionOpts: { ifGenerationMatch: 0 } })), and a refused connection is still pending after 20 s.
Expected (8.0.0)
{ outcome: 'rejected code=503', ms: 15430, requests: 4 }
maxRetries (default 3) with exponential backoff, as retryOptions documents.
Cause
In StorageTransport.makeRequest (build/cjs/src/storage-transport.js):
retryConfig: {
retry: maxRetries,
noResponseRetries: maxRetries,
maxRetryDelay: this.retryOptions.maxRetryDelay,
retryDelayMultiplier: this.retryOptions.retryDelayMultiplier,
totalTimeout: this.retryOptions.totalTimeout,
shouldRetry: (err) => !isRetryDisabled && !!this.retryOptions.retryableErrorFn?.(err),
},
- No retry cap. In gaxios,
retryandnoResponseRetriesare only enforced inside the defaultshouldRetryRequest. Passing a customshouldRetryreplaces that function (config.shouldRetry || shouldRetryRequest), and the custom one never looks atcurrentRetryAttempt, so retries continue for as long as the error is retryable. - Seconds passed as milliseconds.
maxRetryDelay(64) andtotalTimeout(600) are documented in seconds, andresumable-upload.jsmultiplies them by 1000, but gaxios reads them as milliseconds. The backoff is therefore capped at 64 ms, and once 600 ms have passedtotalTimeout - elapsedis negative, which is where theTimeoutNegativeWarningand the 1 ms delay come from.
Impact
During a 5xx or 429 incident, or a network failure, every idempotent call hangs until the caller's own deadline while sending hundreds of requests per second to the API, instead of failing after about 15 seconds.
Suggested fix
Enforce the attempt cap in the shouldRetry wrapper (for example err.config.retryConfig.currentRetryAttempt < maxRetries), and convert maxRetryDelay and totalTimeout to milliseconds when building the gaxios retryConfig.
- Lingua principale
- TypeScript
- Stelle
- 3.2k
- Fork
- 732
- Merge medio
- 2g 11h
- PR unite (30g)
- 99
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di googleapis/google-cloud-node
-
Broken deprecation chainAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
googleapis/google-cloud-node#9854 ·
I maintainer di solito rispondono entro 1 giorno
-
pubsub: "Operation attempted on ended Span" when a modack is flushed after the message was ackedApertaapi: pubsub
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
googleapis/google-cloud-node#9853 ·
I maintainer di solito rispondono entro 1 giorno
-
storage: V4 getSignedUrl sorts canonical headers with localeCompare, so they disagree with X-Goog-SignedHeadersForse già presa @itsrayeedscodespace l’ha presa 9 giorni fa. Apertaapi: storage
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
googleapis/google-cloud-node#9840 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
firestore: getPartitions() emits gax AutopaginateTrueWarning in every processForse già presa @TheRealDanii l’ha presa 13 giorni fa. Apertaapi: firestore
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
googleapis/google-cloud-node#9736 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 82/100
googleapis/google-cloud-node#9723 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di googleapis/google-cloud-node
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
vercel-labs/ui#809 ·
I maintainer di solito rispondono entro 1 giorno
-
bot:ai-assisted component:platform-js status:untriaged
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 74/100
midnightntwrk/platform-js#38 ·
-
[bug] Second false positive on high-cost plugin alert: detection only compares patch entry id, community plugin dsh-auto | [bug] 高成本插件告警第二次误报:检测只比对 patch 条目 id,社区插件 dsh-auto-review 被认成 @deepseek-ai/dsh-experimental-auto-reviewForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
ranxianglei/billion-context#7068 ·
I maintainer di solito rispondono entro 1 giorno
-
vendor: bump whisper.cpp past 6a8addd — the current pin aborts the app on a failed Metal initAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
mybigday/whisper.rn#348 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
six-group/six-webcomponents#600 ·