storage: `file.download()` of a large object emits MaxListenersExceededWarning since 8.0 (response body piped twice)
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 3/5
- 预计耗时
- 半天
- 新手友好度
- 52/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
- 技术栈
- google-cloud, node.js, typescript
- 领域
- cloud
调研方向
Start with the repro script in the issue, which needs no credentials and triggers the warning on the 2 MiB object. The two pipelines that both read the response body are in build/cjs/src/file.js (createReadStream's onResponse) and in teeny-request's build/src/index.js (stream mode, plus the .resume() call in makeRequest). Done means the 5-download loop reports 0 MaxListenersExceededWarning while the downloads still return the correct bytes, and the fix may belong in either package.
由索引模型根据 Issue 内容生成。
描述
Environment
@google-cloud/storage8.4.0 (also 8.2.0); not 7.16.0teeny-request11.1.1,node-fetch3.3.2- Node.js 22.23.2
What happens
Each file.download() / createReadStream() of an object large enough to still be streaming when the response arrives prints:
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [PassThrough]. MaxListeners is 10.
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [PassThrough]. MaxListeners is 10.
The listeners are released when the download ends, so nothing actually leaks. But the warning fires once per download, so it floods the logs.
Cause
The PassThrough is node-fetch's response body. Two pipelines read it:
createReadStream'sonResponserunspipeline(rawResponseStream, ...transformStreams, throughStream, onComplete)(build/cjs/src/file.js, 8.4.0).- teeny-request's stream mode runs
pipeline(responseStream, requestStream, () => {})on the same body (teeny-request/build/src/index.js,requestStream.once('response', ...)). ThenmakeRequestcalls.resume()on that request stream and discards what it reads.
Together with node-fetch's own pump(response_, new PassThrough()) and the error listeners from node-fetch's Body and teeny-request, that comes to 11. With node-fetch 2 (storage 7.x) the total stays under 10, which is presumably why 7.x is quiet. Small objects finish before the second pipeline attaches, so they never warn.
Repro (no credentials, local server)
const http = require('node:http');
const crypto = require('node:crypto');
const { Storage, CRC32C } = require('@google-cloud/storage');
const body = Buffer.alloc(2 * 1024 * 1024, 'x'); // 2 MiB; a few KiB does not trigger it
const md5 = crypto.createHash('md5').update(body).digest('base64');
const c = new CRC32C(); c.update(body); const crc32c = c.toString();
let warnings = 0;
process.on('warning', (w) => { if (w.name === 'MaxListenersExceededWarning') warnings++; });
const server = http.createServer((req, res) => {
res.writeHead(200, {
'content-type': 'application/octet-stream',
'content-length': body.length,
'x-goog-hash': `crc32c=${crc32c},md5=${md5}`,
'x-goog-stored-content-encoding': 'identity',
});
res.end(body);
});
server.listen(0, '127.0.0.1', async () => {
const storage = new Storage({ apiEndpoint: `http://127.0.0.1:${server.address().port}`, projectId: 'test' });
const file = storage.bucket('bucket').file('object');
for (let i = 0; i < 5; i++) await file.download();
console.log(`5 downloads, ${warnings} MaxListenersExceededWarning`);
server.close();
});
Output: 8.4.0 → 5 downloads, 10 MaxListenersExceededWarning; 8.2.0 → 10; 7.16.0 → 0.
Expected
The response body is consumed once, either by storage's pipeline or by the request stream it gets from teeny-request, not by both. Then no warning.
- 主要语言
- TypeScript
- 星标
- 3.2k
- 派生
- 732
- 平均合并
- 2 天 11 小时
- 30 天内合并 PR
- 99
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
googleapis/google-cloud-node 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 66/100
googleapis/google-cloud-node#9854 ·
维护者通常 1 天内回复
-
pubsub: "Operation attempted on ended Span" when a modack is flushed after the message was acked未关闭api: pubsub
难度 2/5 1-3 小时 新手友好度 82/100
googleapis/google-cloud-node#9853 ·
维护者通常 1 天内回复
-
storage: V4 getSignedUrl sorts canonical headers with localeCompare, so they disagree with X-Goog-SignedHeaders可能已有人在做 @itsrayeedscodespace 于 9 天前认领。 未关闭api: storage
难度 2/5 1-3 小时 新手友好度 82/100
googleapis/google-cloud-node#9840 · 1 条评论 ·
维护者通常 1 天内回复
-
firestore: getPartitions() emits gax AutopaginateTrueWarning in every process可能已有人在做 @TheRealDanii 于 13 天前认领。 未关闭api: firestore
难度 2/5 1-3 小时 新手友好度 76/100
googleapis/google-cloud-node#9736 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 82/100
googleapis/google-cloud-node#9723 ·
维护者通常 1 天内回复
查看 googleapis/google-cloud-node 的全部 Issue
相似的 Issue
-
Type/Improvement
难度 2/5 1-3 小时 新手友好度 67/100
OpenNSW/nsw-srilanka#1259 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
oxc-project/tsrs#1010 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
Fission-AI/OpenSpec#2260 ·
维护者通常 1 天内回复
-
Add: ESP 1 UK未关闭check:passed streams:add
难度 1/5 1-3 小时 新手友好度 64/100
维护者通常 1 天内回复
-
area:infra area:storage bug
难度 2/5 1-3 小时 新手友好度 76/100
THU-MAIC/OpenMAIC#2311 · 1 条评论 ·
维护者通常 1 天内回复