Batch commit throw DEADLINE_EXCEEDED but it actually create the documents
@milaGGL 已经在做这个了。
开始于 2025年5月28日。
评估
这个 Issue 还没有评估数据。
描述
Environment
- Operating System version:
Mac M1 arm64 - Firebase SDK version:
13.0.1 - Firebase Product:
firestore - Node.js version:
v20.18.0 - NPM version:
10.9.0
The problem
I've class to handle more then 500 operation by simply creating new batch when reach 500, And when commit I'll run commit on each of the batches, And whenever any commit is failing it go to re-try commit that failed batch.
I had the batch commit fail with error of Deadline exceeded after 60.001s,metadata filters: 0.085s,LB pick: 0.001s,remote_addr=142.250.181.234:443 then after few re-try it was failing for new reason ALREADY_EXISTS: Document already exists and the data was created in the firestore !, If batch commit throw error it shouldn't write in firestore, but it seems commit throw error of the 60s timeout limits but data was actually successfully committed,
full log
commiting batch index 0 , ID: 0_3b6ca345a4a22d6f4d0a2f38eb5507bb
index 0 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.998s,metadata filters: 0.086s,remote_addr=142.250.181.234:443
commiting batch index 1 , ID: 1_a20059deface571877dbc25093dee067
index 1 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.889s,metadata filters: 0.041s,remote_addr=142.250.181.234:443
commiting batch index 2 , ID: 2_de705d40bf041659aa787c5e749ea4d3
index 2 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.025s,remote_addr=142.250.181.234:443
re-commit batchs count 3
total batchs 3 Retry count 2
start commit in sequentially mode
commiting batch index 0 , ID: 0_3b6ca345a4a22d6f4d0a2f38eb5507bb
index 0 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.087s,remote_addr=142.250.181.234:443
commiting batch index 1 , ID: 1_a20059deface571877dbc25093dee067
index 1 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 60.000s,metadata filters: 0.045s,remote_addr=142.250.181.234:443
commiting batch index 2 , ID: 2_de705d40bf041659aa787c5e749ea4d3
index 2 failed 6 ALREADY_EXISTS: Document already exists: projects/PROJECT_ID/databases/(default)/documents/COLLECTION_NAME/KSYlyNKEwJ4tkKGyZ0WR
re-commit batchs count 3
total batchs 3 Retry count 3
start commit in sequentially mode
commiting batch index 0 , ID: 0_3b6ca345a4a22d6f4d0a2f38eb5507bb
index 0 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.086s,remote_addr=142.250.181.234:443
commiting batch index 1 , ID: 1_a20059deface571877dbc25093dee067
index 1 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.043s,LB pick: 0.001s,remote_addr=142.250.181.234:443
commiting batch index 2 , ID: 2_de705d40bf041659aa787c5e749ea4d3
index 2 failed 6 ALREADY_EXISTS: Document already exists: projects/PROJECT_ID/databases/(default)/documents/COLLECTION_NAME/KSYlyNKEwJ4tkKGyZ0WR
re-commit batchs count 3
total batchs 3 Retry count 4
start commit in sequentially mode
commiting batch index 0 , ID: 0_3b6ca345a4a22d6f4d0a2f38eb5507bb
index 0 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.086s,remote_addr=142.250.181.234:443
commiting batch index 1 , ID: 1_a20059deface571877dbc25093dee067
index 1 failed 6 ALREADY_EXISTS: Document already exists: projects/PROJECT_ID/databases/(default)/documents/COLLECTION_NAME/fMQvNaE8y6m8Gx7hUwdb
commiting batch index 2 , ID: 2_de705d40bf041659aa787c5e749ea4d3
index 2 failed 6 ALREADY_EXISTS: Document already exists: projects/PROJECT_ID/databases/(default)/documents/COLLECTION_NAME/KSYlyNKEwJ4tkKGyZ0WR
Reached max try limit, 4 aborting commits.
Steps to reproduce:
I had bad internet and I was writing the 1300 documents with create operation
Relevant Code:
Code of my class
_createBatch(index) {
const batch = {
id: index + '_' + crypto.randomBytes(16).toString('hex'),
batch: this.firestore.batch(),
}
this.batches.push(batch)
return batch
}
async commit() {
for (let i = 0; i < originalBatchsCount; i++) {
const item = this.batches[i]
if (showBatchCounterLog) {
console.log('commiting batch index', i, ', ID:', item.id)
}
try {
await item.batch.commit()
console.log('index is finished', i, 'ID', item.id)
fulfilledIDs.push(item.id)
} catch (e) {
console.log('index ', i, 'failed', e.message)
}
}
}
- 主要语言
- TypeScript
- 星标
- 1.7k
- 派生
- 419
- 平均合并
- 4 天 20 小时
- 30 天内合并 PR
- 16
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
firebase/firebase-admin-node 的其他 Issue
-
难度 3/5 1-2 天 新手友好度 35/100
firebase/firebase-admin-node#3234 ·
-
firebase/firebase-admin-node#3221 · 3 条评论 · 已指派 1 人 ·
-
api: messaging
难度 3/5 1-2 天 新手友好度 70/100
firebase/firebase-admin-node#3215 ·
-
api: messaging
难度 5/5 一周以上 新手友好度 28/100
firebase/firebase-admin-node#3214 ·
-
api: firestore type: feature request
firebase/firebase-admin-node#3183 · 1 条评论 · 已指派 1 人 ·
查看 firebase/firebase-admin-node 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 84/100
bcgov/bc-wallet-mobile#4761 · 1 条评论 ·
-
external-issue to-triage
难度 2/5 1-3 小时 新手友好度 88/100
-
area-deployment area-integrations triage:bot-seen
难度 2/5 半天 新手友好度 86/100
-
难度 2/5 1-3 小时 新手友好度 82/100
-
refactor
难度 2/5 1-3 小时 新手友好度 84/100