Batch commit throw DEADLINE_EXCEEDED but it actually create the documents
@milaGGL is already working on this.
Since May 28, 2025.
Assessment
This issue has not been assessed yet.
Description
Environment
- Operating System version:
Mac M1 arm64 - Firebase SDK version:
13.0.1 - Firebase Product:
firestore - Node.js version:
v20.18.0 - NPM version:
10.9.0
The problem
I've class to handle more then 500 operation by simply creating new batch when reach 500, And when commit I'll run commit on each of the batches, And whenever any commit is failing it go to re-try commit that failed batch.
I had the batch commit fail with error of Deadline exceeded after 60.001s,metadata filters: 0.085s,LB pick: 0.001s,remote_addr=142.250.181.234:443 then after few re-try it was failing for new reason ALREADY_EXISTS: Document already exists and the data was created in the firestore !, If batch commit throw error it shouldn't write in firestore, but it seems commit throw error of the 60s timeout limits but data was actually successfully committed,
full log
commiting batch index 0 , ID: 0_3b6ca345a4a22d6f4d0a2f38eb5507bb
index 0 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.998s,metadata filters: 0.086s,remote_addr=142.250.181.234:443
commiting batch index 1 , ID: 1_a20059deface571877dbc25093dee067
index 1 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.889s,metadata filters: 0.041s,remote_addr=142.250.181.234:443
commiting batch index 2 , ID: 2_de705d40bf041659aa787c5e749ea4d3
index 2 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.025s,remote_addr=142.250.181.234:443
re-commit batchs count 3
total batchs 3 Retry count 2
start commit in sequentially mode
commiting batch index 0 , ID: 0_3b6ca345a4a22d6f4d0a2f38eb5507bb
index 0 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.087s,remote_addr=142.250.181.234:443
commiting batch index 1 , ID: 1_a20059deface571877dbc25093dee067
index 1 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 60.000s,metadata filters: 0.045s,remote_addr=142.250.181.234:443
commiting batch index 2 , ID: 2_de705d40bf041659aa787c5e749ea4d3
index 2 failed 6 ALREADY_EXISTS: Document already exists: projects/PROJECT_ID/databases/(default)/documents/COLLECTION_NAME/KSYlyNKEwJ4tkKGyZ0WR
re-commit batchs count 3
total batchs 3 Retry count 3
start commit in sequentially mode
commiting batch index 0 , ID: 0_3b6ca345a4a22d6f4d0a2f38eb5507bb
index 0 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.086s,remote_addr=142.250.181.234:443
commiting batch index 1 , ID: 1_a20059deface571877dbc25093dee067
index 1 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.043s,LB pick: 0.001s,remote_addr=142.250.181.234:443
commiting batch index 2 , ID: 2_de705d40bf041659aa787c5e749ea4d3
index 2 failed 6 ALREADY_EXISTS: Document already exists: projects/PROJECT_ID/databases/(default)/documents/COLLECTION_NAME/KSYlyNKEwJ4tkKGyZ0WR
re-commit batchs count 3
total batchs 3 Retry count 4
start commit in sequentially mode
commiting batch index 0 , ID: 0_3b6ca345a4a22d6f4d0a2f38eb5507bb
index 0 failed 4 DEADLINE_EXCEEDED: Deadline exceeded after 59.999s,metadata filters: 0.086s,remote_addr=142.250.181.234:443
commiting batch index 1 , ID: 1_a20059deface571877dbc25093dee067
index 1 failed 6 ALREADY_EXISTS: Document already exists: projects/PROJECT_ID/databases/(default)/documents/COLLECTION_NAME/fMQvNaE8y6m8Gx7hUwdb
commiting batch index 2 , ID: 2_de705d40bf041659aa787c5e749ea4d3
index 2 failed 6 ALREADY_EXISTS: Document already exists: projects/PROJECT_ID/databases/(default)/documents/COLLECTION_NAME/KSYlyNKEwJ4tkKGyZ0WR
Reached max try limit, 4 aborting commits.
Steps to reproduce:
I had bad internet and I was writing the 1300 documents with create operation
Relevant Code:
Code of my class
_createBatch(index) {
const batch = {
id: index + '_' + crypto.randomBytes(16).toString('hex'),
batch: this.firestore.batch(),
}
this.batches.push(batch)
return batch
}
async commit() {
for (let i = 0; i < originalBatchsCount; i++) {
const item = this.batches[i]
if (showBatchCounterLog) {
console.log('commiting batch index', i, ', ID:', item.id)
}
try {
await item.batch.commit()
console.log('index is finished', i, 'ID', item.id)
fulfilledIDs.push(item.id)
} catch (e) {
console.log('index ', i, 'failed', e.message)
}
}
}
- Dominant language
- TypeScript
- Stars
- 1.7k
- Forks
- 419
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 16
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from firebase/firebase-admin-node
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
firebase/firebase-admin-node#3234 ·
-
firebase/firebase-admin-node#3221 · 3 comments · 1 assignee ·
-
api: messaging
Difficulty 3/5 1-2 days Newbie friendliness 70/100
firebase/firebase-admin-node#3215 ·
-
api: messaging
Difficulty 5/5 Over a week Newbie friendliness 28/100
firebase/firebase-admin-node#3214 ·
-
api: firestore type: feature request
firebase/firebase-admin-node#3183 · 1 comment · 1 assignee ·
All issues in firebase/firebase-admin-node
Similar issues
-
bug(cli): hapi doctor inline-media prints a fabricated B:\ helper-script path in packaged installs Open
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Crush Open
Difficulty 1/5 Under an hour Newbie friendliness 85/100
catppuccin/catppuccin#3125 ·
-
Add a SECURITY.md Open
Difficulty 1/5 Under an hour Newbie friendliness 90/100
ElementsProject/cln-application#167 · 1 comment · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Quantco/pnpm-licenses#17 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100