Firestore: addMutationBatch puts the whole batch into IndexedDB once per document (N times for a WriteBatch of N)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- firebase, typescript
- Domain
- database, performance
Research direction
Start in packages/firestore/src/local/indexeddb_mutation_queue.ts and read IndexedDbMutationQueue.addMutationBatch, focusing on how mutations are written to the mutations and documentMutations stores. The issue includes a suggested change and a fake-indexeddb comparison; verify the batch record is written once while each document still gets an index entry, including the stated empty-batch behavior.
Written by the indexing model from the issue text.
Description
Operating System
Windows 11 and macOS (any OS: the cost is in the SDK's IndexedDB writes)
Environment (if applicable)
Chrome 14x and headless Chromium; the same build in Electron and in a Capacitor WebView
Firebase SDK Version
13.1.0 (@firebase/firestore 4.19.0). The code below is unchanged on main.
Firebase SDK Product(s)
Firestore
Project Tooling
Next.js web app (also packaged with Electron and Capacitor), Firestore with persistentLocalCache() (IndexedDB).
Detailed Problem Description
With IndexedDB persistence on, IndexedDbMutationQueue.addMutationBatch
(packages/firestore/src/local/indexeddb_mutation_queue.ts) puts the whole serialized batch into the mutations
store once per mutation in it, because mutationStore.put(dbBatch) sits inside for (const mutation of mutations):
for (const mutation of mutations) {
const indexKey = newDbDocumentMutationKey(this.userId, mutation.key.path, batchId);
collectionParents = collectionParents.add(mutation.key.path.popLast());
promises.push(mutationStore.put(dbBatch)); // same key, same value, N times
promises.push(documentStore.put(indexKey, DbDocumentMutationPlaceholder));
}
Each put has the same key (batchId) and the same value, so the stored state is identical whether it runs once or
N times, but a WriteBatch of N documents costs N times its own serialized size in IndexedDB writes, on the main
thread, before commit() is applied locally.
Measured in our app (Chrome, a WriteBatch of 62 documents, about 1.6 MB serialized): 62 puts of the batch, 16 to
31 ms each, in one task of about 1.16 s (trace: IDBObjectStore::putRequestSetup x124, 1,156.7 ms). With the batch
put moved out of the loop, the same commit spends about 22 ms in puts, with no long task, and IndexedDB holds the same
records (compared in a test on fake-indexeddb).
Expected: the batch record is put once per batch; the per-document documentMutations entries stay one per
document.
Suggested change (keeps today's behaviour for an empty batch, where the loop never runs):
const promises: Array<PersistencePromise<void>> = [];
if (mutations.length > 0) {
promises.push(mutationStore.put(dbBatch));
}
for (const mutation of mutations) {
const indexKey = newDbDocumentMutationKey(this.userId, mutation.key.path, batchId);
collectionParents = collectionParents.add(mutation.key.path.popLast());
promises.push(documentStore.put(indexKey, DbDocumentMutationPlaceholder));
}
Steps and code to reproduce issue
import { initializeApp } from "firebase/app";
import { initializeFirestore, persistentLocalCache, writeBatch, doc } from "firebase/firestore";
const app = initializeApp(firebaseConfig);
const db = initializeFirestore(app, { localCache: persistentLocalCache() });
const batch = writeBatch(db);
const payload = "x".repeat(25_000);
for (let i = 0; i < 60; i++) batch.set(doc(db, "repro", `d${i}`), { payload });
batch.commit();
- Record a profile in Chrome DevTools' Performance panel around
batch.commit(). - One long task holds 60
IDBObjectStore.putcalls on themutationsstore, each writing the whole batch (about
1.5 MB here), beside the 60 smalldocumentMutationsputs. Its length grows with documents x batch size. - A counter on
IDBObjectStore.prototype.putfiltered to themutationsstore shows the same: N puts for a batch of
N documents, where one would do.
- Dominant language
- TypeScript
- Stars
- 5.2k
- Forks
- 1k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 77
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from firebase/firebase-js-sdk
-
AI: ChatSession.sendMessage / sendMessageStream throw TypeError on blocked prompts (empty candidates) when tools are setPossibly taken @abhiramshaji-cyber claimed this today. Openapi: firebaseai
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
firebase/firebase-js-sdk#10641 ·
Maintainers usually reply within 1 day
-
api: installations api: performance bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
firebase/firebase-js-sdk#10526 · 3 comments ·
Maintainers usually reply within 1 day
-
@firebase/firestore pins @grpc/grpc-js to ~1.9.0, holding consumers on a flagged releasePossibly taken @cherylEnkidu claimed this 50 days ago. Openapi: firestore needs-attention question stack:Angular
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
firebase/firebase-js-sdk#10400 · 4 comments · 9 reactions · 1 assignee ·
Maintainers usually reply within 1 day
-
Some (sub) package.json do not list the licences which lead to tools like syft report unknown licencePossibly taken @phoekerson claimed this 84 days ago. Openquestion
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
firebase/firebase-js-sdk#9326 · 5 comments ·
Maintainers usually reply within 1 day
-
AI: ONLY_ON_DEVICE falls back to the cloud when the LanguageModel API is missingPossibly taken @hsubox76 claimed this 2 days ago. Openapi: firebaseai
Difficulty 3/5 1-2 days Newbie friendliness 35/100
firebase/firebase-js-sdk#10635 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
All issues in firebase/firebase-js-sdk
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
inabagumi/shinju-date#7406 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
umami-software/umami#4787 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day