Firestore: addMutationBatch puts the whole batch into IndexedDB once per document (N times for a WriteBatch of N)

Open Beginner friendly
#10,595 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
firebase, typescript

Research direction

Start in packages/firestore/src/local/indexeddb_mutation_queue.ts and read IndexedDbMutationQueue.addMutationBatch, focusing on how mutations are written to the mutations and documentMutations stores. The issue includes a suggested change and a fake-indexeddb comparison; verify the batch record is written once while each document still gets an index entry, including the stated empty-batch behavior.

Written by the indexing model from the issue text.

Description

api: firestore needs-attention question Repro Needed
Operating System

Windows 11 and macOS (any OS: the cost is in the SDK's IndexedDB writes)

Environment (if applicable)

Chrome 14x and headless Chromium; the same build in Electron and in a Capacitor WebView

Firebase SDK Version

13.1.0 (@firebase/firestore 4.19.0). The code below is unchanged on main.

Firebase SDK Product(s)

Firestore

Project Tooling

Next.js web app (also packaged with Electron and Capacitor), Firestore with persistentLocalCache() (IndexedDB).

Detailed Problem Description

With IndexedDB persistence on, IndexedDbMutationQueue.addMutationBatch
(packages/firestore/src/local/indexeddb_mutation_queue.ts) puts the whole serialized batch into the mutations
store once per mutation in it, because mutationStore.put(dbBatch) sits inside for (const mutation of mutations):

for (const mutation of mutations) {
  const indexKey = newDbDocumentMutationKey(this.userId, mutation.key.path, batchId);
  collectionParents = collectionParents.add(mutation.key.path.popLast());
  promises.push(mutationStore.put(dbBatch));          // same key, same value, N times
  promises.push(documentStore.put(indexKey, DbDocumentMutationPlaceholder));
}

Each put has the same key (batchId) and the same value, so the stored state is identical whether it runs once or
N times, but a WriteBatch of N documents costs N times its own serialized size in IndexedDB writes, on the main
thread, before commit() is applied locally.

Measured in our app (Chrome, a WriteBatch of 62 documents, about 1.6 MB serialized): 62 puts of the batch, 16 to
31 ms each, in one task of about 1.16 s (trace: IDBObjectStore::putRequestSetup x124, 1,156.7 ms). With the batch
put moved out of the loop, the same commit spends about 22 ms in puts, with no long task, and IndexedDB holds the same
records (compared in a test on fake-indexeddb).

Expected: the batch record is put once per batch; the per-document documentMutations entries stay one per
document.

Suggested change (keeps today's behaviour for an empty batch, where the loop never runs):

const promises: Array<PersistencePromise<void>> = [];
if (mutations.length > 0) {
  promises.push(mutationStore.put(dbBatch));
}
for (const mutation of mutations) {
  const indexKey = newDbDocumentMutationKey(this.userId, mutation.key.path, batchId);
  collectionParents = collectionParents.add(mutation.key.path.popLast());
  promises.push(documentStore.put(indexKey, DbDocumentMutationPlaceholder));
}
Steps and code to reproduce issue
import { initializeApp } from "firebase/app";
import { initializeFirestore, persistentLocalCache, writeBatch, doc } from "firebase/firestore";

const app = initializeApp(firebaseConfig);
const db = initializeFirestore(app, { localCache: persistentLocalCache() });

const batch = writeBatch(db);
const payload = "x".repeat(25_000);
for (let i = 0; i < 60; i++) batch.set(doc(db, "repro", `d${i}`), { payload });
batch.commit();
  1. Record a profile in Chrome DevTools' Performance panel around batch.commit().
  2. One long task holds 60 IDBObjectStore.put calls on the mutations store, each writing the whole batch (about
    1.5 MB here), beside the 60 small documentMutations puts. Its length grows with documents x batch size.
  3. A counter on IDBObjectStore.prototype.put filtered to the mutations store shows the same: N puts for a batch of
    N documents, where one would do.
Dominant language
TypeScript
Stars
5.2k
Forks
1k
Avg merge
2d 2h
Merged PRs (30d)
77

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from firebase/firebase-js-sdk

All issues in firebase/firebase-js-sdk

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.