[cloud_firestore][Android]: query snapshots() can deliver an older snapshot after a newer one since 6.7.0

Read FlutterFirebaseFirestorePlugin.java around the query listener executor and QuerySnapshotsStreamHandler.java around snapshot conversion and main-thread delivery. The issue includes a reproduction and reports that a per-listener sequential executor removes backward deliveries; inspect linked PR #18905 for work already under way. Done means query snapshots arrive in SDK order without leaving the listener stale.

Is there an existing issue for this?

• I have searched the existing issues.

Which plugins are affected?

Other (cloud_firestore)

Which platforms are affected?

Android

Description

Since 6.7.0, an Android query listener can deliver its snapshots out of order: an older snapshot can reach Dart after a newer one. If the older one arrives last, snapshots() keeps showing stale data until the query next changes, even though the local cache already holds the newer data.

Expected: each query listener delivers snapshots in the order the SDK produced them, as it did before 6.7.0.

Cause: #18390 moved query snapshot conversion off the main thread. On Android, querySnapshot passes the shared cachedThreadPool as the listener's executor (FlutterFirebaseFirestorePlugin.java L1022). The handler then converts each snapshot and posts it to the main thread (QuerySnapshotsStreamHandler.java L77-L79):

Object pigeonSnapshot =
    PigeonParser.toPigeonQuerySnapshot(querySnapshot, serverTimestampBehavior);
mainHandler.post(() -> events.success(pigeonSnapshot));

The SDK submits one executor task per snapshot. On a multi-threaded pool, two snapshots of one query can convert at the same time, and whichever finishes first is posted first.

This code is the same in 7.1.0 and on main at d07113e. From reading the code, two paths are not affected (neither was measured):

• iOS: the same PR gave each listener its own serial queue (snapshotQueue).
• Android document listeners: DocumentSnapshotsStreamHandler sets no executor, so they stay on the main thread.

Suggested fix: give each listener its own sequential executor on the same pool.

--- a/packages/cloud_firestore/cloud_firestore/android/src/main/java/io/flutter/plugins/firebase/firestore/FlutterFirebaseFirestorePlugin.java
+++ b/packages/cloud_firestore/cloud_firestore/android/src/main/java/io/flutter/plugins/firebase/firestore/FlutterFirebaseFirestorePlugin.java
@@ -17,6 +17,7 @@
 import com.google.android.gms.tasks.TaskCompletionSource;
 import com.google.android.gms.tasks.Tasks;
 import com.google.firebase.FirebaseApp;
+import com.google.firebase.concurrent.FirebaseExecutors;
 import com.google.firebase.firestore.AggregateField;
 import com.google.firebase.firestore.AggregateQuery;
 import com.google.firebase.firestore.AggregateQuerySnapshot;
@@ -1019,7 +1020,7 @@
                 PigeonParser.parsePigeonServerTimestampBehavior(
                     options.getServerTimestampBehavior()),
                 PigeonParser.parseListenSource(source),
-                cachedThreadPool)));
+                FirebaseExecutors.newSequentialExecutor(cachedThreadPool))));
   }
 
   @Override

Conversion stays off the main thread, so the benefit of #18390 is kept. Each listener has its own first-in, first-out queue, so one listener can't hold up another, and it matches the iOS queue. There are two effects beyond ordering:

• A snapshot that fails to convert is logged at SEVERE and skipped, because the sequential executor catches RuntimeException. On the raw pool the same exception is uncaught and stops the app. You may prefer to handle that case in the handler.
• If snapshots outrun conversion, the listener falls behind instead of jumping backward.

Reproducing the issue

This uses awaited REST writes only, because that is where the backward deliveries came from in the fuller test under "Additional context".

1. Create the app and add the plugins:

   flutter create --org com.example --project-name order_test --platforms=android order_test
   cd order_test
   flutter pub add firebase_core cloud_firestore:7.1.0
   

   Then replace lib/main.dart with the code below.

2. In another folder, save this as firebase.json and start the Firestore emulator:

   {"emulators":{"firestore":{"port":8765,"host":"0.0.0.0"},"ui":{"enabled":false},"singleProjectMode":true}}
   

   firebase emulators:start --only firestore --project demo-order-test
   

3. Start an Android emulator and add CPU load, which makes the problem much more frequent. adb shell pkill yes stops the load.

   emulator -avd <your_avd> -cores 4 -memory 4096
   adb shell 'i=0; while [ $i -lt 6 ]; do nohup yes > /dev/null 2>&1 < /dev/null & i=$((i+1)); done'
   

4. From the app folder, build for your emulator's ABI, run it, and read the results:

   flutter build apk --release --target-platform android-arm64
   adb install -t build/app/outputs/flutter-apk/app-release.apk
   adb logcat -c
   adb shell am start -n com.example.order_test/.MainActivity
   adb logcat -s flutter:I | grep ORDER
   

lib/main.dart:

// Does a query listener deliver an older snapshot after a newer one?
// Local Firestore emulator only. Writes are strictly ordered: each one is awaited
// before the next is sent, from outside the SDK, so they reach the listener over
// the watch stream.
import 'dart:async';
import 'dart:convert';
import 'dart:io';

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:flutter/material.dart';

const host = '10.0.2.2'; // The host machine, seen from the Android emulator.
const port = 8765;
const project = 'demo-order-test';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();
  runApp(const MaterialApp(home: Scaffold(body: Center(child: Text('order test')))));
  await Firebase.initializeApp(
    options: const FirebaseOptions(
      apiKey: 'demo',
      appId: '1:1:android:1',
      messagingSenderId: '1',
      projectId: project,
    ),
  );
  final db = FirebaseFirestore.instance;
  db.settings = const Settings(persistenceEnabled: false);
  db.useFirestoreEmulator(host, port);
  for (var round = 1; round <= 5; round++) {
    await runRound(db, round);
  }
  print('ORDER done');
}

Future<void> runRound(FirebaseFirestore db, int round) async {
  final col = db.collection('t${DateTime.now().millisecondsSinceEpoch}');

  // 20 ordinary documents, so that converting a snapshot takes a little time,
  // and the counter the listener watches.
  final batch = db.batch();
  for (var i = 0; i < 20; i++) {
    batch.set(col.doc('d$i'), {
      'text': 'x' * 400,
      'i': i,
      'at': Timestamp.fromMillisecondsSinceEpoch(1700000000000 + i),
      'nested': {'a': i, 'b': 'y' * 40, 'list': [1, 2, 3]},
    });
  }
  batch.set(col.doc('counter'), {'v': 0});
  await batch.commit();

  var high = -1, delivered = -1, events = 0, backward = 0;
  final first = Completer<void>();
  final sub = col.snapshots().listen((s) {
    final v = s.docs.firstWhere((d) => d.id == 'counter').data()['v'] as int;
    events++;
    delivered = v;
    if (v < high) {
      backward++;
    } else {
      high = v;
    }
    if (!first.isCompleted) first.complete();
  });
  await first.future;

  final http = HttpClient();
  for (var i = 1; i <= 300; i++) {
    final req = await http.patchUrl(Uri.parse(
        'http://$host:$port/v1/projects/$project/databases/(default)/documents/${col.path}/counter'));
    req.headers.set('Authorization', 'Bearer owner');
    req.headers.contentType = ContentType.json;
    req.write(jsonEncode({'fields': {'v': {'integerValue': '$i'}}}));
    await (await req.close()).drain<void>();
  }
  http.close(force: true);

  // Let delivery settle, then compare the listener's last value with the local cache.
  await Future<void>.delayed(const Duration(seconds: 3));
  final truth = (await col.doc('counter').get(const GetOptions(source: Source.cache))).data()!['v'] as int;
  print('ORDER round=$round events=$events backward=$backward delivered=$delivered truth=$truth stale=${delivered != truth}');
  await sub.cancel();
}

Results of this test on an Android emulator (API 37, arm64, 4 cores, 4 GB RAM) with Firestore emulator 1.19.8. Each build ran twice from a fresh install, one build at a time, in the order 6.6.0, 6.7.1, 7.1.0, fixed, fixed, 7.1.0, 6.7.1, 6.6.0. Unrelated builds were also running on the host (load average about 18 to 41 on 10 cores).

• Delivered backward: the snapshot's counter is lower than one already delivered, counted over all snapshots delivered.
• Ended stale: the listener's last delivery differs from the local cache, 3 seconds after the last write. In the log below, events=301 on a stale round shows that the initial snapshot and all 300 updates had arrived, so this is not a slow listener: the last snapshot to arrive was an older one.
• 6.7.0 contains the change, but it does not compile with the published firebase_core_platform_interface (Type 'FirebasePlugin' not found, as in #18438).

Firebase Core version

5.1.0

Flutter Version

3.47.8

Relevant Log Output

# 7.1.0 (logcat tag and timestamps removed)
ORDER round=1 events=301 backward=51 delivered=298 truth=300 stale=true
ORDER round=2 events=301 backward=60 delivered=300 truth=300 stale=false
ORDER round=3 events=301 backward=51 delivered=299 truth=300 stale=true
ORDER round=4 events=301 backward=46 delivered=300 truth=300 stale=false
ORDER round=5 events=301 backward=18 delivered=300 truth=300 stale=false

# 7.1.0 + the fix
ORDER round=1 events=301 backward=0 delivered=300 truth=300 stale=false
ORDER round=2 events=301 backward=0 delivered=300 truth=300 stale=false
ORDER round=3 events=301 backward=0 delivered=300 truth=300 stale=false
ORDER round=4 events=301 backward=0 delivered=300 truth=300 stale=false
ORDER round=5 events=301 backward=0 delivered=300 truth=300 stale=false

Flutter dependencies

# flutter pub deps --style=compact, Firebase packages only. Firebase Android BoM 35.1.0.
Dart SDK 3.13.7
Flutter SDK 3.47.8
order_test 1.0.0+1

dependencies:
- cloud_firestore 7.1.0
- firebase_core 5.1.0

transitive dependencies:
- _flutterfire_internals 1.3.79
- cloud_firestore_platform_interface 8.1.0
- cloud_firestore_web 5.8.0
- firebase_core_platform_interface 8.1.1
- firebase_core_web 4.1.0

Additional context and comments

Fuller test. An earlier test added awaited set() and unawaited set() to the REST writes above, on the same emulator setup and CPU load. Each run had 15 cases, five of each kind: three on 20 documents with 300 writes and two on 200 documents with 150 writes. Each build ran twice, in the order 5.6.9, 6.10.2, fixed, fixed, 6.10.2, 5.6.9.

• Ordered writes: awaited set() and awaited REST PATCH. All 195 came from the REST cases. Awaited set() showed 0 of 2,410 on all three builds.
• Unawaited bursts: these writes go through a thread pool in every version, so they can really land out of order, and the backward count means nothing here (1,297 of 2,410 on 5.6.9, 1,343 of 2,410 with the fix). Only ending different from the local cache counts.

Limits of these numbers:

• All of this ran on emulators. A physical device was not tested.
• The tests write fast, about one write every 4 ms on 20 documents, so real apps should see this less often.
• Without the CPU load it is much rarer: an earlier idle run on 6.10.2 showed 1 backward delivery in 1,204 in the REST cases.

Open PRs that touch this code:

• #18616 (Kotlin migration) carries the same cachedThreadPool argument over.
• #18601 (chunked snapshots) keeps the pool and fills its queue from pool threads, so from reading it, it narrows the window without closing it.

Different problems, for triage: #18158 (stalls in the native SDK) and #18724 (a listener cancelled while it registers).