[Crashlytics] ANR events of the outgoing app version drop by about 50% for 3-5 days during each new version rollout

Đang mở
  • Có gì không ổn?
#8,659 3 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
38/100
Loại issue
Lỗi
Độ rõ ràng
Cần làm rõ
Mức độ hoạt động
Sôi nổi
Công nghệ
android, kotlin
Lĩnh vực
mobile-dev, observability-sre

Hướng nghiên cứu

Start by reproducing the rollout comparison using the BigQuery realtime and batch tables and the app's ApplicationExitInfo records via getHistoricalProcessExitReasons. Then inspect the SessionReportingCoordinator change referenced in #8250; done means identifying whether collection, sampling, rate limiting, processing, or previous-session attribution explains the measured drop and documenting a verified conclusion.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

api: crashlytics needs-attention
[REQUIRED] Step 2: Describe your environment
  • Android Studio version: not relevant, production data
  • Firebase Component: Crashlytics (firebase-crashlytics, firebase-crashlytics-ndk)
  • Component version: Firebase BoM 34.17.0, Crashlytics Gradle plugin 3.0.7
[REQUIRED] Step 3: Describe the problem

When a new version of our app starts its staged rollout, the number of ANR events that Crashlytics records for the outgoing version drops by about half for 3 to 5 days. Then it returns to its usual level. The incoming version does not show the drop. We saw it on three consecutive releases. Our own count of ANR exits from ApplicationExitInfo stays flat in the same days, so the app does not have fewer ANRs.

Steps to reproduce:
  1. Ship version N to all users, then start a staged rollout of version N+1.
  2. Each day, count Crashlytics ANR events of version N in the BigQuery export (realtime and batch tables).
  3. Each day, count ApplicationExitInfo records with REASON_ANR that the app itself reads on start with getHistoricalProcessExitReasons and sends to our own telemetry.
  4. Compare the two counts.

Ratio of Crashlytics ANR events to our ApplicationExitInfo ANR records, per version and day (UTC). The rollout of the next version started on the day of the first low value.

Version Before During the rollout of the next version After
A 2.68 1.25, 1.22, 1.26 2.07
B 3.17 2.41, 2.05, 1.57 2.52
C 2.18 1.25, 1.03, 1.13 1.93

Our own ApplicationExitInfo ANR count stays at about 0.4% of app starts during the same days.

What we ruled out:

  • The drop is the same in the realtime and the batch BigQuery export.
  • It is the same for all rows and for distinct event ids, with and without a user id.
  • It is not a weekday effect: on the same day, version C fell by 43% and the version before it by 1%.
  • It is not only an attribution effect at the update. We see it also between two consecutive app starts on the same version. For restarts on version C that carry an ApplicationExitInfo ANR, the share with a matching Crashlytics ANR event between the two starts was 81%, 46% and 35% on three consecutive days.
  • Event timestamps and the time to the next start stay within seconds. Upload lag grows during the rollout, but days that are a week old still show the drop.

Questions:

  1. Is there a known behavior in collection, sampling, rate limiting or processing that lowers ANR reporting for an app version while a newer version rolls out?
  2. Can a change in how the previous session's ANR is attached on the next launch cause this? For example the SessionReportingCoordinator change in #8250.
  3. Is there a setting that keeps ANR reporting complete for older versions?

We can share the queries and the full daily table.

Relevant Code:
// No custom Crashlytics code: default ANR collection.
// Our own reference count, read on app start (API 30+):
val exits = activityManager.getHistoricalProcessExitReasons(null, 0, 5)
val last = exits.firstOrNull { it.processName == packageName }
// last.reason == ApplicationExitInfo.REASON_ANR is sent to our telemetry once per exit timestamp
Ngôn ngữ chính
Java
Star
2.6k
Fork
733
Merge trung bình
2 ngày 15 giờ
Pull request đã merge (30 ngày)
72

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của firebase/firebase-android-sdk

Tất cả issue của firebase/firebase-android-sdk

Issue tương tự

Thêm issue về Java

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.