[Crashlytics] ANR events of the outgoing app version drop by about 50% for 3-5 days during each new version rollout

オープン
  • 何か問題がありますか？
#8,659 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
38/100
issue の種類
バグ
明瞭さ
説明が足りない
活発さ
活発
技術スタック
android, kotlin
領域
mobile-dev, observability-sre

調査の方向性

Start by reproducing the rollout comparison using the BigQuery realtime and batch tables and the app's ApplicationExitInfo records via getHistoricalProcessExitReasons. Then inspect the SessionReportingCoordinator change referenced in #8250; done means identifying whether collection, sampling, rate limiting, processing, or previous-session attribution explains the measured drop and documenting a verified conclusion.

索引モデルが issue の本文から書いたものです。

説明

api: crashlytics needs-attention
[REQUIRED] Step 2: Describe your environment
  • Android Studio version: not relevant, production data
  • Firebase Component: Crashlytics (firebase-crashlytics, firebase-crashlytics-ndk)
  • Component version: Firebase BoM 34.17.0, Crashlytics Gradle plugin 3.0.7
[REQUIRED] Step 3: Describe the problem

When a new version of our app starts its staged rollout, the number of ANR events that Crashlytics records for the outgoing version drops by about half for 3 to 5 days. Then it returns to its usual level. The incoming version does not show the drop. We saw it on three consecutive releases. Our own count of ANR exits from ApplicationExitInfo stays flat in the same days, so the app does not have fewer ANRs.

Steps to reproduce:
  1. Ship version N to all users, then start a staged rollout of version N+1.
  2. Each day, count Crashlytics ANR events of version N in the BigQuery export (realtime and batch tables).
  3. Each day, count ApplicationExitInfo records with REASON_ANR that the app itself reads on start with getHistoricalProcessExitReasons and sends to our own telemetry.
  4. Compare the two counts.

Ratio of Crashlytics ANR events to our ApplicationExitInfo ANR records, per version and day (UTC). The rollout of the next version started on the day of the first low value.

Version Before During the rollout of the next version After
A 2.68 1.25, 1.22, 1.26 2.07
B 3.17 2.41, 2.05, 1.57 2.52
C 2.18 1.25, 1.03, 1.13 1.93

Our own ApplicationExitInfo ANR count stays at about 0.4% of app starts during the same days.

What we ruled out:

  • The drop is the same in the realtime and the batch BigQuery export.
  • It is the same for all rows and for distinct event ids, with and without a user id.
  • It is not a weekday effect: on the same day, version C fell by 43% and the version before it by 1%.
  • It is not only an attribution effect at the update. We see it also between two consecutive app starts on the same version. For restarts on version C that carry an ApplicationExitInfo ANR, the share with a matching Crashlytics ANR event between the two starts was 81%, 46% and 35% on three consecutive days.
  • Event timestamps and the time to the next start stay within seconds. Upload lag grows during the rollout, but days that are a week old still show the drop.

Questions:

  1. Is there a known behavior in collection, sampling, rate limiting or processing that lowers ANR reporting for an app version while a newer version rolls out?
  2. Can a change in how the previous session's ANR is attached on the next launch cause this? For example the SessionReportingCoordinator change in #8250.
  3. Is there a setting that keeps ANR reporting complete for older versions?

We can share the queries and the full daily table.

Relevant Code:
// No custom Crashlytics code: default ANR collection.
// Our own reference count, read on app start (API 30+):
val exits = activityManager.getHistoricalProcessExitReasons(null, 0, 5)
val last = exits.firstOrNull { it.processName == packageName }
// last.reason == ApplicationExitInfo.REASON_ANR is sent to our telemetry once per exit timestamp
主要言語
Java
スター
2.6k
フォーク
733
平均マージ
2日 15時間
マージ済み PR（30日）
72

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

firebase/firebase-android-sdk のほかの issue

firebase/firebase-android-sdk の issue をすべて見る

似ている issue

Java の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。