Sentry release smoke-test + scripted API tripwire

オープン
  • 何か問題がありますか？
#69 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
38/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
静か
技術スタック
github-actions, javascript, kotlin
領域
ci-cd, documentation, mobile-dev, observability-sre, testing-qa

調査の方向性

まず backend/lib/sentry-frame.test.mjs、backend/lib/sentry.test.mjs、android/src/test/java/com/comapeo/core/SentryFgsBridgeTest.kt にある既存のテストから始め、その後、リリース runbook と現在の CI workflow を確認します。チェックリストは docs/ に、tripwire は scripts/sentry-tripwire.mjs または同様のパスに置き、手動ディスパッチの workflow で build に対して実行する必要があります。1 つのリリースで smoke test の承認が得られ、受け入れ項目が完了していれば完了です。

索引モデルが issue の本文から書いたものです。

説明

priority: P2 sentry testing

Background

Recent Sentry-related regressions (159a997, b576a15, b975101) shared a failure mode that unit tests don't catch well: the JS / Node / native layer compiled and ran fine, but spans either didn't reach Sentry or arrived with wrong structure. The only signal was opening the Sentry UI after a build.

Three small unit tests now cover the cheapest cases (see backend/lib/sentry-frame.test.mjs, backend/lib/sentry.test.mjs, android/src/test/java/com/comapeo/core/SentryFgsBridgeTest.kt). They catch the logic regressions — what a processor does when it runs, whether the Node layer produces any envelope, whether envelopeToFrame stamps length correctly.

What they don't catch:

  • "Production init forgot to install processor X" (wiring, not logic)
  • "JS layer was the wrong shape so the SDK never sent" (cross-SDK behavior)
  • "Native received the frame but the wire send was dropped" (offline/queue/transport)
  • Any cross-process trace stitching regression

These need to be caught either by manual smoke-test or by a real-Sentry tripwire. This issue tracks both.

Part 1 — Release smoke-test checklist

Run before every release. Build the example app with diagnostics on, point at a dedicated test DSN, then verify each item in Sentry's UI within ~10 minutes.

Setup

  • Example app built with the Expo Sentry plugin configured for the test project DSN
  • Diagnostics toggle on; captureApplicationData on
  • Known test trace label (e.g. comapeo-test-YYYY-MM-DD) set as a global tag so the run is findable

Trigger flows

  • Cold-launch the app — boot transaction comapeo.boot should appear with boot.fgs-launch, boot.extract-assets, boot.node-spawn, boot.rootkey-load, boot.loader-init, boot.manager-init as child spans
  • Perform one RPC call (e.g. open a project) — should produce a transaction with rpc.client span parented onto a rpc.server span (cross-process trace stitching)
  • Force a fatal error (test-only menu or error-native injection) — should appear as an exception event

Verify in Sentry UI

  • All three traces appear in Performance / Issues
  • contexts.device.family == "Android" on FGS-emitted spans (not "Google")
  • On iOS: no duplicate native-init signal in breadcrumbs
  • Scope tags present: proc (main / fgs), layer (rn / node / native), comapeo.rn (version label)
  • No PII strings in event JSON: rootkey base64, project IDs, lat/lng
  • release matches the build's release name; environment matches expected

Part 2 — Scripted Sentry API tripwire (release-blocking)

A one-shot script that drives the example app through a known trace, polls Sentry's API, and asserts the trace arrived with the expected span structure. Replaces eyeballing for the "did anything actually arrive on the wire" question.

Shape

  • Inputs: test DSN, Sentry auth token (read-only event:read), org/project slug
  • Build & launch the example app in CI or on a connected device
  • App triggers a known transaction with a hard-coded trace_id (env-var injected at boot)
  • Script polls GET /api/0/projects/{org}/{project}/events/?query=trace:<id> for up to 60s
  • Assert: at least one transaction event matches; op == comapeo.boot; expected child spans present; contexts.device.family == "Android" on the FGS-process span; no PII

Run cadence

  • Pre-release CI workflow (manual trigger or branch-gated)
  • ~5 minutes; doesn't need to run per-PR

Why not always-on per-PR

  • Sentry ingestion latency makes it flaky as a blocking check
  • Quota cost is non-trivial across many PRs
  • Pre-release cadence catches the regression class without burning quota

Open questions

  • Use a dedicated comapeo-rn-test Sentry project (separate from the real one) so test traffic doesn't pollute dashboards / rate limits
  • Decide whether the script runs against a simulator/emulator (cheap, less realistic) or BrowserStack (real device, slower)
  • Auth token storage: GitHub secrets vs. EAS environment secrets

Acceptance

  • Smoke checklist lives in docs/ (e.g. docs/sentry-release-smoke.md) and is linked from the release runbook
  • Tripwire script in scripts/sentry-tripwire.mjs (or similar) with a README block on usage
  • CI workflow (manual-dispatch) that runs the tripwire against a build
  • Smoke checklist run + signed off on at least one release before considering this issue done
主要言語
Kotlin
スター
1
フォーク
0
平均マージ
1日 6時間
マージ済み PR（30日）
5

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

digidem/comapeo-core-react-native のほかの issue

digidem/comapeo-core-react-native の issue をすべて見る

似ている issue

Kotlin の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。