Device view: should the agent produce JPEG frames, and with what encoder?

オープン
#455 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
25/100
issue の種類
機能追加
明瞭さ
説明が足りない
活発さ
静か
技術スタック
dart, flutter, go
領域
mobile, testing-qa

調査の方向性

Read the device view hub's probe.screenshot handling and the Dart agent's screenshot capture path, then compare the listed encoding options. The issue asks for a design decision and physical-device measurements before choosing; completion means selecting an approach based on PNG versus JPEG size, encode time, and Wi-Fi frame rate.

索引モデルが issue の本文から書いたものです。

説明

Question

The device view hub asks the agent for JPEG frames (probe.screenshot with format: "jpeg", quality), but the Dart agent can only produce PNG: dart:ui offers ImageByteFormat.png, rawRgba and rawStraightRgba, no JPEG. Today the hub therefore falls back to PNG and reports caps.jpeg as false. Should the agent gain a JPEG encoder, and if so, which kind?

What a JPEG saves (measured on the host, not on a device)

Same frames, converted with sips (macOS), at 603x1311:

Frame JPEG (q60) PNG Ratio
photographic 41 KiB 474 KiB 11.5x
UI screenshot 28 KiB 87 KiB 3.1x

Typical app screens sit between the two. Over USB or local Wi-Fi the PNG size is tolerable; it matters on a slow link and for the rate the hub can sustain. The WebKit decode spike (#448) shows the viewer side is not a bottleneck either way (JPEG decode 3-6 ms).

Options

  1. package:image (pure Dart). Current version 4.12.0 (published 2026-10-11, pub.dev), MIT licence, null-safe, supports every platform, depends on archive (^4.5.0, which depends on path and posix). The agent today depends on the Flutter SDK only, so this adds two third-party packages to every app that includes the agent, even though the agent is a dev dependency. Cost on the device: the frame must be read as raw RGBA (toByteData(rawRgba), about 3.2 MB at 603x1311) and encoded in Dart on the UI isolate or a helper isolate, which is CPU on the device being tested and can disturb the timing of the app under test. Not measured yet; it needs a device run.
  2. A native channel (a small platform plugin per platform: UIImage.jpegData on iOS, Bitmap.compress on Android). Fast and uses the platform encoder, but the agent currently ships no native plugin code, so this means a federated plugin, native build files in the user's app, and per-platform maintenance. It also breaks the "add one Dart dependency" install story.
  3. Do nothing in the agent; encode on the host. Keep PNG between agent and CLI and re-encode to JPEG in the hub (Go standard library image/jpeg). The viewer gets small frames, the device does no extra work, and nothing changes in the agent. The device-to-host link still carries the PNG, which is the part a slow link would feel.
  4. Lower the PNG cost instead: capture at the scale the viewer needs (already done by the scale parameter), which cuts bytes roughly with the pixel count.

Recommendation (to decide)

Option 3 first (no agent change, works with every released agent) plus option 4, which already exists. Revisit option 1 only if a measurement on a physical phone over Wi-Fi shows the PNG link is the limit; option 2 only if option 1's CPU cost disturbs tests. No code is proposed here.

To measure before choosing

  • PNG vs JPEG size and encode time on a physical phone at the pane's usual scale.
  • Frame rate the hub sustains over Wi-Fi with PNG at scale 0.5 and 1.0.
主要言語
Go
スター
6
フォーク
3
平均マージ
6時間 28分
マージ済み PR(30日)
362

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

AlphaWaveSystems/flutter-probe のほかの issue

AlphaWaveSystems/flutter-probe の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。