Device view: should the agent produce JPEG frames, and with what encoder?

Abierto
#455 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Necesita aclaración
Estado de actividad
Tranquilo
Stack tecnológico
dart, flutter, go

Línea de trabajo

Read the device view hub's probe.screenshot handling and the Dart agent's screenshot capture path, then compare the listed encoding options. The issue asks for a design decision and physical-device measurements before choosing; completion means selecting an approach based on PNG versus JPEG size, encode time, and Wi-Fi frame rate.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Question

The device view hub asks the agent for JPEG frames (probe.screenshot with format: "jpeg", quality), but the Dart agent can only produce PNG: dart:ui offers ImageByteFormat.png, rawRgba and rawStraightRgba, no JPEG. Today the hub therefore falls back to PNG and reports caps.jpeg as false. Should the agent gain a JPEG encoder, and if so, which kind?

What a JPEG saves (measured on the host, not on a device)

Same frames, converted with sips (macOS), at 603x1311:

Frame JPEG (q60) PNG Ratio
photographic 41 KiB 474 KiB 11.5x
UI screenshot 28 KiB 87 KiB 3.1x

Typical app screens sit between the two. Over USB or local Wi-Fi the PNG size is tolerable; it matters on a slow link and for the rate the hub can sustain. The WebKit decode spike (#448) shows the viewer side is not a bottleneck either way (JPEG decode 3-6 ms).

Options

  1. package:image (pure Dart). Current version 4.12.0 (published 2026-10-11, pub.dev), MIT licence, null-safe, supports every platform, depends on archive (^4.5.0, which depends on path and posix). The agent today depends on the Flutter SDK only, so this adds two third-party packages to every app that includes the agent, even though the agent is a dev dependency. Cost on the device: the frame must be read as raw RGBA (toByteData(rawRgba), about 3.2 MB at 603x1311) and encoded in Dart on the UI isolate or a helper isolate, which is CPU on the device being tested and can disturb the timing of the app under test. Not measured yet; it needs a device run.
  2. A native channel (a small platform plugin per platform: UIImage.jpegData on iOS, Bitmap.compress on Android). Fast and uses the platform encoder, but the agent currently ships no native plugin code, so this means a federated plugin, native build files in the user's app, and per-platform maintenance. It also breaks the "add one Dart dependency" install story.
  3. Do nothing in the agent; encode on the host. Keep PNG between agent and CLI and re-encode to JPEG in the hub (Go standard library image/jpeg). The viewer gets small frames, the device does no extra work, and nothing changes in the agent. The device-to-host link still carries the PNG, which is the part a slow link would feel.
  4. Lower the PNG cost instead: capture at the scale the viewer needs (already done by the scale parameter), which cuts bytes roughly with the pixel count.

Recommendation (to decide)

Option 3 first (no agent change, works with every released agent) plus option 4, which already exists. Revisit option 1 only if a measurement on a physical phone over Wi-Fi shows the PNG link is the limit; option 2 only if option 1's CPU cost disturbs tests. No code is proposed here.

To measure before choosing

  • PNG vs JPEG size and encode time on a physical phone at the pane's usual scale.
  • Frame rate the hub sustains over Wi-Fi with PNG at scale 0.5 and 1.0.
Lenguaje dominante
Go
Estrellas
6
Forks
3
Merge medio
6 h 28 min
PR fusionados (30 d)
362

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de AlphaWaveSystems/flutter-probe

Todos los issues de AlphaWaveSystems/flutter-probe

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.