Drive the benchmark's device arm through devicelab
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 22/100
Research direction
Start in devicelab/lab, where DEMO_PACKAGE, DEMO_ACTIVITY, build_demo, install_demo and wait_for_playback are hardwired to the demo; parameterise these per scenario, keeping the demo's values as defaults. Then add devicelab/scenarios/benchmark.sh and cover the device-free parts in devicelab/test/selftest. Done when devicelab/lab run benchmark yields peak RSS and battery per arm and the baseline's device table has numbers.
Written by the indexing model from the issue text.
Description
Problem
benchmark/ (#43) reports every metric of PRD.md §6's matrix except the two that need a device:
peak RSS and battery delta over a 30-minute session. Both are properties of a process on a
device with a battery, and there is no honest way to take either from a JVM — so the committed
baseline prints them as dashes, and says so rather than leaving the metrics out.
The app half exists. BenchmarkActivity takes an arm, one of the public streams in
PublicStreams.kt, and a duration; plays a 30-minute session on a real device over a real network;
writes its telemetry as a JSONL trace in the same format the Robolectric arm uses; and logs one line
under the tag SuperPlayerBench when it is done. It deliberately measures neither number itself — an
app measuring its own memory would be measuring the measurement.
What is missing is the harness around it.
Why it was not done with #43
devicelab/ is this repository's one entry point for measuring on a device, and its README already
names the benchmark as a consumer. But lab is currently shaped around the demo throughout:
DEMO_PACKAGEandDEMO_ACTIVITYare fixed globalsbuild_demobuildsdemo/and knows its APK pathinstall_demoverifies a stale-artifact fingerprint the demo's build writes into its own assetswait_for_playbackreadsdumpsys media_session, and the benchmark app publishes no session
Parameterising all of that is a real change to a harness whose whole value is that it is reliable,
and it cannot be verified without a device. Shipping it half-done inside #43 would have put
untested shell into the one thing that is supposed to be dependable.
What to build
- A scenario-declared app in
devicelab/lab: package, activity, build directory and APK name, with
the demo's values as the defaults, so no existing scenario changes. - A readiness probe a scenario can override. The benchmark app's is its own
SuperPlayerBench
logcat line, not a media session. - The stale-artifact guard for the benchmark's APK — the equivalent of the demo's
RecordSuperPlayerArtifacts, so a device run refuses to measure an APK built against a stale
publish, exactly as the demo's does. devicelab/scenarios/benchmark.sh, running each arm in turn with theprocess_memoryand
batterydata sources, and ascenario_reportthat puts peak RSS and battery beside each arm.- Coverage in
devicelab/test/selftest, which is device-free and incheck.
Done when
devicelab/lab run benchmarkproduces a report with peak RSS and battery per arm.benchmark/baseline/'s device-arm table has numbers in it instead of dashes, with the device
recorded alongside them the wayrun.jsonrecords one.benchmark/README.mdanddevicelab/README.mdstop saying this is not built.
- Dominant language
- Kotlin
- Stars
- 1
- Forks
- 0
- Avg merge
- 1h 12m
- Merged PRs (30d)
- 195
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ramesh130/superplayer
-
bug need-device
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
ramesh130/superplayer#495 ·
Maintainers usually reply within 1 day
-
Flakes (Android): AudioCapabilityChangeTest reselect wait and DownloadConditionsTest metered stopOpenready-for-agent
Difficulty 4/5 1-2 days Newbie friendliness 28/100
ramesh130/superplayer#526 ·
Maintainers usually reply within 1 day
-
ios ready-for-agent
Difficulty 4/5 3-5 days Newbie friendliness 28/100
ramesh130/superplayer#518 ·
Maintainers usually reply within 1 day
-
ready-for-agent
Difficulty 4/5 1-2 days Newbie friendliness 58/100
ramesh130/superplayer#501 ·
Maintainers usually reply within 1 day
-
ready-for-agent
Difficulty 4/5 3-5 days Newbie friendliness 48/100
ramesh130/superplayer#500 ·
Maintainers usually reply within 1 day
All issues in ramesh130/superplayer
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
rustyrazorblade/easy-db-lab#1017 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
callstack/react-native-brownfield#492 · 1 comment ·
Maintainers usually reply within 4 days
-
fix: retain the best partial noise-correction solutionPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
world-federation-of-advertisers/cross-media-measurement#4653 ·
Maintainers usually reply within 1 day