Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Spec: Phase 10 — Tuning: move the benchmark's numbers, after the functional phases

Open
  • Is something wrong?
#151 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
12/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Active
Tech stack
android, kotlin
Domain
mobile, performance

Research direction

Start by reading PRD.md Part 4 and the committed Phase 3 report in benchmark/phase3/, then ExitCriterion and qoe-floors.tsv to see the fixed rule and floors. The issue asks for a Phase 10 entry in PRD.md Part 4 that owns the numeric criteria, not for any policy change. Done means that entry exists with the exit criteria as written, and the open tuning work is left to later issues.

Written by the indexing model from the issue text.

Description

Problem Statement

PRD.md Part 4 used to end Phase 3 on measured improvement over the Phase 1 baseline on the shaped-network suite, with no regression on stable WiFi, and Phase 4 on a p50 TTFF target. Both are criteria that a number moved, not that behaviour exists. The committed Phase 3 report (benchmark/phase3/, PR #150) shows why that matters. The adaptive arm is built, runs behind the boundary, and is graded on all six networks. ExitCriterion still judges it not met on all six:

Network Worse in (adaptive against the static SuperPlayer profile)
stable WiFi VOD switch count and QoE score; short-form rebuffer ratio, rebuffer count (4 → 22) and QoE score
congested WiFi VOD, live and short-form QoE score
LTE with dropouts VOD, live, short-form and data-saver QoE score
3G VOD, live and short-form QoE score (live time to first frame 800 → 4050 ms)
WiFi→cellular VOD, live, short-form and data-saver QoE score (live rebuffer ratio 0.004 → 0.046)
high latency live QoE score (1 → 22 rebuffers, 4.35 → 2.66) and short-form QoE score; VOD is better

Closing that gap is iterative, and nobody can know in advance how much it will take. A policy change that fixes one cell costs another, and every attempt is a full matrix run. Holding the functional phases behind it would stall the roadmap on tuning.

Solution

Phase 10, added after the functional phases in PRD.md Part 4 (branch tuning-phase-roadmap), owns every "a number moved" criterion. Phases 3 and 4 now end on behaviour that is built, tested and measured.

Measurement stays where it is. The harness, the report and the QoE regression gate remain in the phases that build them. Phase 10 starts from committed reports rather than from nothing. A functional phase's report is a finding for this phase, never a reason to retune before committing it.

The rules do not loosen. ExitCriterion's rule was fixed before the matrix ran, and it stays fixed: two standard errors, a QoE win and no QoE loss on each shaped profile, nothing worse on stable WiFi. PRD.md §6's honesty rules also stand. F1's trade prints as a bitrate loss. A floor in qoe-floors.tsv moves only in a commit that says why.

Scope

Issue What it lands
Adaptive policy against the static profile Iterations on AdaptiveBufferPolicy, AdaptiveSelectionPolicy, SelectionPaces and TransportCaps until the Phase 3 criterion is met. Start with the stable WiFi short-form rebuffers, which is a loss on the one profile where adaptive has nothing to gain. The same 22-rebuffer pattern shows on high-latency live.
#132 A WiFi→cellular handover whose cellular stretch cannot carry the top rung, so the profile can separate a good transport reaction from a bad one
Feed p50 TTFF target Set and argue a target for the feed demo (§6 names none yet), then meet it. Comes after Phase 4's preload and cache.
#95 The benchmark's device arm through devicelab: peak RSS and battery delta over a 30-minute session

The issues in the Scope table without a number are filed when work on them starts.

Exit criteria

Per PRD.md Part 4, Phase 10:

  • The adaptive policy shows a measured improvement over the static profile on the shaped-network suite, with no regression on stable WiFi, by ExitCriterion's rule unchanged. The report that shows it is committed as the reference.
  • p50 TTFF in the feed demo is under a target this phase sets and argues.
  • Peak RSS and battery delta are reported from a device.

Non-goals

  • Loosening the rule to pass it. The two-standard-error rule, the reference arm and "unmeasured is not met" are fixed. A change to any of them is argued on its own, not folded into a tuning iteration.
  • Blocking earlier phases. Phases 4–9 proceed on their functional criteria. Work they turn up that is about a number goes here.
  • Tuning the static profiles to make the comparison easier. A change to StaticProfilePolicy shows in the report as a change to arm (c), with its reason.
Dominant language
Kotlin
Stars
1
Forks
0
Avg merge
1h 20m
Merged PRs (30d)
172

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ramesh130/superplayer

All issues in ramesh130/superplayer

Similar issues

More Kotlin issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.