The joining backfill costs a base-layer GOP, on exactly the links that cannot spare it

Open
#6 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Quiet
Tech stack
go

Research direction

Start in internal/conf/remote.go at backfillGroup, SUBGROUP_FILTER, the priority constants, videoBackfillTimeout, resyncCooldown, and checkLag. Run or inspect degraded_test.go and the existing inbound-track telemetry, then measure backfill bytes, wall-clock time, resync behavior, and timeout frequency per participant. Done means the measurements support a documented transport or backfill decision.

Written by the indexing model from the issue text.

Description

enhancement

Deferred from #4. Not urgent — the links this is running on today are fine — but
the shape of the problem is worth writing down while it is fresh, because the
fix is a design choice rather than a repair, and the choice is coupled to the
object-ID inversion in a way that is not obvious later.

What it costs

backfillGroup (internal/conf/remote.go) replays the group in progress with a
Joining FETCH, narrowed to the base layer by SUBGROUP_FILTER. A subscriber
joining at a random point in a five-second GOP pulls, on average, ~2.5 s of
base-layer video — order of a couple of hundred kilobytes at the 1.5 Mbps the
ladder targets.

That is cheap on a healthy link and it is the whole reason a joining tile paints
at all. On a constrained one it is neither.

Why the constrained case is worse than it first looks

It fires on every resubscribe, not just on join. Layer change, demotion,
tile scrolled back into view — and, most importantly, the lag resync: past
maxLag (1.5 s) the subscription is rebuilt at the live edge, rate limited by
resyncCooldown to once every 15 s. So a link that is already failing to hold
the live edge answers that by spending a fresh base-layer GOP, every fifteen
seconds, on top of the live media it was already failing to carry. The response
to "we are behind" is more bytes.

On a slow enough link it pays the cost and gets nothing.
videoBackfillTimeout bounds how long live video waits behind the backfill at
3 s. A base-layer GOP does not cross a 32 kB/s link in three seconds — the
bottleneck the degraded_test.go shapers model — so the wait expires, live is
released, and the bytes already spent were spent for a picture that never
arrived. Worst of both.

Priority cannot arbitrate it. The obvious answer is to rank the backfill
below live audio and video with SUBSCRIBER_PRIORITY, and it would be correct
protocol behaviour — but see the priority constants at the top of remote.go:
moq-go's adapters implement no per-stream scheduling knob, because quic-go
exposes none, so §10.2.7 changes nothing measurable on either transport this app
can use. Stamping a lower priority would be honest and inert.

The fork

The two ends of it are not a matter of tuning; they want different transports.

Full-GOP backfill (what ships). The subscriber gets the whole reference
chain, so the live continuation of that group is decodable and the picture runs
continuously from the moment it paints. This is what the object-ID inversion in
#4 buys — base layer on the top range so a largest-object subscription keeps
receiving it mid-group.

Keyframe-only backfill. Fetch object zero of the base range and nothing
else: one object, near enough free. The tile paints a still immediately and
motion resumes at the next group, which requestNewGroup has already asked for,
so roughly one RTT away on a healthy link. But the frames between the keyframe
and the live edge are then missing, so the live continuation references frames
the decoder never got — which means it must be discarded, and then the
object-ID inversion buys nothing at all
. Reverting it would be a second wire
break.

That coupling is the thing to remember: picking keyframe-only later is not a
narrowing of the current design, it is undoing part of it.

What would decide it

Nothing here should be chosen from reasoning; it wants numbers.

  • What a joining backfill actually costs in bytes and in wall-clock, measured
    per participant on a real call rather than derived from the bitrate target.
  • Whether the resync loop above is observable — does a struggling link visibly
    spend its recovery on backfills? The telemetry counters already separate
    inbound tracks, so a backfill's bytes should be attributable.
  • Whether the 3 s timeout is ever reached in practice, which is the sharpest
    single signal: a backfill that times out is pure waste, and if it never times
    out the constrained case is theoretical.

Middle options, if the numbers say so

  • Skip the backfill and lean on NEW_GROUP_REQUEST alone when the drift meter
    says the path is already behind — the information is local and already
    measured (checkLag).
  • Bound the backfill by age rather than by group: fetch the tail of the base
    layer plus the keyframe. Needs the reference chain to be complete, so it only
    works if the encoder's GOP has no long-term references beyond the keyframe,
    which for the current L1T2/H.264 configuration it does not — worth checking
    rather than assuming.
  • Keep the full backfill but make the timeout adaptive, so a link that cannot
    finish one stops starting them.

Not blocking

Filed as a known trade rather than a defect. The behaviour on a healthy link is
the intended one, and the constrained path degrades to what 0.7.2 did — a blank
tile until the next keyframe — rather than to anything worse.

Dominant language
Go
Stars
1
Forks
1
Avg merge
7m
Merged PRs (30d)
4

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.