Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Spike W2: can webrtc-android hand back encoded frames?

Open
  • Is something wrong?
#338 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
1-2 days
Newbie friendliness
15/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
android, kotlin
Domain
audio-video-rtc, mobile-dev

Research direction

This is a throwaway spike, not a patch: install a pass-through VideoDecoderFactory on a real PeerConnection playing a WHEP stream, and log each EncodedImage the decoder receives (size, keyframe flag, codec, timestamps). Done means the findings record those per-frame fields, whether SPS/PPS arrive in-band or out-of-band, and whether the pipeline stalls without decoder output, ending in a written Path A verdict. Nothing merges to main.

Written by the indexing model from the issue text.

Description

help wanted need-device

What to build

The pivotal spike of the whole workstream. A throwaway app that answers one question: can webrtc-android hand back encoded frames rather than decoded ones?

PeerConnectionFactory takes a VideoDecoderFactory, and a VideoDecoder implementation is handed an EncodedImage. A decoder that forwards that buffer instead of decoding it would give access to the encoded bitstream. Whether that works in practice — whether the frames are complete, correctly ordered, carry usable timestamps and keyframe flags, and whether the pipeline tolerates a decoder that never produces output — is unverified and is what this spike settles.

Why it decides the plan
  • If it works, WHEP frames can be written into Media3 SampleQueues behind an ordinary MediaSource, Media3's own MediaCodec renderers decode them, and WHEP shares the Phase 13 seam with MoQ. Surface handling, tunneling, frame-rate matching (ADR-0014), secure decoders, AudioTrack and dropped-frame telemetry all keep working, because none of them is reimplemented.
  • If it does not, WHEP needs custom Renderers that own decode and rendering, it shares almost nothing with MoQ, and every one of those capabilities has to be rebuilt or declared absent.

That is a difference of several weeks and of how much of the existing TV and telemetry work survives contact with the new path. It is worth a day before anything is committed to.

Note on the shape ExoPlayer actually requires

Worth confirming in passing, because it bears on ADR-0018 either way: ExoPlayer will not run a Renderer that is not attached to a MediaPeriod. Renderer.enable() is handed a SampleStream, renderers are enabled only for tracks selected from a MediaPeriod's TrackGroupArray, and STATE_READY is a function of MediaPeriod.getBufferedPositionUs(). So a custom MediaSource/MediaPeriod pair is needed on both paths, and the fork is only over what flows through the SampleStream. Media3's own RTSP module is the precedent for the push-based case: RTP packets are depacketised into SampleQueues behind an ordinary MediaSource.

Acceptance criteria

  • A pass-through VideoDecoderFactory is installed on a real PeerConnection playing a real WHEP stream.
  • The finding records, for each received frame: buffer size, keyframe flag, codec, and the timestamp fields available — or records that the callback is never reached, with the failure.
  • The finding says whether frames arrive complete and in decode order, and whether parameter sets (SPS/PPS) are present in-band, out-of-band, or not at all.
  • The finding states whether the pipeline tolerates a decoder that produces no output, or whether WebRTC's machinery stalls or tears the track down.
  • A verdict is written: Path A (encoded into SampleQueue) is available for WHEP, or it is not, with the evidence.
  • Nothing merges to main.

Blocked by

None (can start immediately).

Dominant language
Kotlin
Stars
1
Forks
0
Avg merge
1h 20m
Merged PRs (30d)
172

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ramesh130/superplayer

All issues in ramesh130/superplayer

Similar issues

More Kotlin issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.