Transformer: transmuxed Opus audio of a later clipped item in a sequence keeps pre-roll from before the clip start, duplicating audio at the junction

Start with ExoAssetLoaderAudioRenderer.shouldDropInputBuffer and EncodedSampleExporter.queueInputBuffer, where the issue traces pre-roll samples with negative timeUs into the previous item's range, and with TransformerUtil.shouldTranscodeAudio, which ignores clipping when transmuxing. Reproduce with the linked input.webm and a two-item clipped sequence. Done means the transmuxed output has 300 Opus packets with strictly increasing timestamps, matching the setTransmuxAudio(false) output.

Version

Media3 pre-release (alpha, beta or RC not in this list)

More version details

1.12.0-alpha02 (the outputs in the repro repo). 1.11.1 produces the same audio packets and timestamps. Its files additionally carry the already-fixed dOps pre-skip issue (#3431), which is unrelated.

Devices that reproduce the issue

• API 37 arm64 emulator (Google Play image): the repro repo's outputs, on 1.12.0-alpha02 and 1.11.1.
• Pixel 9 (API 37) and an API 26 arm64 emulator, on 1.11.1: the same duplication at the junction, with a different 10 s VP9 + Opus WebM input.

Devices that do not reproduce the issue

None found.

Reproducible in the demo app?

Not tested

Reproduction steps

1. Generate the input, or download input.webm from https://github.com/PierfrancescoSoffritti/media3-transmux-opus-preroll-repro. It is a 6 s VP9 + Opus WebM with video keyframes at 0 s and 3 s. The audio is a rising tone (300 Hz + 100 Hz/s), so its pitch identifies the source time and a repeat is easy to hear. Built with ffmpeg 9.0.2:

ffmpeg -f lavfi -i "testsrc2=size=640x360:rate=30" \
  -f lavfi -i "aevalsrc='0.5*sin(2*PI*(300*t+50*t*t))':s=48000" -t 6 \
  -c:v libvpx-vp9 -b:v 1M -g 90 -deadline realtime -cpu-used 8 -pix_fmt yuv420p \
  -c:a libopus -b:a 64k input.webm

The file has 301 Opus packets of 20 ms each; the last one ends with discard padding. Its Cues index only the video track: there are two CuePoints, at 0 ms and 3000 ms, both on the VP9 track.

2. Export two adjacent clips of that file as one sequence, with audio transmuxed. This uses androidx.media3:media3-transformer with default settings. inputUri is a file:// URI of input.webm, outputPath is a writable path such as filesDir/output-transmuxAudio-true.mp4, and the code runs on the main looper.

fun clip(startMs: Long, endMs: Long) = EditedMediaItem.Builder(
  MediaItem.Builder()
    .setUri(inputUri)
    .setClippingConfiguration(
      MediaItem.ClippingConfiguration.Builder()
        .setStartPositionMs(startMs)
        .setEndPositionMs(endMs)
        .build()
    )
    .build()
).build()

val composition = Composition.Builder(
  EditedMediaItemSequence.withAudioAndVideoFrom(listOf(clip(0, 4_000), clip(4_000, 6_000)))
)
  .setTransmuxAudio(true)
  .build()

Transformer.Builder(context)
  .addListener(listener) // inspect the output after onCompleted
  .build()
  .start(composition, outputPath)

The export completes successfully.

3. Repeat with only .setTransmuxAudio(false) changed, writing output-transmuxAudio-false.mp4.

4. Inspect both audio tracks:

ffprobe -v error -select_streams a -count_packets -show_entries stream=nb_read_packets -of csv=p=0 output-transmuxAudio-true.mp4
ffmpeg -v error -i output-transmuxAudio-true.mp4 -map 0:a -f null -

You can also listen to both files around 4 s.

Expected result

The audio track holds source 0–6 s once: 300 Opus packets with strictly increasing timestamps, and a tone that rises continuously for 6 s.

output-transmuxAudio-false.mp4 shows exactly that. It has 300 packets, ffmpeg reports no errors, and an ffmpeg decode gives 5.994 s of audio that follows the source throughout.

Actual result

In output-transmuxAudio-true.mp4, the second clip's audio starts at the 3 s keyframe instead of at its 4 s clip start. The extra second, 50 packets, lands inside the first clip.

• Packet count: ffprobe reports 350 audio packets instead of 300.

• ffmpeg errors: ffmpeg prints 50 lines like Application provided invalid, non monotonically increasing dts to muxer in stream 0: 144048 >= 144048.

• Duplicated timestamps: 50 timestamps, from 3.001 s through 3.981 s at 20 ms spacing, each occur twice. Here is an excerpt of ffprobe pts_time around the junction:

  2.901 2.921 2.941 2.961 2.981 3.001 3.001 3.021 3.021 3.041 3.041 3.061 3.061 …
  

• Decoded audio: an ffmpeg decode of the audio gives 6.994 s for a 6.0 s file. Mapping the tone frequency back to source time, it plays source 0–4.0 s, then source 3.0–4.0 s again, then 4.0–6.0 s.

The duplicated span is the distance from the clip start back to the previous keyframe cue, 1 s in this example.

Additional observations, not covered by the repro files:

• ExoPlayer playback (1.11.1, PCM captured with a TeeAudioProcessor, video track disabled):

  • It plays the same 1 s repeat. That makes 6.99 s of audio, and the final position is 7005 ms against a duration of 6001 ms.
  • Audio after the junction plays 1 s later than its place in the timeline.
  • The workaround output plays 5.99 s.

• VLC plays the overlapping range as interleaved fragments of both copies.

• Other containers (1.11.1, same export setup):

  Input	Result
  Vorbis in WebM	Same duplication
  Opus in MKV	Same duplication
  Opus in MP4, clip start not on a packet boundary	One overlapping packet
  AAC-LC in MKV	Clean

• First item vs later items: in our tests, only items after the first show this. For a single clipped item, the pre-roll is at the start of the output and is covered by an edit list.

Analysis (1.11.1)

This is our reading of the code. The cue layout above is consistent with it.

1. The seek lands on the video keyframe. The second item is prepared at its 4.0 s clip start. The Matroska seek map is built from the Cues, and the only cue at or before 4.0 s is the 3.0 s video keyframe. So reading resumes from that cluster for all tracks.
2. Opus keeps its pre-roll. SampleQueue discards samples before the start time only when canDiscardAllSamplesToStartTime holds. That requires an audio format for which MimeTypes.allSamplesAreSyncSamples(...) is true, such as AAC-LC with a recognized codec string, MP3 or FLAC. It is false for Opus and Vorbis, which need pre-roll to decode. So the Opus samples from 3.0 s reach the asset loader.
3. The transmux path doesn't drop them. ExoAssetLoaderAudioRenderer.shouldDropInputBuffer subtracts streamStartPositionUs, which leaves these samples between −1.0 s and 0. It drops them only if (decoder != null && inputBuffer.timeUs < 0), that is, only when transcoding.
4. The item offset puts them inside the previous item. EncodedSampleExporter.queueInputBuffer adds mediaItemOffsetUs + initialTimestampOffsetUs, so these samples land at 3.0–4.0 s, on top of the first item's last second.

ExoAssetLoaderAudioRenderer.shouldDropInputBuffer and EncodedSampleExporter.queueInputBuffer are unchanged on main.

For multi-item sequences, TransformerUtil.shouldTranscodeAudio returns !composition.transmuxAudio and doesn't consider clipping. The video-side check, TransformerInternal.clippingRequiresTranscode, is applied to the sequence's first item only.

Workaround: use setTransmuxAudio(false) when any item after the first starts after 0. Verified for this composition: when the audio is decoded, the negative-timestamp buffers are dropped before decoding.

A fix could drop, or transcode, transmuxed audio that precedes the start of an item after the first. We'll leave the right trade-off to you.

Media

https://github.com/PierfrancescoSoffritti/media3-transmux-opus-preroll-repro

• input.webm: the synthetic input; the ffmpeg command in step 1 regenerates it.
• output-transmuxAudio-true.mp4: the affected output (1.12.0-alpha02).
• output-transmuxAudio-false.mp4: the workaround output (1.12.0-alpha02).

Bug Report

No response