`OffthreadVideo` is pathologically slow / hangs on sources with sparse keyframes

Open
  • Is something wrong?
#5 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
react, typescript
Domain
backend, performance

Research direction

Start from the OffthreadVideo component and the ffmpeg call that seeks and decodes each frame; the issue names no file paths, so locate them by searching for OffthreadVideo and the ffmpeg invocation. Reproduce with a sparse-keyframe source such as a 60-fps video encoded with a large GOP. The issue offers three options (probe and warn, re-mux, frame cache) with no decision, so confirm the maintainers' preferred approach before coding. Done means the chosen option either warns or renders the sparse-keyframe source without stalling.

Written by the indexing model from the issue text.

Description

Summary

OffthreadVideo performs a per-frame ffmpeg seek+decode. When the source video has a large GOP (sparse keyframes — common in screen recordings, OBS captures, and re-encodes that didn't override default x264 settings), each seek has to decode from the previous keyframe forward, which is slow and can hang the render entirely on some sources.

Reproduction

  1. Take any 60-fps screen recording (OBS, QuickTime, ffmpeg with defaults). These typically have keyframes every 250 frames (~4s).
  2. Wrap it in <OffthreadVideo src={...} /> and render a composition that plays it.
  3. Render hangs or crawls at ~1 frame/sec when the shot is on-screen. Sometimes hangs indefinitely.

In our project, an 1140-frame <OffthreadVideo> over ubatch_glm.mp4 (20s @ 60fps, ~7 keyframes total) stalled the MyVideo render at the exact frame where that video started playing. Re-encoding the source with -g 30 -keyint_min 30 (keyframe every 0.5s) — going from 7 → 41 keyframes — fully resolved it.

Repro one-liner for the fix

ffmpeg -i input.mp4 -c:v libx264 -crf 20 -preset medium -pix_fmt yuv420p \
       -g 30 -keyint_min 30 -movflags +faststart output.mp4

Expected / suggested behavior

Any one of the following would prevent users from hitting this:

  1. Probe + warn at first use. When OffthreadVideo loads a source, count keyframes vs total frames. If keyframes / total_frames < 1 / fps (i.e. less than 1 keyframe per second of content), log a warning:

    OffthreadVideo: \"video.mp4\" has only N keyframes in M frames (1 every X seconds). Per-frame seek cost will be high. Re-encode with -g <fps> -keyint_min <fps> for fast scrub.

  2. Internal re-mux on first use. Auto-generate a sidecar *.optimized.mp4 with -g 30 and use that for rendering. Cache it next to the source. (Remotion does effectively this by extracting frames to a temp directory.)

  3. Frame cache. Extract all source frames upfront to a temp directory on the first render, then read frames from the cache. Eliminates seek cost entirely after the first pass.

(3) is the cleanest but most invasive. (1) is a one-screen probe at media-load time and prevents the surprise.

Impact

This is the kind of bug where the framework looks broken but the source is at fault — and there's no diagnostic to point users at the source. The render just stalls.

Environment

  • `@rendiv/core`: ^0.1.0
  • `@rendiv/cli`: ^0.1.0
  • ffmpeg: 7.0.2-static
  • OS: Debian inside Docker container (headless)
Dominant language
TypeScript
Stars
159
Forks
32
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from thecodacus/rendiv

All issues in thecodacus/rendiv

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.