`encode_video` truncates a fractional `fps`, so the video runs longer than its audio

Open Beginner friendly
  • Is something wrong?
#15,104 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@guptaishaan is already working on this.

Since Oct 22, 2026.

  • #15105 by @guptaishaan — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
83/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, pytorch
Domain
backend

Research direction

Start with src/diffusers/utils/export_utils.py at the encode_video stream setup; the issue identifies rate=int(fps) as the cause and points to the LTX-2 HDR exporter for a fractional-rate example. Add the proposed regression test in tests/others/test_utils.py, encoding 30 frames at 29.97 fps. Done when the encoded stream reports average_rate == Fraction(2997, 100).

Written by the indexing model from the issue text.

Description

Describe the bug

diffusers.utils.export_utils.encode_video opens the video stream with rate=int(fps)
(export_utils.py#L364),
so 29.97 is written as 29 fps and 23.976 as 23. The audio stream keeps its real sample rate, so the video ends up
longer than the audio and drifts further out of sync the longer the clip is: about 0.35 s over 10 s at 29.97.

The LTX-2 pipelines take frame_rate as a float and the docs pass it straight through as encode_video(..., fps=frame_rate).
The LTX-2 HDR exporter already keeps the fraction with Fraction(frame_rate).limit_denominator(1000)
(ltx2/export_utils.py#L74).

Reproduction

I can't run the LTX-2 checkpoint on my machine, so this calls encode_video the way the LTX-2 docs do, with inputs
shaped like the pipeline's output_type="np" video and stereo audio:

import av
import numpy as np
import torch

from diffusers.utils.export_utils import encode_video

for fps in (24.0, 29.97, 23.976):
    num_frames = round(fps * 10)  # 10 seconds
    video = np.random.rand(num_frames, 64, 64, 3).astype(np.float32)
    audio = torch.zeros(2, 48000 * 10)
    encode_video(video, fps=fps, output_path="out.mp4", audio=audio, audio_sample_rate=48000)

    with av.open("out.mp4") as container:
        v, a = container.streams.video[0], container.streams.audio[0]
        print(fps, v.average_rate, f"video {float(v.duration * v.time_base):.3f}s", f"audio {float(a.duration * a.time_base):.3f}s")
Logs
24.0 24/1 video 10.000s audio 10.000s
29.97 29/1 video 10.345s audio 10.000s
23.976 23/1 video 10.435s audio 10.000s
System Info
  • diffusers main (29d199aba)
  • torch 2.14.1+cpu, PyAV 19.0.2, Python 3.12.13, Windows 11
Proposed fix

Use the same expression as the LTX-2 HDR exporter, rate=Fraction(fps).limit_denominator(1000) (Fraction is already imported in that file), plus a test in tests/others/test_utils.py that encodes 30 frames at 29.97 and checks average_rate == Fraction(2997, 100). The test fails on main with AVRational(29, 1) and passes with the change. With the change the same script prints (300 frames at 29.97 fps really are 10.010 s):

24.0 24/1 video 10.000s audio 10.000s
29.97 2997/100 video 10.010s audio 10.000s
23.976 2997/125 video 10.010s audio 10.000s

Happy to open a PR if that works for you.

Who can help?

@DN6

Dominant language
Python
Stars
34.8k
Forks
7.4k
Avg merge
2d 13h
Merged PRs (30d)
58

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/diffusers

All issues in huggingface/diffusers

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.