`max_fps` on video files still throttles instead of dropping frames; docstring says dropping would be the default by end of Q4 2024

Open
  • Is something wrong?
#3,130 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
performance

Research direction

Start with resolve_limiter_strategy in stream_vision/streamvision/camera/utils.py, then read the flag definition in inference/core/env.py and its CLI override in inference_cli/lib/workflows/core.py. Compare the three docstrings named in the issue and the Python API behavior; first get a maintainer decision on whether file sources should drop frames by default or retain WAIT. Done means the chosen behavior is consistent across CLI and Python API and the expired-date wording is resolved.

Written by the indexing model from the issue text.

Description

bug
Search before asking
  • I have searched the Inference issues and found no similar bug report.
Bug

With a video file as the source, InferencePipeline with max_fps=6 processes every frame and slows down to roughly that rate. I expected it to skip frames, because the max_fps docstring says the frame-dropping behavior "will be the default one end of Q4 2024!" That date has passed, and files still default to the old behavior.

In the code (main at 4c47f51):

  • resolve_limiter_strategy in stream_vision/streamvision/camera/utils.py returns FPSLimiterStrategy.WAIT for file sources when no strategy is passed.
  • Dropping frames on files is opt-in through ENABLE_FRAME_DROP_ON_VIDEO_FILE_RATE_LIMITING, which defaults to "False" in inference/core/env.py.
  • inference_cli/lib/workflows/core.py sets that flag to True itself, with a comment saying it's needed until the new behavior becomes the default.
  • The Q4 2024 note is in three docstrings: once in stream_vision/streamvision/stream/pipeline.py and twice in inference/core/interfaces/legacy_stream/inference_pipeline.py.

So the same max_fps on the same file gives different results from the CLI and from the Python API, and the only explanation is a docstring with an expired date. I couldn't find this in the docs site.

On a 15 s, 36 fps dashcam clip (540 frames) with rfdetr-base and max_fps=6:

Frames processed Wall time
Default 540 92.7 s
ENABLE_FRAME_DROP_ON_VIDEO_FILE_RATE_LIMITING=True 90 7.8 s
Environment
  • inference 1.7.3 from PyPI, Python 3.12.14
  • macOS 27.0.1, Apple M2, no Docker (ONNX Runtime with CoreML)
Minimal Reproducible Example
import time
from inference import InferencePipeline

frames = 0

def on_prediction(predictions, video_frame):
    global frames
    frames += 1

start = time.time()
pipeline = InferencePipeline.init(
    model_id="rfdetr-base",
    video_reference="clip.mp4",  # any video file; 15 s at 36 fps here
    on_prediction=on_prediction,
    max_fps=6,
)
pipeline.start()
pipeline.join()
print(f"frames processed: {frames}, wall time: {time.time() - start:.1f}s")

Run it as-is, then again with ENABLE_FRAME_DROP_ON_VIDEO_FILE_RATE_LIMITING=True.

Additional

I'm not sure which fix you'd want:

  1. Make dropping the default for files. That changes results for anyone who relies on every frame being processed, so it probably needs a release note.
  2. Keep WAIT as the default and update the three docstrings to describe the flag without the date, plus a short note in the InferencePipeline docs.

I'm happy to open a PR for either one once you choose. I noticed InferencePipeline is moving into stream_vision, so I'll wait to hear which way you want to go.

Are you willing to submit a PR?
  • Yes I'd like to help by submitting a PR!
Dominant language
Python
Stars
2.5k
Forks
340
Avg merge
2d 11h
Merged PRs (30d)
92

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from roboflow/inference

All issues in roboflow/inference

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.