Crash: avformat decoder cache (size 4) evicts state another thread is still decoding with
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- c
- Domain
- audio-video-rtc, backend
Research direction
Start with the producer_avformat producer_get_frame path and the mlt_service_cache_put() and mlt_service_cache_set_size() entry points. Run the linked cachelimit.c reproduction with multiple threads, then trace when cached decoder state is evicted and closed. Done means concurrent producers no longer use freed state without requiring the caller to raise the cache size.
Written by the indexing model from the issue text.
Description
Environment: MLT 7.40.0 (Fedora 44 package, x86_64), FFmpeg 8.1.2. The code involved is unchanged on master (0926a75).
Found while building a GTK4 video editor on MLT; the repro below uses only melt/libmlt.
What happens
producer_avformat keeps its decoder state in the service cache "producer_avformat", whose default size is 4 process-wide (mlt_service_cache_put()). When a fifth producer decodes, the least recently used entry is evicted and its state closed, even if another thread is decoding with it at that moment. So more than four producers decoding concurrently on different threads crash.
Repro
Sixteen threads, each with its own avformat producer of the same clip (all opened on the main thread first, so the lazy-init races can't hit), read 150 frames of video and audio (C, 60 lines): https://github.com/iDoMeteor/u-studio-video-editor/tree/70841c3/tools/upstream-repros/mlt/avformat-cache-limit
ffmpeg -f lavfi -i testsrc2=s=640x360:r=25:d=8 -f lavfi -i sine=d=8 \
-c:v libx264 -g 25 -pix_fmt yuv420p -c:a aac clip.mp4
cc -g cachelimit.c -o cachelimit -pthread $(pkg-config --cflags --libs mlt-framework-7)
for i in $(seq 30); do ./cachelimit clip.mp4 || echo crash; done | grep -c crash # 30
for i in $(seq 30); do ./cachelimit clip.mp4 raise || echo crash; done | grep -c crash # 0
Every crash is in __pthread_mutex_lock ← mlt_properties_get ← producer_get_frame (avformat) ← mlt_service_get_frame. raise calls mlt_service_cache_set_size(…, "producer_avformat", 17) first: 0 of 30. With 8 threads the rate varies with load (2 to 13 of 30 here).
Expected: concurrent decoding on separate producers is safe, or at worst slower. Actual: use of freed decoder state.
Notes
Either an entry in use should survive eviction until its user is done with it, or the limit should be documented as having to cover every concurrent decoder. kdenlive raises the size for the same reason; we set it to threads + 2 per track.
- Dominant language
- C
- Stars
- 1.9k
- Forks
- 391
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 12
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from mltframework/mlt
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
mltframework/mlt#1335 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 42/100
mltframework/mlt#1333 ·
Maintainers usually reply within 1 day
-
_unique_id is assigned with a non-atomic increment, so concurrent services can share an idPossibly taken @iDoMeteor claimed this 7 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 72/100
mltframework/mlt#1319 · 2 comments ·
Maintainers usually reply within 1 day
-
Crash: unguarded lazy init in loader, service cache and avformat when producers open on several threadsPossibly taken @iDoMeteor claimed this 7 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 55/100
mltframework/mlt#1317 · 2 comments ·
Maintainers usually reply within 1 day
-
mix transition allocates 9.2 MB per instancePossibly taken @iDoMeteor claimed this 7 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 72/100
mltframework/mlt#1316 · 5 comments ·
Maintainers usually reply within 1 day
All issues in mltframework/mlt
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
libsdl-org/SDL#16464 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
backend
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
BasedHardware/omi#20940 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 Under an hour Newbie friendliness 70/100
Maintainers usually reply within 1 day