GPF / use-after-free in postEvent<->osNotifyEvent<->_gpuEngineEventNotificationListNotify under concurrent GPU decode/encode (RTX 5060 Ti, 580.178.04)

Open
  • Is something wrong?
#1,418 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
c, linux
Domain
embedded-iot, operating-systems

Research direction

Start by reviewing the reported interrupt trace through postEvent, osNotifyEvent, and _gpuEngineEventNotificationListNotify, then compare it with the supplied vmcore backtraces and matching driver sources. Try the described concurrent NVDEC/NVENC workload if suitable hardware is available; done means identifying and fixing the use-after-free or providing a verified explanation and mitigation. The report says a minimal standalone reproduction is not yet available.

Written by the indexing model from the issue text.

Description

NVIDIA Open GPU Kernel Modules Version

580.178.04 (Ubuntu package 580.178.04-0ubuntu0.24.04.1, matches upstream tag 580.178.04 exactly)

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.
  • I confirm that this does not happen with the proprietary driver (of the same version).

Important caveat on the above checkbox: this GPU cannot run the proprietary driver at all — not "doesn't crash," genuinely cannot initialize. Confirmed directly in the kernel log after installing nvidia-driver-580 (proprietary, same 580.178.04 version): RmInitAdapter failed! (0x22:0x56:897) repeating continuously, with NVRM: ...installed in this system requires use of the NVIDIA open kernel modules. nvidia-smi reported "No devices were found" the entire time. So this bug is necessarily open-driver-specific by hardware requirement, not by choice — we could not test the proprietary driver even if we wanted to.

Operating System and Version

Ubuntu 24.04.4 LTS

Kernel Release

Linux qc-rig-1 6.8.0-139-generic #139-Ubuntu SMP PREEMPT_DYNAMIC Sat Aug 1 03:52:05 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux

(stock Ubuntu 24.04 HWE kernel package, not self-built)

Please confirm you are running a stable release kernel
  • I am running on a stable kernel release.
Hardware: GPU

GPU 0: NVIDIA GeForce RTX 5060 Ti (UUID: GPU-ff1a37be-b83d-af87-b7ab-7f7454468ead)

Describe the bug

Recurring general protection fault / use-after-free inside the GPU event-notification interrupt path. Crash signature (identical across all 6 occurrences):

RIP: postEvent+77 [nvidia]
RAX: 0x6b6b6b6b6b6b6b6b   <- SLUB freed-memory poison pattern

Call trace (GPU interrupt -> ISR -> notification dispatch -> fault):
asm_exc_general_protection
  [exception RIP: postEvent+77]
osNotifyEvent [nvidia]
_gpuEngineEventNotificationListNotify [nvidia]
kceServiceNotificationInterrupt_IMPL [nvidia]
intrServiceNotificationRecords_IMPL [nvidia]
_intrServiceNonStallLeaf_TU102 [nvidia]
intrServiceNonStall_TU102 [nvidia]
osInterruptPending.constprop.0 [nvidia]
osIsr [nvidia]
rm_isr [nvidia]
nvidia_isr [nvidia]
nvidia_isr_msix [nvidia]
__handle_irq_event_percpu
handle_irq_event
handle_edge_irq
__common_interrupt
common_interrupt
asm_common_interrupt

Occurred 6 times over 2 days (2026-10-03 through 2026-10-05), always during sustained concurrent GPU decode+encode load (multiple simultaneous ffmpeg NVENC/NVDEC sessions via CUDA). The crashing thread in the most recent occurrence was an active HEVC decode worker. In the hours before the 2026-10-04 22:19 occurrence specifically, dmesg shows recurring NVRM: ...Out of memory [NV_ERR_NO_MEMORY] errors roughly every 20-30 minutes, suggesting sustained VRAM pressure is a contributing trigger, not just raw concurrency.

Mitigation applied after the 2nd occurrence: throttled the application's concurrent GPU-decode pipeline from 4 simultaneous jobs to 2. This reduced the crash rate but did not eliminate it (4 more occurrences since).

Why we don't believe this is a known/already-fixed issue: we found a historically similar-looking report (#716, nvbug 4729070, stated fixed in 570.86.15) in this repo, but source-diffed the relevant code across every driver tag from 570.86.15 through our running 580.178.04 and into 590.48.01 -- the fix code in that exact path is byte-identical across all of them. Also, the RTX 5060 Ti (launched ~April 2026) postdates that issue (filed ~October 2025) by roughly 6 months, so it cannot be the same hardware/driver-era recurrence. As far as we can find, this specific fault under sustained concurrent-decode VRAM pressure on Blackwell is undocumented.

To Reproduce
  • Hardware: RTX 5060 Ti, open kernel driver 580.178.04
  • Run 2 (or more) concurrent ffmpeg processes, each doing hardware decode (NVDEC, -hwaccel cuda) of large (multi-GB, 30+ minute) H.264/HEVC source files while simultaneously encoding a separate lower-resolution proxy output via NVENC (-c:v h264_nvenc)
  • Sustain this concurrent load for an extended period (crashes have occurred anywhere from ~38 minutes to ~22 hours into a run -- no fixed reproduction time)
  • Crash occurs during a GPU interrupt, inside the driver's own event-notification dispatch path -- not obviously tied to any specific ffmpeg command-line flag, just sustained concurrent NVENC+NVDEC+CUDA activity

We do not have a minimal standalone repro outside our actual production pipeline yet. Happy to work with NVIDIA to build one if that would help triage.

Bug Incidence

Sometimes

nvidia-bug-report.log.gz

Pending -- will attach via a follow-up comment on this issue shortly (the file is generated and ready, just needs to be dragged into a comment).

More Info

4 full kernel vmcores (captured via kdump) are available for the 4 most recent occurrences, analyzed via the crash utility against matching linux-image-6.8.0-139-generic-dbgsym. Happy to share specific backtrace details, register state, or additional vmcore excerpts on request. This is a 24/7 unattended video-processing pipeline (not an interactive workstation), so the crashes are costly -- currently working around them with a concurrency throttle, but would very much like a real fix or at least guidance on what else to check.

Dominant language
C
Stars
17.5k
Forks
1.9k
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/open-gpu-kernel-modules

All issues in NVIDIA/open-gpu-kernel-modules

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.