RTX 5070 Ti: Xid 109 CTX SWITCH TIMEOUT then Xid 31 under a Vulkan compute pass (Unreal Engine 5.8.3 hair voxelization), 615.71.09 and 610.43.02

Đang mở
#1,420 3 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
25/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
c, linux, unreal-engine
Lĩnh vực
operating-systems

Hướng nghiên cứu

Start by reviewing the attached run notes, flattened shader diff, Vulkan device and extension log, and nvidia-bug-report log linked in the issue. Compare the reported hangs with voxelization enabled and the clean runs with it disabled, then investigate whether the kernel module can expose a useful faulting shader or pipeline identity for Xid 109. Done would mean identifying a reproducible driver-side cause or providing actionable diagnostic guidance.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

NVIDIA Open GPU Kernel Modules Version

615.71.09 (Arch package nvidia-open-dkms 615.71.09-1). Also reproduced on 610.43.02.

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.
  • I confirm that this does not happen with the proprietary driver package.

Not testable: the GPU is an RTX 5070 Ti (Blackwell), which the proprietary kernel module does not support. Filed here because the open modules are the only driver for this card.

Operating System and Version

Arch Linux (rolling)

Kernel Release

7.0.12-arch1-1 (stock Arch kernel, not self-built)

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.
  • I am running on a stable kernel release.
Hardware: GPU

GPU 1: NVIDIA GeForce RTX 5070 Ti (vbios 98.03.58.40.B4), PCI 0000:0b:00. It also drives the desktop. GPU 0, a GeForce GTX 1660 SUPER, is not used by the application.

Describe the bug

Unreal Engine 5.8.3 (Linux, Vulkan, shader model 6 profile) hangs the graphics channel of an RTX 5070 Ti about one run in two within 60 frames when its stock strand-hair voxelization is enabled. The kernel logs Xid 109 CTX SWITCH TIMEOUT. The driver never returns VK_ERROR_DEVICE_LOST to the application (nothing in 150 s), and about 150 s later, when the process is ended, an Xid 31 MMU fault is logged. The GPU and desktop recover. With the voxelization disabled the same scene is clean. Reproduced on 610.43.02 and 615.71.09.

To Reproduce
  • Unreal Engine 5.8.3 installed build (changelist 58210709, no engine patch), Linux, Vulkan, shader model 6 profile.
  • One skeletal creature with one dense strand groom (static, no deformer). Camera at 3 m, 960x1080, 60 frames of a recorded walk. Run offscreen (-game -RenderOffscreen -graphicsadapter=0).
  • r.HairStrands.Voxelization=1 (the engine default). Lumen, virtual shadow maps and the transient allocator are off; TAA.
  • Within 60 frames, about one run in two, the GPU hangs and the kernel logs Xid 109.
  • With r.HairStrands.Voxelization=0 the same scene runs clean: 0 hangs in 5 runs, plus 3 of 3 runs of 240 frames.
  • The project is private. A small standalone project with any dense strand groom at similar screen coverage can be built if that helps.
Bug Incidence

Sometimes: about one run in two, at frames 2 to 72.

nvidia-bug-report.log.gz

Taken with nvidia-bug-report.sh right after the hang of run 9 below. Uncompressed, with serials, network addresses and user names redacted, in this gist (file nvidia-bug-report.log): https://gist.github.com/shaneconner/2ff5b409c44047016d02f0fe1e4daef5

More Info

System

  • GPU: GeForce RTX 5070 Ti 16 GB (vbios 98.03.58.40.B4), PCI 0000:0b:00; it also drives the desktop (second GPU in the machine: GTX 1660 SUPER, not used by the application).
  • Driver: nvidia-open-dkms 610.43.02, then 615.71.09 (open kernel module). Kernel 7.0.12, Arch Linux. Vulkan loader 1.4.357, device API 1.4.351; engine uses API 1.3.
  • Application: Unreal Engine 5.8.3 installed build, changelist 58210709, no engine patch, a private UE 5.8.3 project. Run with -RenderOffscreen, -graphicsadapter=0, 960x1080.
  • Scene: one skeletal creature with one strand groom (static, no deformer), camera at 3 m, 60 frames of a recorded walk; no Lumen, no virtual shadow maps, async compute queue as default, TAA.
  • Nothing else is running on the card during the test.

Kernel lines (verbatim, host name replaced)

2026-10-05T07:10:15-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 109, pid=3745576, name=UnrealEditor, channel 0x0000000e, errorString CTX SWITCH TIMEOUT, Info 0x7c010
2026-10-05T07:10:17-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 31, pid=3745576, name=UnrealEditor, channel 0x0000000e, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC0 UNRECOGNIZED_CLIENT faulted @ 0x813f7_000efcb0. Fault is of type UNRECOGNIZED_FAULT UNRECOGNIZED_ACCESS_TYPE
2026-10-05T07:11:03-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 109, pid=3749844, name=UnrealEditor, channel 0x0000000e, errorString CTX SWITCH TIMEOUT, Info 0x1c010
2026-10-05T07:11:05-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 31, pid=3749844, name=UnrealEditor, channel 0x0000002f, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC0 UNRECOGNIZED_CLIENT faulted @ 0x813f6_00010800. Fault is of type UNRECOGNIZED_FAULT UNRECOGNIZED_ACCESS_TYPE
(the two pairs above: driver 610.43.02)
2026-10-05T08:10:04-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 109, pid=111419, name=UnrealEditor, channel 0x0000000a, errorString CTX SWITCH TIMEOUT, Info 0x2c00d
2026-10-05T08:23:15-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 109, pid=190466, name=UnrealEditor, channel 0x0000000a, errorString CTX SWITCH TIMEOUT, Info 0x6c00d
2026-10-05T08:25:47-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 31, pid=190466, name=UnrealEditor, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC0 UNRECOGNIZED_CLIENT faulted @ 0x813f4_00e2806e. Fault is of type UNRECOGNIZED_FAULT UNRECOGNIZED_ACCESS_TYPE
2026-10-05T11:23:20-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 109, pid=1291687, name=UnrealEditor, channel 0x0000000a, errorString CTX SWITCH TIMEOUT, Info 0x6c00d
2026-10-05T11:25:53-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 31, pid=1291687, name=UnrealEditor, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC0 UNRECOGNIZED_CLIENT faulted @ 0x813f8_0000ccc0. Fault is of type UNRECOGNIZED_FAULT UNRECOGNIZED_ACCESS_TYPE
2026-10-05T12:44:03-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 109, pid=1701430, name=UnrealEditor, channel 0x0000000a, errorString CTX SWITCH TIMEOUT, Info 0x2c00d
2026-10-05T12:46:34-05:00 <host> kernel: NVRM: Xid (PCI:0000:0b:00): 31, pid=1701430, name=UnrealEditor, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC0 UNRECOGNIZED_CLIENT faulted @ 0x813f0_01a5f00c. Fault is of type UNRECOGNIZED_FAULT UNRECOGNIZED_ACCESS_TYPE
(the last four hangs: driver 615.71.09; the final pair is run 9, the flattened shader, and nvidia-bug-report.log.gz was taken right after it; hostname replaced by <host>)

Runs (all with r.HairStrands.Voxelization=1 forced on)

run change frames rendered result
1 stock hair voxelization, Virtual.Adaptive 0 (610.43.02) 3 hang, Xid 109 then 31
2 GPUDriven 0 (610.43.02) 2 hang, Xid 109 then 31
3 stock, 615.71.09 25 hang, Xid 109
4 -gpucrashdebugging -gpubreadcrumbs, 150 s wait 22 hang, Xid 109, Xid 31 at process kill; no VK_ERROR_DEVICE_LOST, no breadcrumbs
5, 6 Khronos validation plus GPU-assisted validation 60, 60 clean
7 Khronos synchronization validation only, 10 frames 10 clean; 4 messages, none on a hair or voxel pass
8 r.Vulkan.AllowAsyncCompute=0 63 hang, Xid 109, Xid 31 at kill
9 raster loop flattened (no data-dependent branch in the loop body: select()/and(), one unconditional InterlockedAdd per step), 615.71.09 3 hang, Xid 109 (Info 0x2c00d), Xid 31 at kill
about 40 earlier (610.43.02) cvar matrix baseline-class four: 2 clean, 2 hung (frames 25, 49); others hung at frames 3, 5, 7, 12, 34, 72
groom hidden 0 hangs in 5
r.HairStrands.Voxelization=0 0 hangs in 5, plus 3 of 3 runs of 240 frames

Ruled out

  • Driver version (both fault).
  • Voxel allocation and loop-limit settings: Virtual.Adaptive 0, GPUDriven 0, page-count pool size, ComputeRasterMaxVoxelCount 8, GPUDriven.MaxPageIndexResolution 32 all still hang.
  • Load: clean frames measure the voxel raster at 0.87 ms and all voxelization at 3.35 ms, 2099 to 2166 of 2744 pages. This does not look like a long dispatch.
  • Async compute queue: disabling it still hangs.
  • API misuse visible to Khronos synchronization validation: none on the hair passes.

Observations

  • Hang is absent under GPU-assisted validation (2 of 2 clean), which instruments the shaders.
  • The pass that is skipped when voxelization is off includes a compute shader with one thread per strand point and a data-dependent loop. Inside the loop are three nested conditionals around an InterlockedAdd on a RWTexture3D (a conditional page-index fetch, a page-valid test, a repeated-voxel test, and for ray-traced grooms an InterlockedOr). Release notes for 615.71.09 mention a Blackwell fix for "two or more nested levels of thread divergence, where reconvergence could fail". We saw hangs on 615.71.09 too. We do not know whether that fix covers Vulkan shaders; this is a guess, not a finding.
  • The Xid 31 fault address is a high, garbage-looking value with UNRECOGNIZED_CLIENT, and appears only after the hang when the process is killed.
  • Hang timing varies from frame 2 to frame 72.

Questions

Note on question 1: the compute shader's nested-branch structure was removed in run 9 (the engine's raster loop rewritten with selects, one unconditional atomic per step) and it still hangs on 615.71.09, so a nested-divergence reconvergence fault in that loop does not explain it, unless the compiler re-introduces the branches (the compiled output was not inspected). Flattened shader and diff attached.

  1. Is a Blackwell shader code generation or reconvergence problem known for graphics or compute pipelines in 610/615 that matches this?
  2. Why is VK_ERROR_DEVICE_LOST not reported after the context switch timeout?
  3. Is there a driver-side setting or debug facility to get the faulting shader or a pipeline/shader identity for a Xid 109 on Vulkan (Aftermath checkpoints are compiled out of this build)?

The gist also holds the run notes, the flattened shader diff and the Vulkan device and extension log.

Ngôn ngữ chính
C
Star
17.5k
Fork
1.9k
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của NVIDIA/open-gpu-kernel-modules

Tất cả issue của NVIDIA/open-gpu-kernel-modules

Issue tương tự

Thêm issue về C

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.