Vulkan encoder on NVIDIA GPU fails to import KWin DMA-BUF (Intel modifier mismatch)

Start in src/platform/linux/pipewire.cpp and compare the KWin capture path with the modifier-intersection logic described in PR #5734, which currently covers wlroots capture. Trace how the Vulkan encoder imports PipeWire DMA-BUFs, then ensure KWin offers only modifiers readable by the importing GPU and reports a clean failure when none are shared.

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the latest beta/pre-release?

This issue is present in the latest pre-release

Describe the Bug

I'm running a hybrid laptop where the Intel GPU renders the desktop and the NVIDIA GPU handles encoding via Vulkan (encoder = vulkan, adapter_name = /dev/dri/renderD129).
When streaming, Sunshine fails to import the DMA-BUFs provided by KWin. The client just gets green frames (on both 2026.914 stable and 929 pre-release), and the log shows continuous encoder recreation and import failures. On older builds like 2026.516, this failure crashed Sunshine completely.

There is a format modifier mismatch. KWin allocates the buffer using a compressed Intel modifier (0x100000000000004, which is I915_FORMAT_MOD_Y_TILED_CCS). The NVIDIA driver can't read this, so it throws:
vkCreateImage for DMA-BUF failed: -1000158000 (modifier=0x100000000000004, pitch=7680, offset=0)

-1000158000 is VK_ERROR_INVALID_DRM_FORMAT_MODIFIER_PLANE_LAYOUT_EXT.
Looking at src/platform/linux/pipewire.cpp, Sunshine offers PipeWire the modifiers that EGL reports for the compositor's GPU. However, it never checks those modifiers against the Vulkan device that actually has to import the buffer. KWin picks an Intel-specific modifier, and the NVIDIA Vulkan driver rejects it.

Similar issues for context:

• #5659 ("DMA-BUF Failed [2]"): Hits the exact same vkCreateImage error but under kmsgrab with modifier 0x100000000000008. The overlay discussion there doesn't apply to this KWin capture.
• PR #5734: This implements the necessary modifier intersection check between the Vulkan driver and the compositor, but it currently only applies to wlroots capture. KWin capture misses out on this logic.

Note: I used capture = kwin in config. leaving the form dropdown as None since it lacks a KWin option. This is the setup the troubleshooting docs recommend for KDE Plasma (kwin capture with vulkan encoding)

Expected Behavior

When the encoder runs on a secondary GPU, Sunshine should only offer PipeWire format modifiers that the importing GPU can read (like LINEAR). If the GPUs share no common modifiers, it should fail cleanly with an error rather than streaming green frames or looping.

Additional Context

I'm bypassing the Vulkan encoder by using encoder = vaapi directly on the Intel GPU, or using standard NVENC via host memory.

Host Operating System

Linux

Operating System Version

Kubuntu 26.04 (Plasma 6.6.6 Wayland), Kernel 7.0.0-34

Architecture

amd64/x86_64

Package

Linux - deb

GPU Type

NVIDIA

GPU Model

NVIDIA GeForce GTX 1650 Mobile / Max-Q + Intel UHD Comet Lake [8086:9bca]

GPU Driver/Mesa Version

NVIDIA 595.91.07, Mesa 26.0.8 (Intel)

Capture Method

None

Apps

Log output

[2026-10-01 14:03:25.156]: Info: CLIENT CONNECTED
[2026-10-01 14:03:25.223]: Info: [kwingrab] Found output: DVI-I-2 order: 1 position: 0x0 resolution: 1920x1080
[2026-10-01 14:03:25.223]: Info: [kwingrab] Found output: eDP-1 order: 0 position: 0x1080 resolution: 1920x1080
[2026-10-01 14:03:25.223]: Info: Screencasting with KWin ScreenCast
[2026-10-01 14:03:25.223]: Info: [pipewire] Requested variable frame rate (Sunshine pacing required: 8.33333ms)
[2026-10-01 14:03:25.223]: Info: [wayland] Found display [wayland-0]
[2026-10-01 14:03:25.239]: Info: [pipewire] Hybrid GPU system detected (Intel + discrete) - CUDA will use memory buffers
[2026-10-01 14:03:25.249]: Info: [kwingrab] Pipewire stream created: node=136
[2026-10-01 14:03:25.249]: Info: [kwingrab] Screencasting output name DVI-I-2 position 0x0 resolution 1920x1080
[2026-10-01 14:03:25.249]: Info: [pipewire] Streaming display 'DVI-I-2' offset: 0x0 resolution: 1920x1080
[2026-10-01 14:03:25.249]: Info: [wayland] Found display [wayland-0]
[2026-10-01 14:03:25.254]: Info: [wayland] Found interface: zxdg_output_manager_v1(31) version 3
[2026-10-01 14:03:25.254]: Info: [wayland] Found interface: zwp_linux_dmabuf_v1(57) version 5
[2026-10-01 14:03:25.254]: Info: [wayland] Found interface: wl_output(65) version 4
[2026-10-01 14:03:25.254]: Info: [wayland] Found interface: wl_output(99) version 4
[2026-10-01 14:03:25.255]: Info: [wayland] Resolution: 1920x1080
[2026-10-01 14:03:25.255]: Info: [wayland] Resolution: 1920x1080
[2026-10-01 14:03:25.255]: Info: [wayland] Offset: 0x1080
[2026-10-01 14:03:25.255]: Info: [wayland] Logical size: 1920x1080
[2026-10-01 14:03:25.255]: Info: [wayland] Name: eDP-1
[2026-10-01 14:03:25.255]: Info: [wayland] Found monitor: Chimei Innolux Corporation eDP-1-0x15E8
[2026-10-01 14:03:25.255]: Info: [wayland] Offset: 0x0
[2026-10-01 14:03:25.255]: Info: [wayland] Logical size: 1920x1080
[2026-10-01 14:03:25.255]: Info: [wayland] Name: DVI-I-2
[2026-10-01 14:03:25.255]: Info: [wayland] Found monitor: Dell Inc. DELL P2422H
[2026-10-01 14:03:25.255]: Info: [pipewire] Using frame_timestamp (pts) metadata source: Sunshine
[2026-10-01 14:03:25.258]: Info: [pipewire] PipeWire stream state: unconnected -> connecting
[2026-10-01 14:03:25.259]: Info: [pipewire] Connected to pipewire version 1.6.2
[2026-10-01 14:03:25.259]: Info: [pipewire] PipeWire stream state: connecting -> paused
[2026-10-01 14:03:25.266]: Info: [pipewire] Video format: 12
[2026-10-01 14:03:25.266]: Info: [pipewire] Size: 1920x1080
[2026-10-01 14:03:25.266]: Info: [pipewire] Color primaries: 0
[2026-10-01 14:03:25.266]: Info: [pipewire] Transfer function: 0
[2026-10-01 14:03:25.266]: Info: [pipewire] Compositor negotiated frame rate: 0/1 (variable rate capture)
[2026-10-01 14:03:25.266]: Info: [pipewire] using DMA-BUF buffers
[2026-10-01 14:03:25.268]: Info: Creating encoder [hevc_vulkan]
[2026-10-01 14:03:25.268]: Info: Color coding: SDR (Rec. 709)
[2026-10-01 14:03:25.268]: Info: Color depth: 8-bit
[2026-10-01 14:03:25.268]: Info: Color range: MPEG
[2026-10-01 14:03:25.270]: Info: [pipewire] Video format: 12
[2026-10-01 14:03:25.270]: Info: [pipewire] Size: 1920x1080
[2026-10-01 14:03:25.271]: Info: [pipewire] Color primaries: 0
[2026-10-01 14:03:25.271]: Info: [pipewire] Transfer function: 0
[2026-10-01 14:03:25.271]: Info: [pipewire] Compositor negotiated frame rate: 0/1 (variable rate capture)
[2026-10-01 14:03:25.271]: Info: [pipewire] using DMA-BUF buffers
[2026-10-01 14:03:25.274]: Info: [pipewire] Sunshine frame pacing: enabled (8.333333ms)
[2026-10-01 14:03:25.282]: Info: [pipewire] PipeWire stream state: paused -> streaming
[2026-10-01 14:03:25.563]: Info: Setting default sink to: [sink-sunshine-stereo]
[2026-10-01 14:03:25.564]: Info: Found default monitor by name: sink-sunshine-stereo.monitor
[2026-10-01 14:03:25.602]: Info: Opus initialized: 48 kHz, 2 channels, 512 kbps (total), LOWDELAY
[2026-10-01 14:03:25.633]: Info: Streaming bitrate is 56588000
[2026-10-01 14:03:25.633]: Info: [vulkan] Encoder quality set to 2 (balanced), driver range: 0-5
[2026-10-01 14:03:25.640]: Info: Vulkan encode using GPU: NVIDIA GeForce GTX 1650 with Max-Q Design
[2026-10-01 14:03:25.643]: Info: Minimum FPS target set to ~60fps (16.6667ms)
[2026-10-01 14:03:25.650]: Error: vkCreateImage for DMA-BUF failed: -1000158000 (modifier=0x100000000000004, pitch=7680, offset=0)
[2026-10-01 14:03:25.650]: Error: Failed to import DMA-BUF
[2026-10-01 14:03:25.650]: Error: Could not convert image
[2026-10-01 14:03:25.765]: Info: Creating encoder [hevc_vulkan]
[2026-10-01 14:03:25.766]: Info: Color coding: SDR (Rec. 709)
[2026-10-01 14:03:25.766]: Info: Color depth: 8-bit
[2026-10-01 14:03:25.766]: Info: Color range: MPEG
[2026-10-01 14:03:26.134]: Info: Streaming bitrate is 56588000
[2026-10-01 14:03:26.134]: Info: [vulkan] Encoder quality set to 2 (balanced), driver range: 0-5
[2026-10-01 14:03:26.139]: Info: Vulkan encode using GPU: NVIDIA GeForce GTX 1650 with Max-Q Design
[2026-10-01 14:03:26.141]: Info: Minimum FPS target set to ~60fps (16.6667ms)
[2026-10-01 14:03:26.142]: Error: vkCreateImage for DMA-BUF failed: -1000158000 (modifier=0x100000000000004, pitch=7680, offset=0)
[2026-10-01 14:03:26.142]: Error: Failed to import DMA-BUF
[2026-10-01 14:03:26.142]: Error: Could not convert image
[2026-10-01 14:03:26.253]: Info: Creating encoder [hevc_vulkan]
[2026-10-01 14:03:26.253]: Info: Color coding: SDR (Rec. 709)
[2026-10-01 14:03:26.253]: Info: Color depth: 8-bit
[2026-10-01 14:03:26.253]: Info: Color range: MPEG
[2026-10-01 14:03:26.628]: Info: Streaming bitrate is 56588000
[2026-10-01 14:03:26.628]: Info: [vulkan] Encoder quality set to 2 (balanced), driver range: 0-5
[2026-10-01 14:03:26.633]: Info: Vulkan encode using GPU: NVIDIA GeForce GTX 1650 with Max-Q Design
[2026-10-01 14:03:26.634]: Info: Minimum FPS target set to ~60fps (16.6667ms)
[2026-10-01 14:03:27.491]: Error: vkCreateImage for DMA-BUF failed: -1000158000 (modifier=0x100000000000004, pitch=7680, offset=0)
[2026-10-01 14:03:27.491]: Error: Failed to import DMA-BUF
[2026-10-01 14:03:27.491]: Error: Could not convert image
[2026-10-01 14:03:27.607]: Info: Creating encoder [hevc_vulkan]
[2026-10-01 14:03:27.607]: Info: Color coding: SDR (Rec. 709)
[2026-10-01 14:03:27.607]: Info: Color depth: 8-bit
[2026-10-01 14:03:27.607]: Info: Color range: MPEG
[2026-10-01 14:03:27.994]: Info: Streaming bitrate is 56588000
[2026-10-01 14:03:27.994]: Info: [vulkan] Encoder quality set to 2 (balanced), driver range: 0-5
[2026-10-01 14:03:27.999]: Info: Vulkan encode using GPU: NVIDIA GeForce GTX 1650 with Max-Q Design
[2026-10-01 14:03:28.002]: Info: Minimum FPS target set to ~60fps (16.6667ms)
[2026-10-01 14:03:28.003]: Error: vkCreateImage for DMA-BUF failed: -1000158000 (modifier=0x100000000000004, pitch=7680, offset=0)
[2026-10-01 14:03:28.003]: Error: Failed to import DMA-BUF
[2026-10-01 14:03:28.003]: Error: Could not convert image

Online logs

https://gist.github.com/dsvarela/56d6387faedac79347fba6aa576dedd0