Discoverability issue during CDI spec generation when the program lives inside a snap container

Open

#554 opened on Jun 20, 2024 7 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty: 4/5
Estimated time: 3-5 days
Newbie friendliness: 35/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Stale
Tech stack: go
Domain: tooling

Research direction

Start at nvcdi.New and follow the discoverability handling used by WithDriverRoot and WithDevRoot, focusing on the snap host root at /var/lib/snapd/hostfs. Reproduce CDI generation from inside a snap and compare it with the non-snap case. Done means host devices and libraries are discovered correctly without regressing ordinary root-path discovery.

Written by the indexing model from the issue text.

Description

enhancement help-wanted

I'm trying to generate a CDI specification in a program living inside a snap container (see https://github.com/canonical/lxd/pull/13562)

When generating a CDI spec from inside a snap package, we need to tell the CDI discoverability process to start
looking after the host root file system which is defined inside the snap by /var/lib/snapd/hostfs (a.k.a, this is the same as / if we are not in a snap)

In consequence, here is how I created my cdilib object:

package cdi

import (
	"fmt"
	"os/exec"
	"path/filepath"

	"github.com/NVIDIA/nvidia-container-toolkit/pkg/nvcdi"
	"github.com/canonical/lxd/shared"
	cdpiSpec "tags.cncf.io/container-device-interface/pkg/cdi"
	"tags.cncf.io/container-device-interface/specs-go"
)

const (
	// defaultNvidiaTegraMountSpecPath is default location of CSV files that define the modifications required to the OCI spec
	defaultNvidiaTegraMountSpecPath = "/etc/nvidia-container-runtime/host-files-for-container.d"
)

// DefaultNvidiaTegraCSVFiles returns the default CSV files for the Nvidia Tegra platform.
func DefaultNvidiaTegraCSVFiles() []string {
	files := []string{
		"devices.csv",
		"drivers.csv",
		"l4t.csv",
	}

	var paths []string
	for _, file := range files {
		paths = append(paths, filepath.Join(defaultNvidiaTegraMountSpecPath, file))
	}

	return paths
}

// generateNvidiaSpec generates a CDI spec for an Nvidia vendor.
func generateNvidiaSpec(cdiID ID) (*specs.Spec, error) {
	mode := nvcdi.ModeAuto
	if cdiID.DeviceType() == IGPU {
		mode = nvcdi.ModeCSV
	}

	deviceNamer, err := nvcdi.NewDeviceNamer(nvcdi.DeviceNameStrategyIndex)
	if err != nil {
		return nil, fmt.Errorf("Failed to create device namer: %v", err)
	}

	deviceNamers := []nvcdi.DeviceNamer{deviceNamer}
	nvidiaCTKPath, err := exec.LookPath("nvidia-ctk")
	if err != nil {
		return nil, fmt.Errorf("Failed to find the nvidia-ctk binary: %v", err)
	}

	rootPath := ""
	if shared.InSnap() {
		rootPath = "/var/lib/snapd/hostfs"
	}

	cdilib, err := nvcdi.New(
		nvcdi.WithLogger(&CDILogger{}),
		nvcdi.WithDriverRoot(rootPath),
		nvcdi.WithDevRoot(rootPath),
		nvcdi.WithNVIDIACTKPath(nvidiaCTKPath),
		nvcdi.WithDeviceNamers(deviceNamers...),
		nvcdi.WithMode(mode),
		nvcdi.WithCSVFiles(DefaultNvidiaTegraCSVFiles()),
	)
	if err != nil {
		return nil, fmt.Errorf("Failed to create CDI library: %v", err)
	}

	deviceSpecs, err := cdilib.GetAllDeviceSpecs()
	if err != nil {
		return nil, fmt.Errorf("Failed to create device CDI specs: %v", err)
	}

	commonEdits, err := cdilib.GetCommonEdits()
	if err != nil {
		return nil, fmt.Errorf("Failed to create edits common for entities: %v", err)
	}

	return &specs.Spec{
		Version:        cdpiSpec.CurrentVersion,
		Kind:           fmt.Sprintf("%s/%s", cdiID.Vendor(), cdiID.Product()),
		Devices:        deviceSpecs,
		ContainerEdits: *commonEdits.ContainerEdits,
	}, nil
}

Here is the resulting log for the generation:

DEBUG  [2024-06-20T15:41:27Z] Is WSL-based system? false: could not load DXCore library: libdxcore.so: cannot open shared object file: No such file or directory 
DEBUG  [2024-06-20T15:41:27Z] Is NVML-based system? true: found NVML library 
INFO   [2024-06-20T15:41:27Z] Auto-detected mode as "nvml"                 
DEBUG  [2024-06-20T15:41:27Z] Is Tegra-based system? true: /sys/devices/soc0/family has 'tegra' prefix 
DEBUG  [2024-06-20T15:41:27Z] returning cached mounts                      
DEBUG  [2024-06-20T15:41:27Z] Locating /dev/nvidia0                        
DEBUG  [2024-06-20T15:41:27Z] Locating "/dev/nvidia0" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/dev] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/dev/nvidia0' 
DEBUG  [2024-06-20T15:41:27Z] Located /dev/nvidia0 as [/var/lib/snapd/hostfs/dev/nvidia0] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/dev/dri/card1' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/dev/nvidia0 as /dev/nvidia0 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/dev/dri/card1 as /dev/dri/card1 
DEBUG  [2024-06-20T15:41:27Z] Locating /dev/dri/card1                      
DEBUG  [2024-06-20T15:41:27Z] Locating "/dev/dri/controlD65" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/dev] 
DEBUG  [2024-06-20T15:41:27Z] Locating "/dev/dri/card1" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/dev] 
DEBUG  [2024-06-20T15:41:27Z] Locating /dev/dri/controlD65                 
DEBUG  [2024-06-20T15:41:27Z] Located /dev/dri/card1 as [/var/lib/snapd/hostfs/dev/dri/card1] 
WARNING[2024-06-20T15:41:27Z] Could not locate /dev/dri/controlD65: pattern /dev/dri/controlD65 not found 
DEBUG  [2024-06-20T15:41:27Z] Locating /dev/dri/renderD128                 
DEBUG  [2024-06-20T15:41:27Z] Locating "/dev/dri/renderD128" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/dev] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/dev/dri/renderD128' 
DEBUG  [2024-06-20T15:41:27Z] Located /dev/dri/renderD128 as [/var/lib/snapd/hostfs/dev/dri/renderD128] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/dev/dri/renderD128 as /dev/dri/renderD128 
DEBUG  [2024-06-20T15:41:27Z] returning cached mounts                      
WARNING[2024-06-20T15:41:27Z] Failed to evaluate symlink /var/lib/snapd/hostfs/dev/dri/by-path/pci-0005:09:00.0-card; ignoring 
WARNING[2024-06-20T15:41:27Z] Failed to evaluate symlink /var/lib/snapd/hostfs/dev/dri/by-path/pci-0005:09:00.0-render; ignoring 
DEBUG  [2024-06-20T15:41:27Z] Opening ld.conf at /var/lib/snapd/hostfs/etc/ld.so.cache 
DEBUG  [2024-06-20T15:41:27Z] Locating "libcuda.so.*.*" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/usr/lib64 /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/lib64 /var/lib/snapd/hostfs/lib/x86_64-linux-gnu /var/lib/snapd/hostfs/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/lib/x86_64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/lib/aarch64-linux-gnu/nvidia/current] 
DEBUG  [2024-06-20T15:41:27Z] Opening ld.conf at /var/lib/snapd/hostfs/etc/ld.so.cache 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/lib/aarch64-linux-gnu/libcuda.so.535.161.08' 
INFO   [2024-06-20T15:41:27Z] Using driver version 535.161.08              
DEBUG  [2024-06-20T15:41:27Z] Opening ld.conf at /var/lib/snapd/hostfs/etc/ld.so.cache 
DEBUG  [2024-06-20T15:41:27Z] Locating "*.so.535.161.08" in [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu] 
DEBUG  [2024-06-20T15:41:27Z] Locating "libcuda.so.535.161.08" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/usr/lib64 /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/lib64 /var/lib/snapd/hostfs/lib/x86_64-linux-gnu /var/lib/snapd/hostfs/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/lib/x86_64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/lib/aarch64-linux-gnu/nvidia/current] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/lib/aarch64-linux-gnu/libcuda.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvoptix.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcudadebugger.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ml.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvcuvid.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-encode.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-vulkan-producer.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating /dev/nvidia-modeset                 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/dev/nvidia-modeset as /dev/nvidia-modeset 
DEBUG  [2024-06-20T15:41:27Z] Locating "/dev/nvidia-modeset" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/dev] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/dev/nvidia-modeset' 
DEBUG  [2024-06-20T15:41:27Z] Located /dev/nvidia-modeset as [/var/lib/snapd/hostfs/dev/nvidia-modeset] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/dev/nvidia-uvm' 
DEBUG  [2024-06-20T15:41:27Z] Locating /dev/nvidia-uvm-tools               
DEBUG  [2024-06-20T15:41:27Z] Locating "/dev/nvidia-uvm-tools" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/dev] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/dev/nvidia-uvm-tools as /dev/nvidia-uvm-tools 
DEBUG  [2024-06-20T15:41:27Z] Located /dev/nvidia-uvm-tools as [/var/lib/snapd/hostfs/dev/nvidia-uvm-tools] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/dev/nvidia-uvm-tools' 
DEBUG  [2024-06-20T15:41:27Z] Locating /dev/nvidia-uvm                     
DEBUG  [2024-06-20T15:41:27Z] Locating "/dev/nvidia-uvm" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/dev] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/dev/nvidiactl as /dev/nvidiactl 
DEBUG  [2024-06-20T15:41:27Z] Located /dev/nvidia-uvm as [/var/lib/snapd/hostfs/dev/nvidia-uvm] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/dev/nvidia-uvm as /dev/nvidia-uvm 
DEBUG  [2024-06-20T15:41:27Z] Locating /dev/nvidiactl                      
DEBUG  [2024-06-20T15:41:27Z] Locating "/dev/nvidiactl" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/dev] 
DEBUG  [2024-06-20T15:41:27Z] Located /dev/nvidiactl as [/var/lib/snapd/hostfs/dev/nvidiactl] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/dev/nvidiactl' 
DEBUG  [2024-06-20T15:41:27Z] Locating libnvidia-egl-gbm.so.*              
DEBUG  [2024-06-20T15:41:27Z] Locating "libnvidia-egl-gbm.so.*" in [/var/lib/snapd/hostfs /var/lib/snapd/hostfs/usr/lib64 /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/lib64 /var/lib/snapd/hostfs/lib/x86_64-linux-gnu /var/lib/snapd/hostfs/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/lib/x86_64-linux-gnu/nvidia/current /var/lib/snapd/hostfs/lib/aarch64-linux-gnu/nvidia/current] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1.1.0' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1.1.0' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1' 
DEBUG  [2024-06-20T15:41:27Z] Located libnvidia-egl-gbm.so.* as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1.1.0] 
DEBUG  [2024-06-20T15:41:27Z] Locating glvnd/egl_vendor.d/10_nvidia.json   
DEBUG  [2024-06-20T15:41:27Z] Locating "glvnd/egl_vendor.d/10_nvidia.json" in [/var/lib/snapd/hostfs/etc /var/lib/snapd/hostfs/usr/local/share /var/lib/snapd/hostfs/usr/share] 
DEBUG  [2024-06-20T15:41:27Z] Located glvnd/egl_vendor.d/10_nvidia.json as [/var/lib/snapd/hostfs/usr/share/glvnd/egl_vendor.d/10_nvidia.json] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/share/glvnd/egl_vendor.d/10_nvidia.json' 
DEBUG  [2024-06-20T15:41:27Z] Locating vulkan/icd.d/nvidia_icd.json        
DEBUG  [2024-06-20T15:41:27Z] Locating "vulkan/icd.d/nvidia_layers.json" in [/var/lib/snapd/hostfs/etc /var/lib/snapd/hostfs/usr/local/share /var/lib/snapd/hostfs/usr/share] 
DEBUG  [2024-06-20T15:41:27Z] Locating "vulkan/icd.d/nvidia_icd.json" in [/var/lib/snapd/hostfs/etc /var/lib/snapd/hostfs/usr/local/share /var/lib/snapd/hostfs/usr/share] 
DEBUG  [2024-06-20T15:41:27Z] Locating vulkan/icd.d/nvidia_layers.json     
DEBUG  [2024-06-20T15:41:27Z] Located vulkan/icd.d/nvidia_icd.json as [/var/lib/snapd/hostfs/usr/share/vulkan/icd.d/nvidia_icd.json] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/share/vulkan/icd.d/nvidia_icd.json as /usr/share/vulkan/icd.d/nvidia_icd.json 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/share/vulkan/icd.d/nvidia_icd.json' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/share/glvnd/egl_vendor.d/10_nvidia.json as /usr/share/glvnd/egl_vendor.d/10_nvidia.json 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1.1.0 as /usr/lib/aarch64-linux-gnu/libnvidia-egl-gbm.so.1.1.0 
DEBUG  [2024-06-20T15:41:27Z] Locating "vulkan/implicit_layer.d/nvidia_layers.json" in [/var/lib/snapd/hostfs/etc /var/lib/snapd/hostfs/usr/local/share /var/lib/snapd/hostfs/usr/share] 
DEBUG  [2024-06-20T15:41:27Z] Locating vulkan/implicit_layer.d/nvidia_layers.json 
WARNING[2024-06-20T15:41:27Z] Could not locate vulkan/icd.d/nvidia_layers.json: pattern vulkan/icd.d/nvidia_layers.json not found 
DEBUG  [2024-06-20T15:41:27Z] Located vulkan/implicit_layer.d/nvidia_layers.json as [/var/lib/snapd/hostfs/usr/share/vulkan/implicit_layer.d/nvidia_layers.json] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/share/vulkan/implicit_layer.d/nvidia_layers.json' 
DEBUG  [2024-06-20T15:41:27Z] Locating egl/egl_external_platform.d/15_nvidia_gbm.json 
DEBUG  [2024-06-20T15:41:27Z] Located egl/egl_external_platform.d/15_nvidia_gbm.json as [/var/lib/snapd/hostfs/usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json] 
DEBUG  [2024-06-20T15:41:27Z] Locating "egl/egl_external_platform.d/10_nvidia_wayland.json" in [/var/lib/snapd/hostfs/etc /var/lib/snapd/hostfs/usr/local/share /var/lib/snapd/hostfs/usr/share] 
DEBUG  [2024-06-20T15:41:27Z] Locating "egl/egl_external_platform.d/15_nvidia_gbm.json" in [/var/lib/snapd/hostfs/etc /var/lib/snapd/hostfs/usr/local/share /var/lib/snapd/hostfs/usr/share] 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia/xorg/libglxserver_nvidia.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json' 
WARNING[2024-06-20T15:41:27Z] Could not locate nvidia/xorg/libglxserver_nvidia.so.535.161.08: pattern nvidia/xorg/libglxserver_nvidia.so.535.161.08 not found 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json as /usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json 
DEBUG  [2024-06-20T15:41:27Z] Locating egl/egl_external_platform.d/10_nvidia_wayland.json 
DEBUG  [2024-06-20T15:41:27Z] Located nvidia/nvoptix.bin as [/var/lib/snapd/hostfs/usr/share/nvidia/nvoptix.bin] 
WARNING[2024-06-20T15:41:27Z] Could not locate egl/egl_external_platform.d/10_nvidia_wayland.json: pattern egl/egl_external_platform.d/10_nvidia_wayland.json not found 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia/nvoptix.bin                  
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia/nvoptix.bin" in [/var/lib/snapd/hostfs/etc /var/lib/snapd/hostfs/usr/local/share /var/lib/snapd/hostfs/usr/share] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/share/nvidia/nvoptix.bin' 
DEBUG  [2024-06-20T15:41:27Z] Locating "X11/xorg.conf.d/10-nvidia.conf" in [/var/lib/snapd/hostfs/etc /var/lib/snapd/hostfs/usr/local/share /var/lib/snapd/hostfs/usr/share] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/share/nvidia/nvoptix.bin as /usr/share/nvidia/nvoptix.bin 
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia/xorg/nvidia_drv.so" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu] 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia/xorg/nvidia_drv.so           
WARNING[2024-06-20T15:41:27Z] Could not locate nvidia/xorg/nvidia_drv.so: pattern nvidia/xorg/nvidia_drv.so not found 
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia/xorg/libglxserver_nvidia.so.535.161.08" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu] 
DEBUG  [2024-06-20T15:41:27Z] Locating X11/xorg.conf.d/10-nvidia.conf      
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Located X11/xorg.conf.d/10-nvidia.conf as [/var/lib/snapd/hostfs/usr/share/X11/xorg.conf.d/10-nvidia.conf] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/share/X11/xorg.conf.d/10-nvidia.conf' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.161.08" in [/var/lib/snapd/hostfs] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/share/X11/xorg.conf.d/10-nvidia.conf as /usr/share/X11/xorg.conf.d/10-nvidia.conf 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.161.08] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvcuvid.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvcuvid.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcudadebugger.so.535.161.08' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/share/vulkan/implicit_layer.d/nvidia_layers.json as /usr/share/vulkan/implicit_layer.d/nvidia_layers.json 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvcuvid.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvcuvid.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvcuvid.so.535.161.08] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvcuvid.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvcuvid.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.535.161.08] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.535.161.08] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-encode.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.161.08' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-ml.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-encode.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-encode.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-ml.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-encode.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-encode.so.535.161.08] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-encode.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-encode.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ml.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ml.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-ml.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libcudadebugger.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-ml.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ml.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.535.161.08' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.161.08] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.535.161.08" in [/var/lib/snapd/hostfs] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libcuda.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libcudadebugger.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.535.161.08] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/nvidia-fabricmanager/socket" in [/var/lib/snapd/hostfs/run /var/lib/snapd/hostfs/var/run] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libcudadebugger.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcudadebugger.so.535.161.08] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-vulkan-producer.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-vulkan-producer.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvoptix.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvoptix.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvoptix.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-vulkan-producer.so.535.161.08" in [/var/lib/snapd/hostfs] 
WARNING[2024-06-20T15:41:27Z] Could not locate /tmp/nvidia-mps: pattern /tmp/nvidia-mps not found 
DEBUG  [2024-06-20T15:41:27Z] Locating "/nvidia-persistenced/socket" in [/var/lib/snapd/hostfs/run /var/lib/snapd/hostfs/var/run] 
DEBUG  [2024-06-20T15:41:27Z] Locating /nvidia-persistenced/socket         
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.161.08" in [/var/lib/snapd/hostfs] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/run/nvidia-persistenced/socket as /run/nvidia-persistenced/socket 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-vulkan-producer.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-vulkan-producer.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvoptix.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvoptix.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating /usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.161.08 
WARNING[2024-06-20T15:41:27Z] Could not locate /nvidia-fabricmanager/socket: pattern /nvidia-fabricmanager/socket not found 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvoptix.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvoptix.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating /nvidia-fabricmanager/socket        
DEBUG  [2024-06-20T15:41:27Z] Locating "/tmp/nvidia-mps" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating /tmp/nvidia-mps                     
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/run/nvidia-persistenced/socket' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Locating "/usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.161.08" in [/var/lib/snapd/hostfs] 
DEBUG  [2024-06-20T15:41:27Z] Found 1 candidates; ignoring further candidates 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-tls.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Located /nvidia-persistenced/socket as [/var/lib/snapd/hostfs/run/nvidia-persistenced/socket] 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libnvidia-vulkan-producer.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libnvidia-vulkan-producer.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia/535.161.08/gsp*.bin          
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia/535.161.08/gsp*.bin" in [/var/lib/snapd/hostfs/lib/firmware/updates/5.15.0-1012-nvidia-tegra-igx /var/lib/snapd/hostfs/lib/firmware/updates /var/lib/snapd/hostfs/lib/firmware/5.15.0-1012-nvidia-tegra-igx /var/lib/snapd/hostfs/lib/firmware] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libcudadebugger.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libcudadebugger.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/lib/firmware/nvidia/535.161.08/gsp_tu10x.bin' 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/lib/firmware/nvidia/535.161.08/gsp_ga10x.bin' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/lib/firmware/nvidia/535.161.08/gsp_ga10x.bin as /lib/firmware/nvidia/535.161.08/gsp_ga10x.bin 
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia-smi" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/current/bin /var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/snap/lxd/current/bin] 
DEBUG  [2024-06-20T15:41:27Z] Located /usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.161.08 as [/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.161.08] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.161.08' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/lib/firmware/nvidia/535.161.08/gsp_tu10x.bin as /lib/firmware/nvidia/535.161.08/gsp_tu10x.bin 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia-smi                          
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.535.161.08' 
DEBUG  [2024-06-20T15:41:27Z] Located nvidia/535.161.08/gsp*.bin as [/var/lib/snapd/hostfs/lib/firmware/nvidia/535.161.08/gsp_ga10x.bin /var/lib/snapd/hostfs/lib/firmware/nvidia/535.161.08/gsp_tu10x.bin] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/bin/nvidia-smi' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.161.08 as /usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.535.161.08 
DEBUG  [2024-06-20T15:41:27Z] Found 1 candidates; ignoring further candidates 
DEBUG  [2024-06-20T15:41:27Z] Located nvidia-smi as [/var/lib/snapd/hostfs/usr/bin/nvidia-smi] 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia-debugdump                    
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/bin/nvidia-smi as /usr/bin/nvidia-smi 
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia-debugdump" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/current/bin /var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/snap/lxd/current/bin] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/bin/nvidia-debugdump' 
DEBUG  [2024-06-20T15:41:27Z] Found 1 candidates; ignoring further candidates 
DEBUG  [2024-06-20T15:41:27Z] Located nvidia-debugdump as [/var/lib/snapd/hostfs/usr/bin/nvidia-debugdump] 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/bin/nvidia-debugdump as /usr/bin/nvidia-debugdump 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia-persistenced                 
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia-persistenced" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/current/bin /var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/snap/lxd/current/bin] 
DEBUG  [2024-06-20T15:41:27Z] Located nvidia-persistenced as [/var/lib/snapd/hostfs/usr/bin/nvidia-persistenced] 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/bin/nvidia-persistenced' 
DEBUG  [2024-06-20T15:41:27Z] Found 1 candidates; ignoring further candidates 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/bin/nvidia-persistenced as /usr/bin/nvidia-persistenced 
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia-cuda-mps-control" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/current/bin /var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/snap/lxd/current/bin] 
DEBUG  [2024-06-20T15:41:27Z] Located nvidia-cuda-mps-control as [/var/lib/snapd/hostfs/usr/bin/nvidia-cuda-mps-control] 
DEBUG  [2024-06-20T15:41:27Z] Found 1 candidates; ignoring further candidates 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia-cuda-mps-server              
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/bin/nvidia-cuda-mps-control' 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/bin/nvidia-cuda-mps-control as /usr/bin/nvidia-cuda-mps-control 
DEBUG  [2024-06-20T15:41:27Z] Checking candidate '/var/lib/snapd/hostfs/usr/bin/nvidia-cuda-mps-server' 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia-cuda-mps-control             
DEBUG  [2024-06-20T15:41:27Z] Found 1 candidates; ignoring further candidates 
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia-cuda-mps-server" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/var/lib/snapd/hostfs/snap/lxd/current/bin /var/lib/snapd/hostfs/run/bin /var/lib/snapd/hostfs/snap/lxd/x1/usr/sbin /var/lib/snapd/hostfs/snap/lxd/x1/usr/bin /var/lib/snapd/hostfs/snap/lxd/x1/sbin /var/lib/snapd/hostfs/snap/lxd/x1/bin /var/lib/snapd/hostfs/usr/local/sbin /var/lib/snapd/hostfs/usr/local/bin /var/lib/snapd/hostfs/usr/sbin /var/lib/snapd/hostfs/usr/bin /var/lib/snapd/hostfs/sbin /var/lib/snapd/hostfs/bin /var/lib/snapd/hostfs/usr/games /var/lib/snapd/hostfs/usr/local/games /var/lib/snapd/hostfs/snap/lxd/current/bin] 
DEBUG  [2024-06-20T15:41:27Z] Located nvidia-cuda-mps-server as [/var/lib/snapd/hostfs/usr/bin/nvidia-cuda-mps-server] 
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia/xorg/nvidia_drv.so" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu] 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia/xorg/nvidia_drv.so           
WARNING[2024-06-20T15:41:27Z] Could not locate nvidia/xorg/nvidia_drv.so: pattern nvidia/xorg/nvidia_drv.so not found 
DEBUG  [2024-06-20T15:41:27Z] Locating nvidia/xorg/libglxserver_nvidia.so.535.161.08 
INFO   [2024-06-20T15:41:27Z] Selecting /var/lib/snapd/hostfs/usr/bin/nvidia-cuda-mps-server as /usr/bin/nvidia-cuda-mps-server 
DEBUG  [2024-06-20T15:41:27Z] returning cached mounts                      
DEBUG  [2024-06-20T15:41:27Z] Locating "nvidia/xorg/libglxserver_nvidia.so.535.161.08" in [/var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu] 
WARNING[2024-06-20T15:41:27Z] Could not locate nvidia/xorg/libglxserver_nvidia.so.535.161.08: pattern nvidia/xorg/libglxserver_nvidia.so.535.161.08 not found

What is quite odd is the fact that the discoverability logic is looking for path starting by /var/lib/snapd/hostfs/var/lib/snapd/hostfs and not /var/lib/snapd/hostfs...
Is it a bug or is it because the cdilib object has not been well configured?

This actually leads to a problem where nvidia/xorg/libglxserver_nvidia.so.535.161.08 is not generated in the output (missing mount entry + hook entry)
The logic look for the pattern inside /var/lib/snapd/hostfs/var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu and /var/lib/snapd/hostfs/usr/lib/x86_64-linux-gnu.
If it was looking in /var/lib/snapd/hostfs/usr/lib/aarch64-linux-gnu, then it would have worked.

What do you think @elezar ? /cc @tomponline

Dominant language: Go
Stars: 4.6k
Forks: 609
Avg merge: 3d 8h
Merged PRs (30d): 36

Getting set up

No Dockerfile or Docker Compose file
Has a pull request template
Read the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

More from NVIDIA/nvidia-container-toolkit

[Bug]: Containers with NVIDIA_VISIBLE_DEVICES=all fail to start in jit-cdi mode when every GPU is bound to vfio-pci
Possibly taken @efegokdemir claimed this 7 days ago. Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 85/100

NVIDIA/nvidia-container-toolkit#2163 · Opened Oct 24, 2026

Maintainers usually reply within 1 day
Address TODO comments in internal/platform-support/dgpu/dgpu.go
Possibly taken @truongbo17 claimed this 33 days ago. Open
- Is something wrong?
dsx-ws-0930 good-first-issue

Difficulty 2/5 1-3 hours Newbie friendliness 82/100

NVIDIA/nvidia-container-toolkit#2114 · 1 comment · Opened Sep 30, 2026

Maintainers usually reply within 1 day
cdi generate: a per-function spec for ConnectX network functions (nvidia.com/nic)
Open
- Is something wrong?
Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nvidia-container-toolkit#2197 · Opened Nov 2, 2026

Maintainers usually reply within 1 day
[Bug]: Containerized installer removes the existing Toolkit installation before the replacement is complete and durable
Possibly taken @henry118 claimed this 13 days ago. Open
- Is something wrong?
bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/nvidia-container-toolkit#2153 · 2 comments · 1 assignee · Opened Oct 20, 2026

Maintainers usually reply within 1 day
nvidia-cdi-hook chmod: adopt os.Root confinement like sibling hooks
Open
- Is something wrong?
Difficulty 3/5 1-2 days Newbie friendliness 78/100

NVIDIA/nvidia-container-toolkit#2147 · 1 comment · Opened Oct 17, 2026

Maintainers usually reply within 1 day

All issues in NVIDIA/nvidia-container-toolkit

Similar issues

mqtt_paho: transformer.DeleteExtension and DeleteAttribute make WritePubMessage fail
Possibly taken @RaphaelFakhri claimed this today. Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 86/100

cloudevents/sdk-go#1349 · Opened Nov 4, 2026
getChaosHub always returns totalFaults "0"
Possibly taken @Aryanbhargava18 claimed this today. Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 78/100

litmuschaos/litmus#5710 · Opened Nov 4, 2026

Maintainers usually reply within 3 days
vtselect: data race in Tempo search tags APIs for invalid TraceQL queries
Possibly taken @Shubham19032004 claimed this today. Open
- Is something wrong?
bug

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

VictoriaMetrics/VictoriaTraces#304 · Opened Nov 4, 2026

Maintainers usually reply within 1 day
kn secret create --tls-cert creates the certificate under the wrong key
Possibly taken @RaphaelFakhri claimed this today. Open
- Is something wrong?
Difficulty 1/5 Under an hour Newbie friendliness 91/100

knative/client#2284 · Opened Nov 4, 2026

Maintainers usually reply within 1 day
Routing rule or circuit breaker trip expression with an unknown CEL function hangs daprd
Possibly taken @RaphaelFakhri claimed this today. Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 75/100

dapr/dapr#11050 · Opened Nov 4, 2026

Maintainers usually reply within 1 day

More Go issues

Discoverability issue during CDI spec generation when the program lives inside a snap container

Is something wrong?

Assessment

Research direction

Description

Getting set up

First steps

More from NVIDIA/nvidia-container-toolkit

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Similar issues

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Get new issues in your inbox