[Issue]: amdgpu-dkms 7.1.3 builds fail non-deterministically on Ubuntu 24.04, abort GNU make, and hard-lock the host during kernel upgrade
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức phù hợp với người mới
- 8/100
- Loại issue
- Lỗi
- Độ rõ ràng
- Khá rõ ràng
- Mức độ hoạt động
- Sôi nổi
- Công nghệ
- c, docker, linux, ubuntu
- Lĩnh vực
- operating-systems
Hướng nghiên cứu
This is a kernel-driver build failure with no comments and no assignee. Start from the mmhub_v3_3.o compile error in the amdgpu DKMS tree and the raw_atomic_fetch_add fallback header the report cites. Reproducing it needs a disposable GPU host with the Ubuntu 6.8 headers, and any fix needs a maintainer. Done means a maintainer has confirmed the root cause.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
Description
amdgpu-dkms from the AMD 31.50 repository fails non-deterministically while building against Ubuntu 24.04 HWE kernels. The failures have included:
- a compile-time
raw_atomic_fetch_adderror while compilingmmhub_v3_3.o; /usr/bin/makeitself aborting withSIGABRTanddouble free or corruption (out);- an Ubuntu DKMS Apport hook crash while trying to classify the failed build; and
- a complete host hard-lock during a later unattended DKMS retry.
The hard-lock happened while the system was running normally, not after a reboot. The BMC console image froze and the host stopped responding. The persistent journal stops abruptly during apt-daily-upgrade; the unattended-upgrade dpkg log ends in the middle of the DKMS make progress line. No orderly shutdown, kernel panic, OOM, MCE, uncorrectable EDAC event, watchdog report, or pstore record was found.
After power cycling, the newly installed 6.8.0-146-generic kernel was incomplete: its kernel image existed, but /boot/initrd.img-6.8.0-146-generic did not. The host is currently recovered and running the previous 6.8.0-142-generic kernel. I have not deliberately re-run the build because it has already hard-locked this production host.
Environment
- OS: Ubuntu Server 24.04.4 LTS (
noble), x86_64 - Current working kernel:
6.8.0-142-generic(6.8.0-142.142) - Failing target kernel:
6.8.0-146-generic(6.8.0-146.146) amdgpu-dkms:1:7.1.3.31500000-2390945.24.04- DKMS module version:
7.1.3-2390945.24.04 amdgpu-dkms-firmware:1:31.50.0.0.31500000-2390945.24.04- AMD repository:
https://repo.radeon.com/amdgpu/31.50/ubuntu noble main - ROCm Core packages:
10.0.0-4, targetgfx90a - AMD SMI: ROCm
10.0.0, amdgpu7.1.3.31500000 - GPU: 4 x AMD Instinct MI250 cards / 8 x Aldebaran PCI functions (
1002:740c) - Server: Supermicro AS-4124GQ-TNMI
- CPU: 2 x AMD EPYC 7443, 96 logical CPUs
- Memory: 1 TiB
- Compiler: GCC
13.3.0 - GNU Make:
4.3-4.1build2 - DKMS:
3.0.11-1ubuntu13
The AMD DKMS top-level Makefile expands its kernel build to make -j$(nproc), so this host runs the build with -j96.
The currently running stack is healthy on 6.8.0-142-generic:
amdgpu/7.1.3-2390945.24.04, 6.8.0-142-generic, x86_64: installed
filename: /lib/modules/6.8.0-142-generic/updates/dkms/amdgpu.ko.zst
version: 7.1.3.31500000
vermagic: 6.8.0-142-generic SMP preempt mod_unload modversions
How it was triggered
Ubuntu unattended-upgrades installed the following kernel packages on 2026-10-10:
linux-image-6.8.0-146-generic 6.8.0-146.146
linux-headers-6.8.0-146 6.8.0-146.146
linux-headers-6.8.0-146-generic 6.8.0-146.146
linux-modules-6.8.0-146-generic 6.8.0-146.146
linux-modules-extra-6.8.0-146-generic 6.8.0-146.146
The normal kernel post-install hooks then invoked:
dkms autoinstall -k 6.8.0-146-generic
which invokes the package's configured build:
'make' KERNELVER=6.8.0-146-generic
No custom kernel, DKMS patch, compiler override, or manual source modification was involved.
Failure signature 1: raw_atomic_fetch_add
One preserved DKMS report failed consistently at mmhub_v3_3.o:
CC [M] /tmp/amd.REDACTED/amd/amdgpu/mmhub_v3_3.o
In file included from ./include/linux/atomic.h:80,
...
./include/linux/atomic/atomic-arch-fallback.h: In function 'raw_atomic_fetch_add':
./include/linux/atomic/atomic-arch-fallback.h:675:2: error: #error "Unable to define raw_atomic_fetch_add"
675 | #error "Unable to define raw_atomic_fetch_add"
| ^~~~~
./include/linux/atomic/atomic-arch-fallback.h:677:1: error: no return statement in function returning non-void [-Werror=return-type]
...
./include/linux/atomic/atomic-arch-fallback.h: In function 'raw_atomic_fetch_add_acquire':
./include/linux/atomic/atomic-arch-fallback.h:702:2: error: #error "Unable to define raw_atomic_fetch_add_acquire"
...
./include/linux/atomic/atomic-arch-fallback.h: In function 'raw_atomic_fetch_add_release':
./include/linux/atomic/atomic-arch-fallback.h:728:2: error: #error "Unable to define raw_atomic_fetch_add_release"
...
./include/linux/atomic/atomic-arch-fallback.h: In function 'raw_atomic_fetch_add_relaxed':
./include/linux/atomic/atomic-arch-fallback.h:751:2: error: #error "Unable to define raw_atomic_fetch_add_relaxed"
...
cc1: some warnings being treated as errors
make[4]: *** [scripts/Makefile.build:243: /tmp/amd.REDACTED/amd/amdgpu/mmhub_v3_3.o] Error 1
make[3]: *** [scripts/Makefile.build:481: /tmp/amd.REDACTED/amd/amdgpu] Error 2
make[2]: *** [/usr/src/linux-headers-6.8.0-146-generic/Makefile:1946: /tmp/amd.REDACTED] Error 2
make[1]: *** [Makefile:240: __sub-make] Error 2
make: *** [Makefile:54: modules] Error 2
The following header is byte-identical between the working and failing kernel header packages:
c5bed3e24d0ca84e7548a2e3943ceab666337a72944ac82377bd99878f5dbdd9 6.8.0-142/include/linux/atomic/atomic-arch-fallback.h
c5bed3e24d0ca84e7548a2e3943ceab666337a72944ac82377bd99878f5dbdd9 6.8.0-146/include/linux/atomic/atomic-arch-fallback.h
Failure signature 2: GNU make heap corruption
On another retry, Ubuntu generated /var/crash/_usr_bin_make.0.crash:
ProblemType: Crash
Date: Mon Oct 12 06:21:54 2026 UTC
Package: make 4.3-4.1build2
ExecutablePath: /usr/bin/make
Signal: 6 (SIGABRT)
Uname: Linux 6.8.0-142-generic x86_64
ProcCmdline: make -f ./scripts/Makefile.build obj=/tmp/amd.REDACTED/amd/amdgpu need-builtin= need-modorder=1
Strings recovered from its core dump contain:
double free or corruption (out)
kernelver=6.8.0-146-generic
module=amdgpu
HOSTCC=gcc-13
KERNELRELEASE=6.8.0-146-generic
module_version=7.1.3-2390945.24.04
This is the actual make process aborting; the generic DKMS message about a segmentation fault is not sufficient to describe this failure.
Failure signature 3: DKMS Apport hook crash
The DKMS Apport hook also crashes while handling some failed attempts:
Traceback (most recent call last):
File "/usr/share/apport/package-hooks/dkms_packages.py", line 87, in <module>
if 'Segmentation fault' in report['DKMSBuildLog']:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: a bytes-like object is required, not 'str'
This appears to be a separate Ubuntu reporting bug, but it obscures the underlying AMD DKMS failure.
Hard-lock evidence
The final boot's journal shows apt-daily-upgrade starting at 06:01:06 UTC. The dpkg log reaches the DKMS build and then stops mid-line:
2026-10-13 06:01:54 UTC: unattended-upgrade starts the next dpkg transaction
Setting up linux-headers-6.8.0-146-generic (6.8.0-146.146) ...
* dkms: running auto installation service for kernel 6.8.0-146-generic
Building module:
Cleaning build area...
'make' KERNELVER=6.8.0-146-generic.........
There are no later entries in that boot after 06:02:03 UTC, no shutdown record, and the next boot begins at 19:08:15 UTC after a power cycle.
Additional checks after recovery:
EDAC mc0 ce_count: 0
EDAC mc0 ue_count: 0
EDAC mc1 ce_count: 0
EDAC mc1 ue_count: 0
pstore: empty
swap in use after recovery: 0
The persistent kernel journal contains no OOM, MCE, uncorrectable EDAC error, panic, watchdog lockup, or amdgpu reset/timeout immediately preceding the freeze.
Resulting package/boot state
linux-headers-6.8.0-146-generic 6.8.0-146.146 iF
linux-image-6.8.0-146-generic 6.8.0-146.146 iF
/boot/vmlinuz-6.8.0-146-generic exists
/boot/initrd.img-6.8.0-146-generic missing
The previous kernel remains bootable and has a valid initramfs:
/boot/vmlinuz-6.8.0-142-generic
/boot/initrd.img-6.8.0-142-generic
Non-deterministic history
This package also failed several automatic builds against 6.8.0-142-generic on 2026-09-25 through 2026-09-27, sometimes producing the generic DKMS segmentation-fault classification. A later retry on 2026-09-28 completed successfully without changing the amdgpu-dkms package, and that module is currently running.
Against 6.8.0-146-generic, retries from 2026-10-10 through 2026-10-13 produced different observable failure signatures, culminating in the host hard-lock. This variability is why I suspect a parallel-build race, memory corruption in the build process, or another non-deterministic defect rather than only a deterministic kernel API incompatibility. This is an inference, not a confirmed root cause.
Expected behavior
amdgpu-dkmsshould build reproducibly against a supported Ubuntu kernel; or- if the kernel is unsupported/incompatible, the build should fail deterministically and cleanly without aborting
make, hard-locking the host, or leaving a new default kernel without an initramfs.
Questions
- Is
amdgpu-dkms 1:7.1.3.31500000-2390945.24.04expected to support Ubuntu kernel6.8.0-146.146? - Are there known races or excessive-parallelism issues in this DKMS source when the top-level build uses
make -j96? - Is the
raw_atomic_fetch_adderror known to occur non-deterministically during parallel builds? - Would AMD like a controlled comparison using capped job counts (
-j1,-j8, etc.) on a disposable test host? I will not run that experiment on this production host because the build has already caused a hard lock. - Which additional artifacts would be most useful? I can provide the redacted Apport reports and core metadata, but I have intentionally not uploaded the full core dump because it may contain host data.
Related reports
- ROCm/amdgpu#220: similar DKMS/compiler crash symptoms after a kernel upgrade, but different OS, kernel, and driver versions.
- ROCm/ROCm#4164: similar
amdgpu-dkmsfailure class on Ubuntu with a 6.8 kernel, but a different driver/kernel combination.
- Ngôn ngữ chính
- C
- Star
- 470
- Fork
- 150
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Chuẩn bị môi trường
Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của ROCm/amdgpu
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 74/100
-
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 25/100
-
amdgpu unified RAS: GFX correctable errors aggregated by MP5 firmware are dropped on MI450 (no GFX error count, no CPER)Có thể đã có người làm @tpatward-amd đã nhận 18 ngày trước. Đang mở
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 28/100
-
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 48/100
-
Độ khó 5/5 Hơn một tuần Mức phù hợp với người mới 30/100
Issue tương tự
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
InukaAsith/TwinDesk#84 ·
Maintainer thường phản hồi trong vòng 2 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
CanadaHonk/porffor#426 ·
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 65/100
FujiNetWIFI/fujinet-firmware#2221 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 77/100
raysan5/raylib.com#132 ·
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100