cpufreq collector hangs forever on arm64 with cppc_cpufreq, leaking a goroutine and an fd per scrape

从 procfs/sysfs/system_cpu.go 开始，重点关注 SystemCpufreq 和 parseCpufreqCpuinfo，然后在 arm64 上使用 cppc_cpufreq 重现 cpufreq collector，同时观察 goroutine、文件描述符和 scrape 完成情况。完成的标准是：不可读取或卡住的 policy 文件不会导致 scrape 永久阻塞或泄漏资源；验证 collector 的失败行为和指标。

What did you do?

Ran the cpufreq collector on two arm64 machines (NVIDIA GB10 / DGX Spark) whose scaling driver is cppc_cpufreq, with 20 CPUs and 20 cpufreq policies.

The collector hangs permanently. It does not merely get slow: a goroutine blocked at process start was still blocked 1229 minutes (20.5 h) later, and every subsequent scrape leaks another goroutine and another open file descriptor.

Observed via Grafana Alloy's embedded prometheus.exporter.unix (Alloy v1.18.1, arm64, go1.26.5, github.com/prometheus/procfs v0.21.0). The stack is entirely inside node_exporter/procfs, but to be upfront: I have not tested standalone node_exporter on this hardware, so I cannot claim the standalone binary reproduces it — I would expect it to, since nothing in the path is Alloy-specific.

What did you expect to see?

The scrape completing, or at worst the collector failing and reporting node_scrape_collector_success{collector="cpufreq"} 0.

What did you see instead?

Roughly 40–60% of scrapes never return. From /debug/pprof/goroutine?debug=2:

goroutine 185317 [IO wait, 1229 minutes]:
internal/poll.runtime_pollWait(0xeb287c0f7e00, 0x72)
	/usr/local/go/src/runtime/netpoll.go:351 +0xa0
internal/poll.(*pollDesc).wait(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*FD).Read(...)
	/usr/local/go/src/internal/poll/fd_unix.go:165 +0x234
os.(*File).Read(...)
	/usr/local/go/src/os/file.go:144 +0x68
os.ReadFile(...)
	/usr/local/go/src/os/file.go:871 +0xb4
github.com/prometheus/procfs/internal/util.ReadUintFromFile(...)
	/go/pkg/mod/github.com/prometheus/[email protected]/internal/util/parse.go:85 +0x20
github.com/prometheus/procfs/sysfs.parseCpufreqCpuinfo(...)
	/go/pkg/mod/github.com/prometheus/[email protected]/sysfs/system_cpu.go:284 +0x154
github.com/prometheus/procfs/sysfs.FS.SystemCpufreq.func1()
	/go/pkg/mod/github.com/prometheus/[email protected]/sysfs/system_cpu.go:249 +0x40
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:93 +0x4c

It is stuck in runtime_pollWait — the descriptor was registered with the netpoller and the kernel never marks it readable, so os.ReadFile never returns. Because SystemCpufreq waits on its errgroup, the whole NodeCollector.Collect blocks with it and the HTTP scrape times out.

The concurrency appears to be what triggers it. SystemCpufreq reads every policy in parallel (system_cpu.go:249), deliberately, per the comment about the kernel's intentional 50 ms per-CPU delay. Reading the same files sequentially from a shell is fast and never hangs:

$ for c in /sys/devices/system/cpu/cpu*/cpufreq/scaling_cur_freq; do timeout 3 cat $c; done
# all 20 return in < 200 ms total, no process ever enters D state

So a manual check "proves" the machine is healthy, which made this quite misleading to track down.

Impact

It is not only missing metrics — it is an unbounded resource leak, since each hung scrape leaves a goroutine and an fd behind forever. After ~21 h on a 120 s scrape interval:

	host A	host B	unaffected x86 host
alloy goroutines	2557	2128	313
open fds / pointing at cpufreq	1083 / 740	820 / 543	—
RSS	~800 MB	~800 MB	—
scrapes hanging (of 12)	5	4	0

Restarting the process reclaims it (goroutines 2612 → 273, cpufreq fds 760 → 0, RSS 800 → 386 MB), but the leak starts over immediately.

Workaround

Disabling the collector (--no-collector.cpufreq, or disable_collectors = ["cpufreq"] in Alloy) fixes it completely and instantly — 0 hung scrapes out of 12, max scrape time 30 s → 0.13 s.

Worth noting the trigger only became visible after a same-day upgrade of both the kernel (6.17.0-1021 → 6.17.0-1029-nvidia) and Alloy (1.17.x → 1.18.1) on these two hosts, so I cannot cleanly attribute it to either one; both hosts are the only ones in the fleet with both changes. What is clear is where it blocks. A 50 ms/CPU kernel delay cannot produce a 20-hour block, so something on the kernel/firmware side is failing to ever complete the read, and the parallel access is what exposes it. cppc_cpufreq reads go through the PCC firmware mailbox, which is a plausible place for concurrent access to serialize badly.

Even if the kernel is at fault, node_exporter arguably should not be able to leak unboundedly because of one unreadable sysfs file — a timeout or context on the per-policy reads would contain the damage.

System information

Linux 6.17.0-1029-nvidia aarch64 (Ubuntu 24.04, #29-Ubuntu SMP PREEMPT_DYNAMIC)
NVIDIA GB10 (DGX Spark class), 20 CPUs, 20 cpufreq policies
scaling_driver: cppc_cpufreq

Reproduced on two physically different machines from two different OEMs (Lenovo ThinkStation PGX and ASUS Ascent GX10), so it is not a single-vendor firmware quirk.

node_exporter version

Embedded in Grafana Alloy v1.18.1 (revision 6012ec4, goarch=arm64, goversion=go1.26.5), which vendors github.com/prometheus/procfs v0.21.0.