Wrong cpu usage reported in top and other tools that use procps
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- c, docker, linux
- Domain
- devops, infrastructure, operating-systems
Research direction
Start by running the provided C lseek reproducer inside Ubuntu under sysbox-runc and compare it with runc. Trace the sysbox-fs handler serving /proc/uptime, then separately inspect the uptime and /proc//stat values used by procps. Done means repeated reads work correctly and top and ps report consistent CPU and elapsed-time values without stderr errors.
Written by the indexing model from the issue text.
Description
Summary
/proc/uptime inside a sysbox container is backed by sysbox-fs (FUSE) and rejects lseek() with ESPIPE, even though stat() reports it as a regular file. Every other emulated node I tested (/proc/swaps, /proc/stat, /proc/sys/*, /sys/kernel/*) is seekable, so this looks like a per-handler inconsistency inside sysbox-fs rather than a FUSE limitation.
This silently breaks libprocps. Its FILE_TO_BUF macro opens the file once and then does lseek(fd, 0, SEEK_SET); read(...) on every refresh, without checking lseek's return value and treating only read() < 0 as an error:
lseek(fd, 0L, SEEK_SET); /* fails with ESPIPE, result ignored */
if ((local_n = read(fd, buf, sizeof buf - 1)) < 0) { ... } /* == 0 is not an error */
buf[local_n] = '\0'; /* -> empty string */
So the first read works and every subsequent read returns 0 bytes. uptime() then hits sscanf(...) < 2, prints bad data in /proc/uptime to stderr and returns without setting its output parameter.
Visible consequences:
top's%CPUcolumn becomes meaningless - a process using exactly one core is reported as600.0%.psreports%CPU 0.0andELAPSED 00:00for every process.bad data in /proc/uptimespam on stderr.
Related issue #1016 (which reports the stderr message, and whose attached top output shows containerd at exactly 600.0%).
Environment
| Host | Ubuntu 22.04.3 LTS, kernel 5.15.0-163-generic, x86_64 |
| Docker | 28.4.0, build d8eb465 |
| sysbox-runc | 0.7.1 CE, commit 081856cc5d17e7095f066b08d0eca6bb0b515c47 |
| Container image | ubuntu:22.04 + procps (procps-ng 3.3.17) |
Also observed on an unrelated sysbox host (different kernel, 512-CPU machine), where top reported 2300% / 3500% instead of 600% - the bogus value is deterministic per environment but differs between environments.
Minimal reproduction (no build required)
docker run --rm --runtime=sysbox-runc ubuntu:22.04 sh -c \
'apt-get update -qq && apt-get install -y -qq procps >/dev/null; top -b -n 2 -d 1 -p 1 >/dev/null'
Output:
bad data in /proc/uptime
bad data in /proc/uptime
The same command with --runtime=runc prints nothing.
Reproduction of the underlying lseek failure
/* uptime_seek.c - reproduces libprocps' FILE_TO_BUF access pattern */
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
int main(void)
{
char buf[512];
int fd = open("/proc/uptime", O_RDONLY);
for (int i = 0; i < 3; i++) {
off_t pos = lseek(fd, 0L, SEEK_SET);
ssize_t n = read(fd, buf, sizeof buf - 1);
buf[n > 0 ? n : 0] = '\0';
buf[strcspn(buf, "\n")] = '\0';
printf("refresh %d: lseek=%-14s read=%3zd %s\n", i,
pos == (off_t)-1 ? strerror(errno) : "ok", n,
n > 0 ? buf : "EMPTY");
}
return 0;
}
--runtime=sysbox-runc:
refresh 0: lseek=Illegal seek read= 10 0.14 0.14
refresh 1: lseek=Illegal seek read= 0 EMPTY
refresh 2: lseek=Illegal seek read= 0 EMPTY
--runtime=runc:
refresh 0: lseek=ok read= 25 25419940.40 152274893.47
refresh 1: lseek=ok read= 25 25419940.40 152274893.47
refresh 2: lseek=ok read= 25 25419940.40 152274893.47
Only /proc/uptime is affected. Same program, same container:
/proc/uptime lseek=Illegal seek read= 0 EMPTY
/proc/swaps lseek=ok read= 81 Filename Type Size ...
/proc/stat lseek=ok read=511 cpu 6610034 663831 ...
/proc/meminfo lseek=ok read=511 MemTotal: 49328972 kB
/proc/loadavg lseek=ok read= 24 0.33 0.29 0.12 3/404 63
/proc/cpuinfo lseek=ok read=511 processor : 0
/proc/sys/kernel/hostname lseek=ok read= 22 domain.ompartners.com
/sys/kernel/mm/transparent_hugepage/enabled lseek=ok read= 23 always [madvise] never
(/proc/swaps and /proc/sys/kernel/hostname return sysbox-emulated content here, so they are served by sysbox-fs too - and they are seekable.)
The mount and the file type:
$ grep -w /proc/uptime /proc/mounts
sysboxfs /proc/uptime fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other 0 0
$ stat -c 'type=%F size=%s' /proc/uptime
type=regular file size=4096
Impact on top %CPU
Test program: 9 threads, of which exactly one busy-loops, so true CPU usage is exactly 100% of one core.
/* spin.c - gcc -static -pthread -o spin spin.c */
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
static void *idle(void *a) { (void)a; for (;;) sleep(3600); return NULL; }
int main(void)
{
for (int i = 0; i < 8; i++) { pthread_t t; pthread_create(&t, NULL, idle, NULL); }
printf("pid=%d threads=9 busy=1\n", getpid());
fflush(stdout);
volatile double x = 0;
for (;;) x += 1.0;
}
| runtime | threads | ground truth (/proc/<pid>/stat utime+stime delta) |
top %CPU |
top TIME+ |
|---|---|---|---|---|
runc |
9 | 105% | 99.7% | 0:05.14 |
sysbox-runc |
9 | 106% | 600.0% | 0:05.19 |
Fully deterministic - 600.0 on five consecutive runs under sysbox, 100.0 on five consecutive runs under runc.
Note that top's own TIME+ column stays correct (it is derived from cumulative ticks and never touches /proc/uptime), so top visibly contradicts itself: TIME+ advances ~3 s per 3 s of wall clock while %CPU prints 600.
Impact on ps - a second, related inconsistency
sysbox-runc: ps -o pid,nlwp,pcpu,time,etime -> 70 9 0.0 00:00:03 00:00
runc: ps -o pid,nlwp,pcpu,time,etime -> 9 9 99.6 00:00:02 00:03
ps computes ELAPSED = uptime - starttime/HZ. sysbox virtualizes /proc/uptime to container uptime, but /proc/<pid>/stat field 22 (starttime) is still host-boot relative:
container /proc/uptime : 2.17 s
/proc/1/stat starttime : 2542012294 ticks = 25420122 s (host-boot relative)
=> ELAPSED = 2.17 - 25420122 -> negative -> clamped to 0
=> %CPU = cputime / elapsed -> 0.0
This one is independent of the lseek bug and would remain even if seeking were fixed.
- Dominant language
- Shell
- Stars
- 3.9k
- Forks
- 230
- Avg merge
- 11h 18m
- Merged PRs (30d)
- 2
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from nestybox/sysbox
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
0.7.1: /sys/class/net inside a sysbox container lists the host's interfaces, not the container'sOpen
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
Similar issues
-
area/documentation squad/marvin
Difficulty 1/5 Under an hour Newbie friendliness 90/100
rancher/stackstate-product-docs#443 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
getumbrel/umbrel-apps#6142 ·
Maintainers usually reply within 2 days
-
triage/confirmed
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
agentscope-ai/agentscope#3030 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
WingedGuardian/GENesis-AGI#2599 ·
Maintainers usually reply within 1 day
-
bug good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
petry-projects/.github-private#1981 · 1 comment ·
Maintainers usually reply within 1 day