Shadow FDs: point-in-time snapshots, 256MB cap, no write-back
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- c
- Domain
- operating-systems
Research direction
Start by reading src/shadow-fd.h and src/shadow-fd.c for the 256MB cap and snapshot path, then inspect src/seccomp-dispatch.c and tests/guest/errno-test.c. The issue contains several proposals with different scopes, so first narrow it to one prioritized change. Done should include bounded behavior and the relevant documentation or test updates for that selected change.
Written by the indexing model from the issue text.
Description
Rationale
Shadow FDs copy LKL file contents into a memfd at open time, enabling native host kernel mmap (critical for dynamic linkers that mmap .so files with MAP_PRIVATE). Three inherent limitations:
- Point-in-time: if the LKL file changes after open, the shadow is stale. The guest sees old data.
- 256MB cap (
KBOX_SHADOW_MAX_SIZE): files larger than 256MB cannot be shadowed, returning EFBIG at open time. - O_RDONLY only: files opened for writing use virtual FDs and cannot be mmapped by the host kernel.
Additionally, shadowed files become writable from the host side since the tracee receives a memfd where write() can succeed despite the O_RDONLY open mode -- a known semantic quirk.
These limitations are acceptable for the primary use case (dynamic linker .so loading) but restrict workloads involving large read-only datasets or files that change after open.
Proposed Changes (priority order)
- Configurable size cap: allow users to set the shadow size limit via environment variable or CLI flag, with 256MB as default. Users with sufficient memory can raise it for specific workloads. Include an upper bound to prevent OOM.
- Document limitations: add user-facing documentation explaining mmap behavior, snapshot semantics, and the size cap.
- Staleness detection: compare LKL file mtime/size with snapshot metadata on fstat. Useful for observability and debugging only -- cannot repair already-established mmaps.
- Future exploration: lazy demand-paged population via userfaultfd. Significant scope increase (new kernel dependency, portability concerns, security implications).
Considerations
- Current design is correct for its intended purpose; avoid over-engineering the hot path
- Raising the cap without limits risks OOM on the host -- enforce an upper bound or memory budget
- Re-snapshotting on staleness could cause subtle bugs if the guest has already mmapped pages from the old snapshot
- Write-back would contradict the O_RDONLY open mode invariant
- The host-writable memfd quirk is documented but not enforced; consider
memfd_createwithMFD_NOEXEC_SEALorF_SEAL_WRITEif the kernel supports it
References
src/shadow-fd.h:KBOX_SHADOW_MAX_SIZEdefinition (256MB)src/shadow-fd.c: EFBIG enforcement; snapshot creation (pread64 loop into memfd)src/seccomp-dispatch.c: O_RDONLY gating for shadow creationtests/guest/errno-test.c: documents the host-writable memfd quirk
- Dominant language
- C
- Stars
- 95
- Forks
- 27
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from sysprog21/kbox
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Similar issues
-
level/task module/gcp type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
hapostgres/pg_auto_failover#1190 ·
-
docs
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
P3 sonic-vpp
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
sonic-net/sonic-buildimage#29662 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
spack/spack-packages#6586 ·