Shadow FDs: point-in-time snapshots, 256MB cap, no write-back

Open
#11 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
c

Research direction

Start by reading src/shadow-fd.h and src/shadow-fd.c for the 256MB cap and snapshot path, then inspect src/seccomp-dispatch.c and tests/guest/errno-test.c. The issue contains several proposals with different scopes, so first narrow it to one prioritized change. Done should include bounded behavior and the relevant documentation or test updates for that selected change.

Written by the indexing model from the issue text.

Description

Rationale

Shadow FDs copy LKL file contents into a memfd at open time, enabling native host kernel mmap (critical for dynamic linkers that mmap .so files with MAP_PRIVATE). Three inherent limitations:

  1. Point-in-time: if the LKL file changes after open, the shadow is stale. The guest sees old data.
  2. 256MB cap (KBOX_SHADOW_MAX_SIZE): files larger than 256MB cannot be shadowed, returning EFBIG at open time.
  3. O_RDONLY only: files opened for writing use virtual FDs and cannot be mmapped by the host kernel.

Additionally, shadowed files become writable from the host side since the tracee receives a memfd where write() can succeed despite the O_RDONLY open mode -- a known semantic quirk.

These limitations are acceptable for the primary use case (dynamic linker .so loading) but restrict workloads involving large read-only datasets or files that change after open.

Proposed Changes (priority order)

  1. Configurable size cap: allow users to set the shadow size limit via environment variable or CLI flag, with 256MB as default. Users with sufficient memory can raise it for specific workloads. Include an upper bound to prevent OOM.
  2. Document limitations: add user-facing documentation explaining mmap behavior, snapshot semantics, and the size cap.
  3. Staleness detection: compare LKL file mtime/size with snapshot metadata on fstat. Useful for observability and debugging only -- cannot repair already-established mmaps.
  4. Future exploration: lazy demand-paged population via userfaultfd. Significant scope increase (new kernel dependency, portability concerns, security implications).

Considerations

  • Current design is correct for its intended purpose; avoid over-engineering the hot path
  • Raising the cap without limits risks OOM on the host -- enforce an upper bound or memory budget
  • Re-snapshotting on staleness could cause subtle bugs if the guest has already mmapped pages from the old snapshot
  • Write-back would contradict the O_RDONLY open mode invariant
  • The host-writable memfd quirk is documented but not enforced; consider memfd_create with MFD_NOEXEC_SEAL or F_SEAL_WRITE if the kernel supports it

References

  • src/shadow-fd.h : KBOX_SHADOW_MAX_SIZE definition (256MB)
  • src/shadow-fd.c : EFBIG enforcement; snapshot creation (pread64 loop into memfd)
  • src/seccomp-dispatch.c : O_RDONLY gating for shadow creation
  • tests/guest/errno-test.c : documents the host-writable memfd quirk
Dominant language
C
Stars
95
Forks
27
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from sysprog21/kbox

All issues in sysprog21/kbox

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.