2.0: asks from running several agents on one codebase across machines

Đang mở
  • Có gì không ổn?
#632 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
25/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
linux, postgresql, rust, sqlite
Lĩnh vực
ai-infra-agents, backend, databases, distributed-systems

Hướng nghiên cứu

The issue describes several possible improvements and gives a publish race reproduction, but does not identify a single agreed change. For the publish safeguard, start by reading the branch.publish documentation and the implementation and tests for publishing; the repro provides the sequence to reproduce the stale publish. Done would mean an opt-in or expected-base check refuses publication after main moves, with a test showing the other Branches' files remain on main.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

We work on a setup where agents from different vendors (Claude Code, Codex and others) work on one codebase from different machines, and a person approves what lands on main. On 11-08 we ran layerfs_v2_0_0 at 716197c4 inside E2B sandboxes to see how far 2.0 gets us, and re-ran the publish repro below at 833d04cf the same day.

What worked:

  • No Docker at run time. Native Linux Sandboxes ran in the microVM. We built the image archive once with tools/linux_bundle.py.
  • Forks of a 121 MB checkout took 8–10 ms each through the CLI. Two agents changed the same README line, and branch.merge refused in 15 ms with the whole payload (content, the line, both sides, sources, mergeBase). Nothing was created.
  • Two homes on two VMs shared one PostgreSQL 17 Store. The second read the first one's Branch, and merging across them gave the same Conflict payload.
  • The sqlite Store, the daemon and the Sandbox came through a full VM pause and resume. Mounts afterwards took 13–73 ms.
  • A Claude Code turn ran inside a Workspace through layerfs-sandbox-helper exec and committed.

What we'd still need, most important first:

  1. An opt-in for branch.publish to refuse when main moved since the Branch's base. We know the publish replaces main's view, as documented. The README covers it since #469. But with several agents publishing, one stale publish took two other agents' files off main, and overwrote_count: 1 was all that showed it. A flag, or the base the caller expects, would make that a refusal. Repro below.
  2. Expected heads on branch.merge. A person reads the Conflict and decides, and then the merge runs again. If branch.merge took the heads the person saw and refused when either had moved, a decision couldn't land on a tree nobody looked at. Today branch/merge.rs takes message and units only. #487's guarded head move is close to this.
  3. Env passthrough for layerfs workspace exec. An agent's turn needs a credential. We found no supported way to give one to a Workspace command. Calling the Sandbox's helper directly works, because it inherits the caller's env. #534 is the macOS side of something similar.
  4. Git in a Workspace without index conflicts. Coding agents run git status all the time, and that rewrites .git/index. On main at 582bc786 (10-28), every different-file merge between two such agents then refused on /.git/index, kind binary, unless we set GIT_OPTIONAL_LOCKS=0. A documented recipe would fix it for everyone. Not re-measured on 2.0; related to #514.
  5. A readable PostgreSQL error and a stated version. On 2.0 we got making the PostgreSQL Store: Malformed in two cases: from Postgres 15, and from a Postgres with an untrusted TLS certificate. 17 works, and sslmode=disable got past the TLS one.
  6. Linux Sandboxes without root. limitations.md says namespaces need root for FUSE. That's fine in a VM, where we ran it. It rules out a person's own Linux machine and shared hosts.
  7. Upkeep for a PostgreSQL Store: backups and project remove, both listed as not built. Also a way to fetch the Sandbox bundle, which we built by hand.
  8. Events on commit, merge, publish and refusal, so the host can show the approving person who changed what. In the 2.0 docs we found progress and exec-output events only.

We'd like to keep testing 2.0 builds as they land, in E2B and on plain Linux, and post what we find with the commands, output and ref. Your roadmap lists E2B under D6, and we're happy to share our setup notes for it. If any of these is already planned, a pointer is plenty. And if you're short on time for any of them, we'd be glad to help build it, starting with the publish option, if you'd take a PR.

Repro for 1: a stale publish removes other Branches' files from main

Ref: layerfs_v2_0_0 at 833d04cf44e8e747929d50ab1170a48260d9381c (layerfs 2.0.0), built with Rust 1.85.1: cargo build --release --locked -p layerfs-cli -p layerfs-daemon -p layerfs-sandbox-backend --features layerfs-daemon/postgresql,layerfs-cli/postgresql. Environment: an E2B sandbox (Firecracker microVM), Debian GNU/Linux 13 (trixie), kernel 6.1.189+, 4 vCPU, 4 GiB, root in the guest, /dev/fuse present. The Store was sqlite on the native machine. The project was a 122 MB git checkout. We got the same result earlier on 716197c4, and the same overwroteCount: 1 through the 1.x SDK on main at 67310175.

export LAYERFS_HOME=/srv/lh
layerfs store create s1
layerfs project init s1/c /root/repo
layerfs branch fork s1/c a main
layerfs branch fork s1/c c main
layerfs branch fork s1/c d main
layerfs machine update native --daemon <build>/layerfs-daemon
layerfs sandbox create s1/c sb native
# for each Branch: workspace mount sb <branch>, workspace exec, workspace commit, workspace unmount
#   a: sed -i '3s/.*/LINE-FROM-A/' README.md
#   c: echo c > from-c.txt
#   d: echo d > from-d.txt
layerfs branch merge s1/c cd c d    # {"kind":"Merged","fast_forward":false,"conflicts":[]}
layerfs branch publish s1/c cd      # {"kind":"Added","overwrote_count":0}
layerfs file read s1/c main /from-c.txt
# {"kind":"ok","snapshot":"main@9f7ebe737f99",...,"text":"c\n",...}
layerfs branch publish s1/c a       # {"kind":"Added","overwrote_count":1}
layerfs file read s1/c main /from-c.txt
# {"code":"NoSuchPath","message":"/from-c.txt does not exist at main@2b14ee15a1f0",...}
layerfs file read s1/c main /from-d.txt
# {"code":"NoSuchPath","message":"/from-d.txt does not exist at main@2b14ee15a1f0",...}

Expected, for this use case: the second publish is refused because main moved after a was forked, or there's an option that makes it refuse. Actual, as documented: Added, and both /from-c.txt and /from-d.txt are gone from main. overwrote_count is 1 because it counts Layers, and the merged cd publish was one Layer.

I work on Commonly (commonly.me). Written with AI assistance; the repro was run as described.

Ngôn ngữ chính
Rust
Star
2.1k
Fork
110
Merge trung bình
23 giờ 52 phút
Pull request đã merge (30 ngày)
138

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của Ephemeral-AI-Lab/layerfs

Tất cả issue của Ephemeral-AI-Lab/layerfs

Issue tương tự

Thêm issue về Rust

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.