2.0: asks from running several agents on one codebase across machines

Abierto
  • ¿Algo no está bien?
#632 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
linux, postgresql, rust, sqlite
Área
ai-infra-agents, backend, databases, distributed-systems

Línea de trabajo

The issue describes several possible improvements and gives a publish race reproduction, but does not identify a single agreed change. For the publish safeguard, start by reading the branch.publish documentation and the implementation and tests for publishing; the repro provides the sequence to reproduce the stale publish. Done would mean an opt-in or expected-base check refuses publication after main moves, with a test showing the other Branches' files remain on main.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

We work on a setup where agents from different vendors (Claude Code, Codex and others) work on one codebase from different machines, and a person approves what lands on main. On 11-08 we ran layerfs_v2_0_0 at 716197c4 inside E2B sandboxes to see how far 2.0 gets us, and re-ran the publish repro below at 833d04cf the same day.

What worked:

  • No Docker at run time. Native Linux Sandboxes ran in the microVM. We built the image archive once with tools/linux_bundle.py.
  • Forks of a 121 MB checkout took 8–10 ms each through the CLI. Two agents changed the same README line, and branch.merge refused in 15 ms with the whole payload (content, the line, both sides, sources, mergeBase). Nothing was created.
  • Two homes on two VMs shared one PostgreSQL 17 Store. The second read the first one's Branch, and merging across them gave the same Conflict payload.
  • The sqlite Store, the daemon and the Sandbox came through a full VM pause and resume. Mounts afterwards took 13–73 ms.
  • A Claude Code turn ran inside a Workspace through layerfs-sandbox-helper exec and committed.

What we'd still need, most important first:

  1. An opt-in for branch.publish to refuse when main moved since the Branch's base. We know the publish replaces main's view, as documented. The README covers it since #469. But with several agents publishing, one stale publish took two other agents' files off main, and overwrote_count: 1 was all that showed it. A flag, or the base the caller expects, would make that a refusal. Repro below.
  2. Expected heads on branch.merge. A person reads the Conflict and decides, and then the merge runs again. If branch.merge took the heads the person saw and refused when either had moved, a decision couldn't land on a tree nobody looked at. Today branch/merge.rs takes message and units only. #487's guarded head move is close to this.
  3. Env passthrough for layerfs workspace exec. An agent's turn needs a credential. We found no supported way to give one to a Workspace command. Calling the Sandbox's helper directly works, because it inherits the caller's env. #534 is the macOS side of something similar.
  4. Git in a Workspace without index conflicts. Coding agents run git status all the time, and that rewrites .git/index. On main at 582bc786 (10-28), every different-file merge between two such agents then refused on /.git/index, kind binary, unless we set GIT_OPTIONAL_LOCKS=0. A documented recipe would fix it for everyone. Not re-measured on 2.0; related to #514.
  5. A readable PostgreSQL error and a stated version. On 2.0 we got making the PostgreSQL Store: Malformed in two cases: from Postgres 15, and from a Postgres with an untrusted TLS certificate. 17 works, and sslmode=disable got past the TLS one.
  6. Linux Sandboxes without root. limitations.md says namespaces need root for FUSE. That's fine in a VM, where we ran it. It rules out a person's own Linux machine and shared hosts.
  7. Upkeep for a PostgreSQL Store: backups and project remove, both listed as not built. Also a way to fetch the Sandbox bundle, which we built by hand.
  8. Events on commit, merge, publish and refusal, so the host can show the approving person who changed what. In the 2.0 docs we found progress and exec-output events only.

We'd like to keep testing 2.0 builds as they land, in E2B and on plain Linux, and post what we find with the commands, output and ref. Your roadmap lists E2B under D6, and we're happy to share our setup notes for it. If any of these is already planned, a pointer is plenty. And if you're short on time for any of them, we'd be glad to help build it, starting with the publish option, if you'd take a PR.

Repro for 1: a stale publish removes other Branches' files from main

Ref: layerfs_v2_0_0 at 833d04cf44e8e747929d50ab1170a48260d9381c (layerfs 2.0.0), built with Rust 1.85.1: cargo build --release --locked -p layerfs-cli -p layerfs-daemon -p layerfs-sandbox-backend --features layerfs-daemon/postgresql,layerfs-cli/postgresql. Environment: an E2B sandbox (Firecracker microVM), Debian GNU/Linux 13 (trixie), kernel 6.1.189+, 4 vCPU, 4 GiB, root in the guest, /dev/fuse present. The Store was sqlite on the native machine. The project was a 122 MB git checkout. We got the same result earlier on 716197c4, and the same overwroteCount: 1 through the 1.x SDK on main at 67310175.

export LAYERFS_HOME=/srv/lh
layerfs store create s1
layerfs project init s1/c /root/repo
layerfs branch fork s1/c a main
layerfs branch fork s1/c c main
layerfs branch fork s1/c d main
layerfs machine update native --daemon <build>/layerfs-daemon
layerfs sandbox create s1/c sb native
# for each Branch: workspace mount sb <branch>, workspace exec, workspace commit, workspace unmount
#   a: sed -i '3s/.*/LINE-FROM-A/' README.md
#   c: echo c > from-c.txt
#   d: echo d > from-d.txt
layerfs branch merge s1/c cd c d    # {"kind":"Merged","fast_forward":false,"conflicts":[]}
layerfs branch publish s1/c cd      # {"kind":"Added","overwrote_count":0}
layerfs file read s1/c main /from-c.txt
# {"kind":"ok","snapshot":"main@9f7ebe737f99",...,"text":"c\n",...}
layerfs branch publish s1/c a       # {"kind":"Added","overwrote_count":1}
layerfs file read s1/c main /from-c.txt
# {"code":"NoSuchPath","message":"/from-c.txt does not exist at main@2b14ee15a1f0",...}
layerfs file read s1/c main /from-d.txt
# {"code":"NoSuchPath","message":"/from-d.txt does not exist at main@2b14ee15a1f0",...}

Expected, for this use case: the second publish is refused because main moved after a was forked, or there's an option that makes it refuse. Actual, as documented: Added, and both /from-c.txt and /from-d.txt are gone from main. overwrote_count is 1 because it counts Layers, and the merged cd publish was one Layer.

I work on Commonly (commonly.me). Written with AI assistance; the repro was run as described.

Lenguaje dominante
Rust
Estrellas
2.1k
Forks
110
Merge medio
23 h 52 min
PR fusionados (30 d)
138

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de Ephemeral-AI-Lab/layerfs

Todos los issues de Ephemeral-AI-Lab/layerfs

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.