Research Review: Pushsync Silent Chunk Loss
Los mantenedores suelen responder en 2 días
Evaluación
Este issue todavía no se ha evaluado.
Descripción
PR: https://github.com/ethersphere/bee/pull/5390
Branch: fix/pushsync-out-of-depth-chunk-loss
Summary
Chunks can silently disappear from the network after a successful upload. Three bugs in the pushsync/pusher pipeline allow a chunk to be "delivered" (from the origin's perspective) while no node in the correct neighborhood actually retains it. The proposed fixes change protocol-level forwarding and receipt handling behavior.
Note: These issues are reproducible in smaller clusters such as the public testnet, where limited neighborhood sizes make out-of-depth forwarding and shallow receipts more likely.
Bug 1: Out-of-Depth Storing
Problem: When a node has no closer peer (ErrWantSelf) but the chunk is outside its AOR, it stores the chunk anyway. The chunk lands in a low-proximity bin, unreserve() evicts it immediately, but the origin already has a valid receipt. Chunk is gone.
Fix: Guard with proximity(chunk, self) < rad before storing — return ErrOutOfDepthStoring instead, origin retries with next peer. (pkg/pushsync/pushsync.go:301-309)
Doubling concern (@martinconic): The check uses StorageRadius() which already accounts for doubling (CommittedDepth - doublings). Using CommittedDepth instead would incorrectly reject chunks in sister neighborhoods. Appears correct, but needs research validation for edge cases during radius transitions.
Bug 2: Shallow Receipt Short-Circuits Parallel Pushes
Problem: On any shallow receipt, pushToClosest returns immediately with ErrShallowReceipt, discarding results from other inflight multiplex pushes (up to 32). Only 1 attempt of the error budget is used. Valid receipts from parallel pushes are thrown away.
Fix: Treat shallow receipt like any other peer failure — cache it, burn the budget, wait for inflight pushes. Only return ErrShallowReceipt after the full budget is exhausted or when falling back to ErrWantSelf. (pkg/pushsync/pushsync.go:369-530)
Bug 3: False ChunkSynced After Retry Exhaustion
Problem: After 6 pusher retries, chunks are marked ChunkSynced even though no node in the correct neighborhood stored them. The upload appears complete but the chunk is lost.
Fix: Report ChunkCouldNotSync instead of ChunkSynced. (pkg/pusher/pusher.go:283-289)
Gap identified (@martinconic): upload.Report() tag switch (pkg/storer/internal/upload/uploadstore.go:609-617) doesn't handle ChunkCouldNotSync — chunk gets cleaned up from the upload store but tag counters are never updated. Uploads with undeliverable chunks will show permanently incomplete sync counts.
Possible approaches:
- A. Add
ChunkCouldNotSynccase with a newFailedcounter inTagItem+ surface via API - B. Keep chunk in push queue for later retry (risk: infinite loops for undeliverable chunks)
- C. Surface via metrics/API only, accept the tag gap
Protocol Impact
| Scenario | Before | After |
|---|---|---|
| Node outside AOR, no closer peer | Stores + sends receipt (evicted shortly after) | Returns error, origin retries |
| Shallow receipt during multiplex | Immediate abort, 1 of 32 attempts used | Full budget exhausted before giving up |
| Pusher exhausts 6 retries | Marked ChunkSynced (false positive) |
Marked ChunkCouldNotSync (correct, but tag gap) |
- Lenguaje dominante
- Go
- Estrellas
- 1.5k
- Forks
- 390
- Merge medio
- 2 d 23 h
- PR fusionados (30 d)
- 24
Preparar el entorno
- Incluye un Dockerfile o un archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de ethersphere/bee
-
Chequebook deployment uses a fixed 175,000 gas limit; it needs about 630,000 under Glamsterdam (live on Sepolia since 2026-10-06)Posiblemente ocupada @sbackend123 la tomó hace 17 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
ethersphere/bee#5650 · 1 asignado ·
Los mantenedores suelen responder en 2 días
-
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
ethersphere/bee#5666 ·
Los mantenedores suelen responder en 2 días
-
Add a transaction inbox for outbound chain transactionsPosiblemente ocupada @sbackend123 la tomó hace 10 días. Abiertoseen
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
ethersphere/bee#5660 · 1 asignado ·
Los mantenedores suelen responder en 2 días
-
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
ethersphere/bee#5659 · 6 comentarios ·
Los mantenedores suelen responder en 2 días
-
accepted seen
Dificultad 4/5 3-5 días Aptitud para principiantes 40/100
ethersphere/bee#5648 · 1 comentario ·
Los mantenedores suelen responder en 2 días
Todos los issues de ethersphere/bee
Issues similares
-
Project submission:Abierto
Dificultad 1/5 Menos de una hora Aptitud para principiantes 72/100
slavakurilyak/awesome-ai-agents#845 ·
Los mantenedores suelen responder en 1 día
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
juanfont/headscale#3583 · 1 reacción ·
Los mantenedores suelen responder en 3 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
-
[BUG] deploy.restart_policy without condition drops the restart policyPosiblemente ocupada @akshat-lakhera la tomó hace 1 día. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
docker/compose#14383 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 72/100
Azure/azure-storage-fuse#2429 ·
Los mantenedores suelen responder en 1 día