FUSE cold-read path: single-flight and async read-ahead, one BLAKE3 check (~1.9x); profile tuning alone ≤1.32x

Abierto
  • ¿Algo no está bien?
#684 10 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
8/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
linux, rust, sqlite
Área
backend, performance

Línea de trabajo

This is a performance design thread, not a starter task. Read the report at core/docs/issues/fuse-profile-tuning/README.md and the receipts/ directory, then the CanonicalClient read path and the Store resolver where the BLAKE3 re-check happens. Done means a maintainer-agreed design for single-flight demands and a single BLAKE3 check, with the benchmark rerun on the same cold-read setup.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

Diagnostic work on the 2.0 line (layerfs_v2_0_0 at 8cfb6e2734) asked where LayerFS's cold large-file read time goes, and how much of the FUSE I/O gap (cold read about 150 MB/s, against several hundred MB/s to GB/s for kernel overlay approaches) is profile tuning rather than design.

Answer: mostly design. Within the current contract (kernel write-back off), changing the FUSE profile alone gives at most 1.32×. Three read-path changes give about 1.9×.

Method: one sample per arm, Disposable Store, write-back off, fuser unpatched, Linux 6.12 aarch64 in the Docker VM. The guest page cache was evicted and verified before each cold read; the macOS host cache was not controlled, so throughput is diagnostic and request counts are exact.

Findings

1. Where a cold 500 MiB read goes (current profile: 128 KiB requests, 1 in flight)
Part Share of wall time
Synchronous 8 MiB read-ahead inside the READ request (one request in 64 waits ~42 ms) 75%
Hashing the same bytes three times: pack SHA-256, BLAKE3 in the Store resolver, BLAKE3 again in CanonicalClient 38% (overlaps the row above)
SQLite locate and pack reads 27%
Kernel round trip outside the callback 11%
Reply copy into /dev/fuse 7.6%
Overlay owner visit 1.3%
zstd decode ~0% (random data is stored raw)
2. Profile alone (request size × max_background, 9 arms)
  • Larger requests (512 KiB, 1 MiB) with one in flight: no cold-read gain (147–162 MB/s against 152). READ frames fall 4,000 → 500.
    • The kernel caps the INIT readahead at 128 KiB. Larger READs only appear after raising the mount's bdi read_ahead_kb.
  • More requests in flight (4/16) at 512 KiB / 1 MiB: +32%, but only by fetching every object twice (27k → 54k demands) and using 50–90% more callback time.
  • Writes:
    • Sequential: possibly +10–20%, within the noise of one sample.
    • 4 KiB random: unchanged.
  • Memory: receive buffers grow +0.75 MiB per mount at 512 KiB and +1.75 MiB at 1 MiB.
3. Read-path changes, quantified (round 2)

All arms of this round ran on the same CPU state. Control 206 MB/s.

Change 128 KiB / 1 512 KiB / 4
Control 206.0 279.9 (duplicate fetches)
One BLAKE3 check (skip the client re-check) 257.3 —
Single-flight (concurrent demands share one fetch) 219.9 386.0 (objects 53,771 → 27,214)
Asynchronous read-ahead (own thread, own Store custody) 283.4 278.0
Async + one BLAKE3 check 361–383 340.0
Single-flight + one BLAKE3 check — 392.5

How to read it:

  • Next ceiling: after these changes, the Workspace's single Store read session (one lock, one fetch thread) is busy 94% of the time. That caps the design at about 410 MB/s; going further needs parallel Store reads within one Workspace.
  • Async with more in flight: asynchronous read-ahead is slower with 4 in flight than single-flight alone, because it is a third claimant on that lock.
  • The failed run: the first run with async read-ahead at 512 KiB / 4 FAILED. Its background fetch reused the request's StorePorts, which refuses a concurrent demand, and the split-READ path turned the refusal into a short read. The fix uses StorePorts::fresh(); the failure and its diagnosis are kept in the receipts.
4. Small files and ls -l
  • FLUSH elision saves nothing. FLUSH already returns ENOSYS, which the kernel remembers per connection, so it is sent once per mount.
  • Creates: a create costs 5 requests. The GETATTR per create is about 20% of create time; I suspect parent re-validation under default_permissions, but that is not verified.
  • READDIRPLUS (estimate): ls -l of 2,000 entries would drop from 2,022 requests to about 84.

Proposed work (ranked)

  • Single-flight object demands in CanonicalClient: concurrent demands wait for an in-flight fetch instead of re-demanding.
  • Authenticate once per trust boundary. Drop the client BLAKE3 re-check of objects the in-process Store just authenticated (~370 ms per 500 MiB). Whether pack SHA-256 and object BLAKE3 are both needed is an owner decision.
  • Asynchronous read-ahead off the request path, over its own Store custody. Use it only while Store fetches are serialized per Workspace, or together with the next item.
  • Parallel Store reads within one Workspace (more than one read session or connection), the next ceiling.
  • Then revisit the profile: 512 KiB with 4 in flight. It needs:
    • a single-visit composed READ
    • a defined multi-window WRITE contract
    • a daemon-side bdi read_ahead_kb write after mount
    • a larger MAX_INPUT_BYTES
    • 0.75 MiB more receive buffer per mount
  • Small files:
    • Identify the GETATTR per create.
    • Evaluate adaptive READDIRPLUS.

Not done or not verified

  • Clippy: not run. The substitute rust:1.85.1-bookworm image has no Clippy; the registered build image was removed by another session's cleanup.
  • Existing mount tests: not run against the prototype.
  • Correctness proofs for the prototype read switches (concurrent writers, unmount during a background fetch, failure custody): not run.
  • Prototype limits: split READs make several owner visits and split WRITEs publish several windows before one reply. Neither is merge quality.
Lenguaje dominante
Rust
Estrellas
2.1k
Forks
110
Merge medio
23 h 52 min
PR fusionados (30 d)
138

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de Ephemeral-AI-Lab/layerfs

Todos los issues de Ephemeral-AI-Lab/layerfs

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.