Can we get c2606aa merged in?

Abierto
  • ¿Algo no está bien?
#267 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
rust
Área
backend, performance

Línea de trabajo

Start by reviewing commit c2606aae and its five-commit branch in helios, including the docs and TLA+ spec updates. Compare its io_uring reader lanes with the approach in closed PR #216 and determine whether that earlier rejection applies here. The change is behind --embedding-table-prefetch and --embedding-table-workers; validation should include the reported prefill metrics and measurement on real DeepSeek hardware, which the issue says is still outstanding.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

I'd propose topic/engramcache (head c2606aae, five commits) to be merged into main. On disk-backed Engram tables, prefill stalls because a graph publishes its row ids only when it starts, so each miss is read while the GPU waits, and the prefetch meant to hide that loaded nothing: one 4x RTX PRO 6000 run logged 65 submissions, 0 rows loaded and 5,568 dropped. The branch holds a job until its ids are readable, pins read-ahead rows so they can't be evicted before use, and gives each reader worker its own io_uring lane. A host emulation at DeepSeek V4.1's table shape on a GB10 NVMe measured 10.8k vs 15.1k tokens/s and 60% vs 84% GPU busy on a cold 256k prompt with a 16 GiB cache. Real DeepSeek hardware is still unmeasured. The whole series sits in helios behind --embedding-table-prefetch and --embedding-table-workers and carries its docs and TLA+ spec updates. PR #216 tried the same io_uring lanes and was closed unmerged, so it would help to know whether it was rejected for something this branch does too.

https://github.com/randomvariable/spark-local-inference/commit/c2606aae

Lenguaje dominante
Rust
Estrellas
800
Forks
106
Merge medio
1 d 3 h
PR fusionados (30 d)
100

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/hyperflare-inference

Todos los issues de NVIDIA/hyperflare-inference

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.