Unoptimized Build

Abierto
#141 15 comentarios 2 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
30/100
Tipo de issue
Refactorización
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
cmake, cpp
Área
build-system

Línea de trabajo

Start from the CUDA target definition under ggml/src/ggml-cuda and the generated template-instances/*.cu files shown in the build log, especially the fattn-mma-kvarn-* objects. Reproduce with a clean CUDA build and time those objects to confirm where the slowdown is. Done means the build is faster without changing kernel behavior and the CUDA build/tests still pass.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Hey there, thanks for this project!

I have been trying it out, and it's quite clear that the build is highly unoptimized. Trying it on two different, beefy machines, shows the build significantly slowing down around here:

[ 14%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_1-ncols2_8.cu.o
[ 14%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_16-ncols2_1.cu.o
[ 14%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_16-ncols2_2.cu.o
[ 14%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_16-ncols2_4.cu.o
[ 14%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_16-ncols2_8.cu.o
[ 15%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_2-ncols2_4.cu.o
[ 15%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_2-ncols2_8.cu.o
[ 15%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_32-ncols2_1.cu.o
[ 15%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_32-ncols2_2.cu.o
[ 15%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_4-ncols2_2.cu.o
[ 15%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_4-ncols2_4.cu.o
[ 15%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_4-ncols2_8.cu.o
[ 15%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_64-ncols2_1.cu.o
[ 16%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_8-ncols2_1.cu.o
[ 16%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_8-ncols2_2.cu.o
[ 16%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_8-ncols2_4.cu.o
[ 16%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-instance-ncols1_8-ncols2_8.cu.o
[ 16%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k8-v8.cu.o
[ 16%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k8-v6.cu.o
[ 16%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k8-v5.cu.o
[ 16%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k6-v6.cu.o
[ 17%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k6-v5.cu.o
[ 17%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k6-v4.cu.o
[ 17%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k5-v5.cu.o
[ 17%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k5-v4.cu.o
[ 17%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k5-v3.cu.o
[ 17%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k4-v4.cu.o
[ 17%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k4-v3.cu.o
[ 17%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k4-v2.cu.o
[ 18%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k3-v3.cu.o
[ 18%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k3-v2.cu.o
[ 18%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/fattn-mma-kvarn-decode-instance-k2-v2.cu.o

Notice that it is the kvarn object files that are the significant slowdown. Would you be willing to implement a build optimization fix? I'm guessing these are pulling in a significant amount of headers. It's quite unbelievable how slow the build is for these object files, each taking in the order of tens of minutes.

Lenguaje dominante
C++
Estrellas
1.2k
Forks
80
Merge medio
4 d 20 h
PR fusionados (30 d)
2

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de Anbeeld/beellama.cpp

Todos los issues de Anbeeld/beellama.cpp

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.