Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

generate tasks within a kind concurrently

Abierto
#716 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
python
Área
devops, performance

Línea de trabajo

Start in src/taskgraph/generator.py around lines 76-89, where transforms are currently run serially for tasks within a kind. Investigate safe worker-thread or process approaches and when their setup overhead is worthwhile, then benchmark generation for the tests kind and smaller kinds. Done means transforms run concurrently without changing generated results and performance impact is measured.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

In #5 we've discussed parallelizing the loading of kinds to improve decision task performance. One of the discoveries in that thread was that for Gecko (where we have the longest decision task), this would only incrementally improve performance. As it turns out, most of the work is inside of the test kind, and kind-level concurrency won't help there at all.

Something we could do to improve generate within a kind is to concurrently run transforms for individual tasks. This is currently done serially over here, and AFAICT, there should be nothing stopping us from running those across multiple threads or processes. It's unclear to me if this will be a massive win, but in a quick test it appears there are ~22,000 calls to transforms for the tests kind, so presumably it would be something.

We may need some sort of intelligent way to push these into worker threads/processes; it's possible that for kinds with just a few tasks that the overhead of setup/teardown would be more than the reduction we'd get from the concurrency. Perhaps a minimum number tasks should be needed before spawning workers (either an absolute number, or one relative to the number of workers we'd spawn, which would presumably be based on the # of cores available).

Lenguaje dominante
Python
Estrellas
26
Forks
55
Merge medio
2 d 6 min
PR fusionados (30 d)
15

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de taskcluster/taskgraph

Todos los issues de taskcluster/taskgraph

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.