Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

runners.yaml: add per-cluster directory mappings (centralize paths hardcoded in launch_*.sh) / runners.yaml：添加集群目录映射（集中管理 launch_*.sh 中硬编码的路径）

Abierto
  • ¿Algo no está bien?
#1,973 0 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@cquil11 ya está trabajando en esto.

Desde el 1/7/2026.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

Summary

.github/configs/runners.yaml currently only maps two things per cluster: labels: (scheduling-label → runner-name groups) and hardware: (available-cpu-dram-mib, gpus-per-node). Everything else that is genuinely per-cluster — most importantly filesystem directories — is hardcoded and duplicated across the 16 runners/launch_*.sh scripts. We should extend runners.yaml to be the single source of truth for these per-cluster mappings and have the launchers read from it.

Problem

Per-cluster paths are scattered across every launcher and diverge by site, so adding/moving a cluster or fixing a path means editing many bash files by hand. A grep across runners/ shows the kinds of directories currently baked in:

  • HF hub cache: /mnt/vast/gharunner/hf-hub-cache, /mnt/data/gharunners/hf-hub-cache/, ...
  • Squashfs / container image dirs: /mnt/vast/gharunner/squash/, /data/squash/, /home/slurm-shared/gharunners/squash, /data/home/sa-shared/gharunners/squash/, /mnt/nfs/lustre/containers/, /data/containers/
  • Model weight-staging: /data/models/{dsv,dsr,MiniMax-M...}, /mnt/nfs/lustre/models/...
  • AIPerf cache / dataset mmap cache: /mnt/vast/gharunner/ai-perf-cache, AIPERF_DATASET_MMAP_CACHE_DIR
  • NFS home mounts: /home/sa-shared/, /mnt/nfs/sa-shared/, /data/home/sa-shared/
  • Scratch/lustre roots: /mnt/lustre, /data/

These are exactly the values that differ per cluster and belong next to the existing hardware: metadata. A concrete example of why this matters: the gb300 NFS ELOOP workaround requires using /data/home/sa-shared/... instead of /home/sa-shared/... on that specific cluster — the sort of per-cluster path divergence that should be declared in one place, not remembered and hand-edited into a launcher.

Proposal

Add a new per-cluster section to runners.yaml, keyed by the same cluster:<name> keys already used under hardware:, e.g.:

paths:
  cluster:h200-cw:
    hf-cache-dir: /mnt/vast/gharunner/hf-hub-cache
    squash-dir: /mnt/vast/gharunner/squash
    aiperf-cache-dir: /mnt/vast/gharunner/ai-perf-cache
    container-image-dir: /mnt/nfs/lustre/containers
    model-weights-dir: /mnt/nfs/lustre/models
    home-mount: /home/sa-shared
    scratch-dir: /mnt/lustre
  cluster:gb300-nv:
    ...
    home-mount: /data/home/sa-shared   # NFS ELOOP workaround, per-cluster

Then have the launchers resolve these from runners.yaml (via the same loader generate_sweep_configs.py already uses for labels/hardware, or a small shared helper) instead of hardcoding them, keeping the current values as defaults during migration.

Exact field names/shape are up for discussion — the goal is: per-cluster directories (and other cluster-specific config) declared once in runners.yaml, consumed by launchers.

Acceptance criteria

  • runners.yaml gains a per-cluster mapping (keyed by cluster:<name>) for the directories currently hardcoded in runners/launch_*.sh (HF cache, squash/container dirs, model-weights/staging, aiperf + dataset mmap cache, home mount, scratch).
  • Launchers read these paths from runners.yaml rather than embedding literals (shared loader/helper).
  • Per-cluster overrides (e.g. gb300 /data/home/sa-shared NFS workaround) are expressed as data, not special-cased in bash.
  • Existing behavior preserved (same resolved paths per cluster) after migration.

Notes

Could also fold in other per-cluster knobs that are currently implicit (e.g. filesystem quirks, default partitions/accounts) as follow-ups. Scope this issue to directories first.

中文说明

建议在 runners.yaml 中添加每集群目录映射，将当前分散在各 launch_*.sh 脚本中硬编码的路径集中管理。

Lenguaje dominante
Python
Estrellas
1.9k
Forks
343
Merge medio
2 d 40 min
PR fusionados (30 d)
431

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de SemiAnalysisAI/InferenceX

Todos los issues de SemiAnalysisAI/InferenceX

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.