Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

runners.yaml: add per-cluster directory mappings (centralize paths hardcoded in launch_*.sh) / runners.yaml：添加集群目录映射（集中管理 launch_*.sh 中硬编码的路径）

Aperta
  • Qualcosa non va?
#1,973 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@cquil11 ci sta già lavorando.

Dal 1/7/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

Summary

.github/configs/runners.yaml currently only maps two things per cluster: labels: (scheduling-label → runner-name groups) and hardware: (available-cpu-dram-mib, gpus-per-node). Everything else that is genuinely per-cluster — most importantly filesystem directories — is hardcoded and duplicated across the 16 runners/launch_*.sh scripts. We should extend runners.yaml to be the single source of truth for these per-cluster mappings and have the launchers read from it.

Problem

Per-cluster paths are scattered across every launcher and diverge by site, so adding/moving a cluster or fixing a path means editing many bash files by hand. A grep across runners/ shows the kinds of directories currently baked in:

  • HF hub cache: /mnt/vast/gharunner/hf-hub-cache, /mnt/data/gharunners/hf-hub-cache/, ...
  • Squashfs / container image dirs: /mnt/vast/gharunner/squash/, /data/squash/, /home/slurm-shared/gharunners/squash, /data/home/sa-shared/gharunners/squash/, /mnt/nfs/lustre/containers/, /data/containers/
  • Model weight-staging: /data/models/{dsv,dsr,MiniMax-M...}, /mnt/nfs/lustre/models/...
  • AIPerf cache / dataset mmap cache: /mnt/vast/gharunner/ai-perf-cache, AIPERF_DATASET_MMAP_CACHE_DIR
  • NFS home mounts: /home/sa-shared/, /mnt/nfs/sa-shared/, /data/home/sa-shared/
  • Scratch/lustre roots: /mnt/lustre, /data/

These are exactly the values that differ per cluster and belong next to the existing hardware: metadata. A concrete example of why this matters: the gb300 NFS ELOOP workaround requires using /data/home/sa-shared/... instead of /home/sa-shared/... on that specific cluster — the sort of per-cluster path divergence that should be declared in one place, not remembered and hand-edited into a launcher.

Proposal

Add a new per-cluster section to runners.yaml, keyed by the same cluster:<name> keys already used under hardware:, e.g.:

paths:
  cluster:h200-cw:
    hf-cache-dir: /mnt/vast/gharunner/hf-hub-cache
    squash-dir: /mnt/vast/gharunner/squash
    aiperf-cache-dir: /mnt/vast/gharunner/ai-perf-cache
    container-image-dir: /mnt/nfs/lustre/containers
    model-weights-dir: /mnt/nfs/lustre/models
    home-mount: /home/sa-shared
    scratch-dir: /mnt/lustre
  cluster:gb300-nv:
    ...
    home-mount: /data/home/sa-shared   # NFS ELOOP workaround, per-cluster

Then have the launchers resolve these from runners.yaml (via the same loader generate_sweep_configs.py already uses for labels/hardware, or a small shared helper) instead of hardcoding them, keeping the current values as defaults during migration.

Exact field names/shape are up for discussion — the goal is: per-cluster directories (and other cluster-specific config) declared once in runners.yaml, consumed by launchers.

Acceptance criteria

  • runners.yaml gains a per-cluster mapping (keyed by cluster:<name>) for the directories currently hardcoded in runners/launch_*.sh (HF cache, squash/container dirs, model-weights/staging, aiperf + dataset mmap cache, home mount, scratch).
  • Launchers read these paths from runners.yaml rather than embedding literals (shared loader/helper).
  • Per-cluster overrides (e.g. gb300 /data/home/sa-shared NFS workaround) are expressed as data, not special-cased in bash.
  • Existing behavior preserved (same resolved paths per cluster) after migration.

Notes

Could also fold in other per-cluster knobs that are currently implicit (e.g. filesystem quirks, default partitions/accounts) as follow-ups. Scope this issue to directories first.

中文说明

建议在 runners.yaml 中添加每集群目录映射，将当前分散在各 launch_*.sh 脚本中硬编码的路径集中管理。

Lingua principale
Python
Stelle
1.9k
Fork
343
Merge medio
2g 40m
PR unite (30g)
431

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di SemiAnalysisAI/InferenceX

Tutte le issue di SemiAnalysisAI/InferenceX

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.