[Bug]: managed CPU Ray template has no /mnt hot volume

Open Beginner friendly
  • Is something wrong?
#563 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
go, kubernetes
Domain
backend, cloud

Research direction

Start with managed-workflow-rayjob-cpu.yaml.tmpl and compare it with managed-workflow-rayjob.yaml.tmpl and the managed Job template named in the issue. Trace how the managed CPU Ray template is rendered and run tau run --dry-run=client with compute.gpus: 0 and Ray workers; done means both head and worker pods have tau-hot mounted at /mnt and the three expected environment variables.

Written by the indexing model from the issue text.

Description

bug
Problem

Managed Ray workflows that use no GPUs don't get /mnt at all. managed-workflow-rayjob-cpu.yaml.tmpl has no tau-hot volume or mount and doesn't set TAU_HOT_DIR, TAU_DATASETS_DIR or TAU_CHECKPOINTS_DIR. The GPU managed Ray template (managed-workflow-rayjob.yaml.tmpl) and the managed Job template both mount tau-hot at /mnt and set those variables.

Expected: the CPU managed Ray template provides the same /mnt scratch volume and environment as the GPU template. Code that reads TAU_HOT_DIR then works the same with and without GPUs.

Reproduction
  1. Write a managed workflow config: set schema_version, leave engine unset, and use compute.gpus: 0 with Ray workers.
  2. Run tau run --dry-run=client.
  3. Neither the head nor the worker pods have a tau-hot volume or TAU_HOT_DIR.
TauGrid or tau version

main at bd2b98c2.

Environment

Reproduces with tau run --dry-run=client; no cluster-specific setup needed.

Additional context

This predates #561. That PR renders the profile's hotVolume only for direct engine: job and engine: rayjob configs and leaves the managed templates unchanged. Supporting hotVolume in managed templates would be a separate follow-up.

Dominant language
Go
Stars
59
Forks
8
Avg merge
1d 15h
Merged PRs (30d)
84

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Azure/taugrid

All issues in Azure/taugrid

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.