TPU.scp_to() has hardcoded 300s timeout — no way to override for large transfers
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 67/100
Research direction
Start in tpuz/tpu.py at scp_to around line 360, then inspect _gcloud around line 116 and the related scp_from, push, and pull entry points. Trace how each transfer invokes gcloud and verify that the chosen timeout behavior is consistently exposed through the public methods without changing unrelated command handling.
Written by the indexing model from the issue text.
Description
TPU.scp_to() in tpuz/tpu.py:360 calls self._gcloud(args, timeout=300) with a hardcoded 300 second timeout. There's no way to pass a longer timeout through the public API, so any transfer larger than roughly 100–200 MB at typical residential upload speeds hits subprocess.TimeoutExpired even though the underlying gcloud scp is making forward progress.
Repro
from tpuz import TPU
tpu = TPU("some-tpu", accelerator="v6e-8", zone="europe-west4-a", preemptible=True)
tpu.up_queued(timeout_hours=2)
# flaxchat repo is ~1 GB (mostly models/ and exports/). scp_to will time out at 300s:
tpu.scp_to("/Users/me/code/flaxchat/", "/home/me/workdir/flaxchat")
Traceback
File ".../tpuz/tpu.py", line 360, in scp_to
self._gcloud(args, timeout=300)
File ".../tpuz/tpu.py", line 116, in _gcloud
result = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
...
subprocess.TimeoutExpired: Command '['gcloud', 'compute', 'tpus', 'tpu-vm', 'scp',
'/Users/me/code/flaxchat/', 'some-tpu:/home/me/workdir/flaxchat',
'--zone=europe-west4-a', '--worker=0', '--recurse']' timed out after 300 seconds
Fix
Add a timeout kwarg to scp_to/scp_from/push/pull:
def scp_to(self, local, remote, worker=0, timeout=300):
...
self._gcloud(args, timeout=timeout)
Also consider defaulting to None (no timeout) for scp, since transfer time is inherently unpredictable and the operation is interruptible. Timeouts make sense for ssh where a hang indicates a dead command, not for bulk data movement.
Environment
tpuz==0.1.11- Python 3.13 on macOS (client), TPU v6e-8 europe-west4-a (remote)
Workaround
Use git clone on the TPU for code, and scp_to only for small untracked files:
tpu.ssh("git clone --depth 1 https://github.com/org/repo /home/me/workdir/repo", timeout=180)
tpu.scp_to("/local/untracked_script.py", "/home/me/workdir/repo/scripts/untracked_script.py")
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from mlnomadpy/tpuz
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 3/5 1-2 days Newbie friendliness 74/100
-
Difficulty 3/5 1-2 days Newbie friendliness 58/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100