clawbench-batch builds the image before discovering that a resumed batch has nothing to do

Open Beginner friendly
  • Is something wrong?
#475 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
75/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
cli, testing

Research direction

Read src/clawbench/runner/batch.py, especially async_main and its --resume path. Check the existing batch tests and run the focused tests for resume behavior. Reorder the resume scan and image build so a completed batch exits successfully without calling docker_build, while pending jobs still trigger exactly one build; verify the summary prints first.

Written by the indexing model from the issue text.

Description

bug

Problem

Resuming a completed batch rebuilds the container image first. If the build cannot run — daemon down, registry unreachable — the command fails instead of reporting that all jobs were already done.

At d8faa73, async_main in src/clawbench/runner/batch.py calls _run_mod.docker_build(args.harness) before it enters the --resume branch that marks jobs from existing logs and checks whether any remain. docker_build exits the process on failure. So clawbench-batch --resume <finished-batch> requires a working engine and a successful build to tell the user there was nothing to run.

Acceptance

  • The resume scan happens before the image build, and a batch with no remaining jobs exits successfully without touching the engine.
  • The build still happens exactly once when there is work to do.
  • A test with a stubbed failing docker_build and a complete resume directory returns success; the same stub with pending jobs attempts the build.
  • The printed resume summary appears before any build output.

@Perry2004 please maintain this issue.

Dominant language
Python
Stars
1.2k
Forks
74
Avg merge
19h 1m
Merged PRs (30d)
5

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from TIGER-AI-Lab/ClawBench

All issues in TIGER-AI-Lab/ClawBench

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.