`eval_clean_hashes` is missing or stale for some `task_cleanroom_v6` images

Start with src/programbench/data/tasks/*/task.yaml, eval_batch.py, and Evaluator._remove_hashed_files(). Run the provided docker sha256sum command for the affected task_cleanroom_v6 images and compare each digest with its metadata. Done means every real task has current, well-formed deduplicated hashes and the update process is documented or automated for rebuilt images.

Summary

eval_clean_hashes is intended to remove byte-for-byte copies of the gold executable from a submission before compile.sh runs. However, some task metadata does not define the field, and some existing values do not match the reference executable currently shipped in the corresponding task_cleanroom_v6 image.

This makes the metadata incomplete for downstream evaluators that rely on eval_clean_hashes to scrub renamed copies of the reference executable.

Current state

At repository commit 963063c:

• 201 task directories total (including testorg__calculator.abc1234)
• 40 task YAML files do not define eval_clean_hashes
• 161 define at least one hash
• none define an explicitly empty list

Excluding the test fixture, 39 of the 200 real tasks are missing the field.

In addition, a non-empty list does not necessarily contain the SHA-256 of the executable in the current task_cleanroom_v6 image.

Examples observed:

As a control, xorg62__tty-clock.f2f847c does contain the current reference hash (cd400708...).

Reproduction

For a task image:

docker run --rm \
  --network none \
  --user root \
  --entrypoint sha256sum \
  programbench/rs_1776_curlie.5dfcbb1:task_cleanroom_v6 \
  /workspace/executable

Compare the resulting digest with:

src/programbench/data/tasks/rs__curlie.5dfcbb1/task.yaml

The evaluator consumes this metadata in eval_batch.py and recursively removes matching files in Evaluator._remove_hashed_files() before compilation. The evaluator comment describes these hashes as protection against byte-for-byte copies of the gold binary, so including the current cleanroom reference hash appears consistent with the existing field semantics.

Suggested fix

1. Compute the SHA-256 of /workspace/executable for every current task_cleanroom_v6 image.
2. Append it to that task's eval_clean_hashes, preserving historical hashes and deduplicating the list.
3. Add a maintenance command or release script that updates these values whenever cleanroom images are rebuilt.
4. Validate that every real task has at least one well-formed 64-character SHA-256 value.
5. Ideally pin cleanroom images by digest, or verify/update the hashes as part of the image publication workflow, so mutable tags cannot silently drift from task metadata.

I can prepare a follow-up PR to populate the current hashes if this approach matches the intended use of eval_clean_hashes.