`eval_clean_hashes` is missing or stale for some `task_cleanroom_v6` images

src/programbench/data/tasks/*/task.yaml、eval_batch.py、Evaluator._remove_hashed_files()から始めます。影響を受けるtask_cleanroom_v6イメージに対して、提供されているdocker sha256sumコマンドを実行し、各ダイジェストをメタデータと比較します。すべての実タスクに最新かつ適切な形式の重複排除済みハッシュがあり、再ビルドされたイメージの更新プロセスが文書化または自動化されていれば完了です。

Summary

eval_clean_hashes is intended to remove byte-for-byte copies of the gold executable from a submission before compile.sh runs. However, some task metadata does not define the field, and some existing values do not match the reference executable currently shipped in the corresponding task_cleanroom_v6 image.

This makes the metadata incomplete for downstream evaluators that rely on eval_clean_hashes to scrub renamed copies of the reference executable.

Current state

At repository commit 963063c:

• 201 task directories total (including testorg__calculator.abc1234)
• 40 task YAML files do not define eval_clean_hashes
• 161 define at least one hash
• none define an explicitly empty list

Excluding the test fixture, 39 of the 200 real tasks are missing the field.

In addition, a non-empty list does not necessarily contain the SHA-256 of the executable in the current task_cleanroom_v6 image.

Examples observed:

As a control, xorg62__tty-clock.f2f847c does contain the current reference hash (cd400708...).

Reproduction

For a task image:

docker run --rm \
  --network none \
  --user root \
  --entrypoint sha256sum \
  programbench/rs_1776_curlie.5dfcbb1:task_cleanroom_v6 \
  /workspace/executable

Compare the resulting digest with:

src/programbench/data/tasks/rs__curlie.5dfcbb1/task.yaml

The evaluator consumes this metadata in eval_batch.py and recursively removes matching files in Evaluator._remove_hashed_files() before compilation. The evaluator comment describes these hashes as protection against byte-for-byte copies of the gold binary, so including the current cleanroom reference hash appears consistent with the existing field semantics.

Suggested fix

1. Compute the SHA-256 of /workspace/executable for every current task_cleanroom_v6 image.
2. Append it to that task's eval_clean_hashes, preserving historical hashes and deduplicating the list.
3. Add a maintenance command or release script that updates these values whenever cleanroom images are rebuilt.
4. Validate that every real task has at least one well-formed 64-character SHA-256 value.
5. Ideally pin cleanroom images by digest, or verify/update the hashes as part of the image publication workflow, so mutable tags cannot silently drift from task metadata.

I can prepare a follow-up PR to populate the current hashes if this approach matches the intended use of eval_clean_hashes.