Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Same unconfined `predictions/<item_id>` path pattern remains in four benchmark rollouts

オープン
  • 何か問題がありますか？
#306 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 6 日以内に返信

@adongwanai がすでに取り組んでいます。

2026年10月8日 から。

  • #309 @adongwanai による — オープン

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
72/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
python
領域
security

調査の方向性

Start with the merged reference implementation in skillopt/envs/spreadsheetbench/rollout.py (_is_safe_task_id(), _confined_task_out_dir()), then decide whether to lift them into a shared helper. Apply the pattern to the listed call sites in skillopt/envs/docvqa/rollout.py:173,:232, livemathematicianbench/rollout.py:144, searchqa/rollout.py:204, and officeqa/rollout.py:530, keeping in mind livemathematicianbench's colon-shaped ids need an id-to-safe-dirname mapping rather than validation-and-reject. Any mapping must be applied on the read side too: skillopt/optimizer/slow_update.py:113 reads predictions/<task_id>/conversation.json. Done when ids like ../../x cannot escape out_root and existing rollouts plus the reader still resolve their own outputs.

索引モデルが issue の本文から書いたものです。

説明

Follow-up from #264 (now merged). That PR validated the SpreadsheetBench task identifier and confined its persistent output directory. The same unconfined pattern remains in four other benchmark rollouts:

  • skillopt/envs/docvqa/rollout.py:173 and :232
  • skillopt/envs/livemathematicianbench/rollout.py:144
  • skillopt/envs/searchqa/rollout.py:204
  • skillopt/envs/officeqa/rollout.py:530

Each is os.path.join(out_root, "predictions", item_id), where item_id comes straight from the dataset item — no validation of the identifier and no containment check on the destination. An id such as ../../x therefore derives a destination outside out_root and processing continues into the model / code-execution path. skillopt/envs/spreadsheetbench/rollout.py now has _is_safe_task_id() and _confined_task_out_dir(), which could be lifted into a shared helper.

Two caveats are why this is filed rather than sent as a direct port:

  1. A charset validator is not sufficient for livemathematicianbench. Its ids are colon-shaped (202602:12), so all 177 ids across the shipped splits would be rejected by the SpreadsheetBench rule. Those runs need confinement by construction (map an id to a safe directory name) rather than validate-and-reject. Separately, an id containing : cannot be a Windows directory name at all, so those runs are already broken on Windows for an unrelated reason.

  2. Renaming the directory is not free. Readers look the task up by its raw id — skillopt/optimizer/slow_update.py:113 reads predictions/<task_id>/conversation.json — so any id-to-dirname mapping has to be applied on the read side too.

docvqa (63180), searchqa (hex) and officeqa (UID0003) use ids that are already safe shapes, so for those three the SpreadsheetBench approach can be applied as-is.

Filed separately because it is outside #264's reviewed scope.

主要言語
Python
スター
18k
フォーク
1.7k
平均マージ
6日 18時間
マージ済み PR（30日）
12

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

microsoft/SkillOpt のほかの issue

microsoft/SkillOpt の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。