Audit runs can leave .posthog-audit-checks.json in the user's project

オープン 初心者向け
#1,326 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
2/5
見積もり時間
1〜3時間
初心者へのやさしさ
75/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
typescript
領域
cli, tooling

調査の方向性

監査台帳ファイルは src/lib/programs/audit/seed.ts で作成され、src/lib/programs/run-agent-legacy.ts で管理されます。ledger.stop() が呼び出されている finally ブロックを確認し、programConfig.auditLedgerFile のファイル削除を追加してください。また、スコープ付きファイル削除のログ順序を調整するために src/agent/runner/harness/pi/security.ts を確認してください。監査プログラムをローカルで実行し、ファイルがクリーンアップされ、誤解を招く 'bash denied' ログが表示されないことを確認します。

索引モデルが issue の本文から書いたものです。

説明

Problem

The wizard writes .posthog-audit-checks.json into the project root before the audit agent starts. No wizard code deletes it. The audit skills end with an instruction for the agent to run rm -f .posthog-audit-checks.json. The file is removed only if the agent runs that last step.

When the agent skips the step, the run still completes and the user keeps an untracked JSON file in their repo root, next to their own code. The next run does not break: the watcher ignores a leftover ledger and the seed replaces it. But the file is clutter, and a user can commit it by accident.

There is a second, smaller problem in the same path. On pi, a scoped rm of that file is logged as Denying bash command (not in allowlist) and captured as a bash denied event. Then the scoped-rm rule from #834 lets it run. The log and the telemetry say "denied". The command ran.

Evidence

Real-TUI sweep of the audit program on the wizard-workbench app audit/posthog-demo-3000. Harness pi (binding), sequence linear, model openai/gpt-5.6-sol.

  • Refactor stack B2 head d938a43b (#1307). The agent read step 6, uploaded the notebook, resolved write-report and upload-notebook, and published the handoff. It never ran the rm. After exit, .posthog-audit-checks.json (3929 bytes) was still in the project root.
  • Release A top 200961f8 (#1303). The agent ran rm -f .posthog-audit-checks.json and the file was removed. The wizard log shows the misleading pair:
    [pi] → bash {"command":"rm -f .posthog-audit-checks.json"}
    Denying bash command (not in allowlist): rm -f .posthog-audit-checks.json
    [pi] ← bash: {"content":[{"type":"text","text":"(no output)"}]}
    

The same prompt, harness and model gave both outcomes. The cleanup depends on the agent.

Code on main (d8486dc1):

  • src/lib/programs/audit/seed.ts:118-124: seedAuditLedger writes the ledger. src/lib/programs/audit/index.ts:26-29 and :58 call it before the agent starts. The wizard creates the file.
  • src/lib/programs/run-agent-legacy.ts:81-95: runProgramAgent starts the ledger watcher and stops it in finally. Nothing removes the file.
  • src/lib/programs/audit/ledger-watcher.ts:32: "A ledger an earlier run left behind stays ignored until this run writes." The code already expects leftovers.
  • src/lib/runners/run-wizard.ts:223: the task stream reads the checks from the session (getAuditChecks(session)), not from the file. Nothing reads the file after the run.
  • src/agent/runner/harness/pi/index.ts:741-749: pi deletes .posthog-events.json host-side after the run. There is no matching cleanup for the audit ledger.
  • src/agent/runner/harness/pi/runtime-notes.ts:53: pi still tells the agent that bash is only for install, build, typecheck, lint and format commands. #894 names this line as the reason agents skip rm. #1000 would update it.
  • src/agent/runner/harness/pi/security.ts:372-385: evaluateToolCall calls wizardCanUseTool first and applies isScopedFileRemoval after.
  • src/agent/agent-interface.ts:523-528: wizardCanUseTool logs Denying bash command and captures bash denied before it returns the deny that pi then overrides.

Skill side, context-mill main:

  • context/skills/audit/description.md:49 and context/skills/audit/references/6-report.md:284 tell the agent to delete the ledger.
  • The same step is in the six audit-* leaf skills, for example context/skills/audit-events/references/4-report.md:21, and in context/skills/events-audit/references/6-dashboard.md:270.
  • context-mill#306 (open) moves reports to publish_handoff and keeps the agent-driven delete.

The refactor stack has the same behavior. On d938a43b, src/programs/program-file-watchers.ts:16-58 owns the ledger for the run, but stop() only stops the watcher (src/programs/run-program.ts:706-708).

Prior work. #833 added a host-side cleanupArtifacts seam for .posthog-events.json. It was closed because it worked around pi blocking rm, and #834 fixed that instead. This case is different. rm works here, and the agent did not run it. The wizard also seeds this file, so the program is the file's owner. #898 and #901 fixed the misleading deny log and telemetry. Both were closed without merging, and main still has the old order. #612 (open) raises the same class of leak for /tmp scratch files that skills name themselves, and leans toward a skill-side fix because the wizard does not know those names. That concern does not apply here: the program declares the name in auditLedgerFile, so the wizard can delete it without learning skill filenames. #894 (open) would make the agent more likely to run the rm, but cleanup would still depend on the agent.

Suggested fix

  1. Delete the ledger in the wizard after the agent run settles, on every exit path. On main, remove path.join(session.installDir, programConfig.auditLedgerFile) with force: true after ledger.stop() in the finally of runProgramAgent, and in the registerCleanup callback so aborts clean up too. On the stack, do it in startProgramFileWatchers().stop() or next to it in the run-program.ts finally. This covers audit, events-audit and the family leaves, because they all set auditLedgerFile.
  2. In context-mill, remove the "delete .posthog-audit-checks.json" step from the audit skills, or mark it optional, once the wizard deletes the file. Then the skill and the wizard do not both own the file.
  3. In evaluateToolCall, decide the scoped-rm rule before the deny is logged and captured. Keep the disallowedTools check. Log an allow line for the command that runs, and do not capture bash denied for it. #898 has a small version of this change.
主要言語
TypeScript
スター
204
フォーク
54
平均マージ
1日 5時間
マージ済み PR(30日)
51

環境構築

  • Dockerfile・Docker Compose ファイルなし
  • プルリクエストのテンプレートあり
  • コントリビューションガイドなし

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

PostHog/wizard のほかの issue

PostHog/wizard の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。