tickets: run: a UTF-8 BOM makes an otherwise accepted plan retry and defer or pause

bmad_loop/frontmatter.pyとその関数read_frontmatterから始めます。この関数は、issueによると、先頭にBOMが付いたプランに対して空のマッピングを返します。レポートで挙げられているチケット固有のreaderと比較してください。frontmatter解析の既存テストを確認し、先頭にBOMが付いた入力のテストを追加してください。BOM付きプランが通常のプランと同様に読み込まれ、報告されている延期フローで受け付けられれば完了です。

Summary

When a dev session leaves its ticket plan with a UTF-8 byte-order mark (U+FEFF) ahead of the opening delimiter, the tested ticket-specific readers (tickettree._plan_fields, tickets_engine._plan_state, and tickets.py find) still recognize it as built, but frontmatter.read_frontmatter returns an empty mapping. In the mock-adapter repro, the same session output is accepted without the BOM; with it, deferral harvesting rejects the attempt, worktree isolation exhausts the two default dev attempts and defers the ticket, and in-place isolation pauses for manual rollback. The Ubuntu/WSL2 run below shows the same result as Windows; the decoder and delimiter comparison explain why the mechanism is not Windows-specific. Source inspection also shows that tickets.py preserves a BOM when editing frontmatter, so marking the plan does not normalize it away.

A BOM is what Windows PowerShell 5.1 Set-Content -Encoding UTF8 / Out-File -Encoding UTF8 and "UTF-8 with signature" editors write, so a session or a person editing the plan on Windows can introduce one.

Environment

bmad-loop a5a8f097, Windows 11 (10.0.26200), PowerShell 7.6.6 (SHELL unset), Python 3.13.2, git 2.49.0.windows.1; mock adapter (TicketsEngine with the real TicketTree over the sandbox's installed tickets.py, the vendored tests/fixtures/bmad_v7 copy); Linux oracle: same result (Ubuntu on WSL2, Python 3.13.2, git 2.43.0)

Steps to reproduce

From a bmad-loop checkout at a5a8f097, in PowerShell 7 with SHELL removed (uv sync --all-extras done once). The steps below are complete and runnable as shown:

Remove-Item Env:SHELL -ErrorAction SilentlyContinue
$env:PYTHONUTF8 = '1'
# save the script below as repro_bom.py in the checkout root, then:
uv run python repro_bom.py plain worktree   # positive control, worktree isolation: no BOM
uv run python repro_bom.py bom worktree     # trigger: the same plan with a BOM (and CRLF)
uv run python repro_bom.py plain none       # positive control, in-place isolation (default policy)
uv run python repro_bom.py bom none         # trigger, in place

Each run exits 0 (the script itself) and prints the two readers' view of the session's plan, then the run's outcome and the relevant journal records. The script builds the test suite's own v7 sandbox (tests/conftest.py install_bmad_v7 + write_ticket_tree + _commit_template) under $env:TEMP\bom-repro, with one ticket 1.1. It then runs TicketsEngine with a mock adapter whose dev session writes src.txt plus the plan at status: built, with the right ticket: and baseline_revision:.

"""Standalone repro: a ticket plan whose build wrote it with a UTF-8 BOM.
Run from a bmad-loop checkout: uv run python repro_bom.py <bom|plain> <none|worktree>"""
import json, os, subprocess, sys
from pathlib import Path

variant, iso = sys.argv[1], sys.argv[2]
base = Path(os.environ.get("TEMP", "/tmp")).resolve() / "bom-repro"
os.environ["BMAD_LOOP_STATE_DIR"] = str(base / "state")
sys.path.insert(0, "tests")
import conftest as C  # the test suite's own v7 sandbox builders
from bmad_loop import frontmatter, tickettree, verify
from bmad_loop.adapters.base import SessionResult
from bmad_loop.adapters.mock import MockAdapter
from bmad_loop.bmadconfig import load_paths
from bmad_loop.journal import Journal, load_state
from bmad_loop.model import RunState
from bmad_loop.policy import GatesPolicy, NotifyPolicy, Policy, ReviewPolicy, ScmPolicy
from bmad_loop.tickets_engine import TicketsEngine

root = base / f"{variant}-{iso}"
if root.exists():
    C.remove_tree(root)
root.mkdir(parents=True)
(root / "src.txt").write_text("original\n", encoding="utf-8")
(root / ".gitignore").write_text(C.V7_GITIGNORE, encoding="utf-8")
tree = (C.EpicSpec(1, "epic-cart", "Cart", (C.EntrySpec(1, "Cart model"),)),)
C.write_ticket_tree(C.install_bmad_v7(root), tree)
C._commit_template(root)
lay = C.v7_layout(root)
plan = Path(C.run_tickets(lay, "find", "1.1")["plan"])


def plan_bytes(baseline: str) -> bytes:
    nl = "\r\n" if variant == "bom" else "\n"
    text = nl.join(["---", "ticket: 1", "status: built", f"baseline_revision: '{baseline}'", "---", "", "# plan", ""])
    return (b"\xef\xbb\xbf" if variant == "bom" else b"") + text.encode()


def build(spec):  # a dev session that builds the ticket and leaves its plan at `built`
    cwd = Path(spec.cwd)
    here = cwd / plan.relative_to(root)
    (cwd / "src.txt").write_bytes(b"cart model\n")
    here.write_bytes(plan_bytes(verify.rev_parse_head(cwd)))
    print("readers on the session's plan:",
          {"read_frontmatter": frontmatter.read_frontmatter(here),
           "tickettree._plan_fields": tickettree._plan_fields(here)[:2]})
    return SessionResult(status="completed",
                         result_json={"workflow": "auto-dev", "spec_file": str(here), "status": "built"})


run_dir = root / ".bmad-loop" / "runs" / "r1"
state = RunState(run_id="r1", project=str(root), started_at="now",
                 initiative=lay.initiative, initiative_folder=str(lay.initiative_folder))
state.max_stories = 1
policy = Policy(gates=GatesPolicy(mode="none"), notify=NotifyPolicy(desktop=False, file=True),
                review=ReviewPolicy(enabled=False), scm=ScmPolicy(isolation=iso))
adapter = MockAdapter([build] * 4)
TicketsEngine(paths=load_paths(root, want_tickets=True), policy=policy, adapter=adapter,
              run_dir=run_dir, journal=Journal(run_dir), state=state).run()
st = load_state(run_dir)
t = st.tasks["1.1"]
print(f"sessions={len(adapter.sessions)} phase={t.phase.value} attempt={t.attempt} commit={t.commit_sha} paused_stage={st.paused_stage}")
for line in (run_dir / "journal.jsonl").read_text(encoding="utf-8").splitlines():
    e = json.loads(line)
    if e["kind"] in ("spec-read-failed", "dev-decision", "story-deferred", "story-done", "rollback-manual-required"):
        print(e["kind"], {k: e[k] for k in ("action", "reason", "site", "error") if k in e})

Observed (Windows, PowerShell 7.6.6 with SHELL removed; 2 of 2 runs, identical apart from hashes). Displayed plan paths are replaced with ; all other transcript fields are unchanged.

### uv run python repro_bom.py plain worktree
readers on the session's plan: {'read_frontmatter': {'ticket': 1, 'status': 'built', 'baseline_revision': '07150cc46b879e0cc37d2fa76193c5d7b19bb040'}, 'tickettree._plan_fields': (True, 'built')}
sessions=1 phase=done attempt=1 commit=9f75be1f582fad4ca13a3b3f13fea7a8bed293ca paused_stage=epic-boundary
dev-decision {'action': 'proceed', 'reason': ''}
story-done {}
rc=0  (6s)
### uv run python repro_bom.py bom worktree
readers on the session's plan: {'read_frontmatter': {}, 'tickettree._plan_fields': (True, 'built')}
readers on the session's plan: {'read_frontmatter': {}, 'tickettree._plan_fields': (True, 'built')}
sessions=2 phase=deferred attempt=2 commit=None paused_stage=None
spec-read-failed {'site': 'spec-deferrals', 'error': 'empty or invalid frontmatter'}
dev-decision {'action': 'retry', 'reason': 'spec unreadable while harvesting deferrals (empty or invalid frontmatter): <plan-path>'}
spec-read-failed {'site': 'spec-deferrals', 'error': 'empty or invalid frontmatter'}
dev-decision {'action': 'defer', 'reason': 'spec unreadable while harvesting deferrals (empty or invalid frontmatter): <plan-path>'}
story-deferred {'reason': 'spec unreadable while harvesting deferrals (empty or invalid frontmatter): <plan-path>'}
rc=0  (6s)
### uv run python repro_bom.py plain none
readers on the session's plan: {'read_frontmatter': {'ticket': 1, 'status': 'built', 'baseline_revision': '1741b2bfb966faa929308854fe05522f0451f18a'}, 'tickettree._plan_fields': (True, 'built')}
sessions=1 phase=done attempt=1 commit=1819e483e1a3a95d2f2583dd416144edf26cf51c paused_stage=epic-boundary
dev-decision {'action': 'proceed', 'reason': ''}
story-done {}
rc=0  (4s)
### uv run python repro_bom.py bom none
readers on the session's plan: {'read_frontmatter': {}, 'tickettree._plan_fields': (True, 'built')}
sessions=1 phase=dev-verify attempt=1 commit=None paused_stage=escalation
spec-read-failed {'site': 'spec-deferrals', 'error': 'empty or invalid frontmatter'}
dev-decision {'action': 'retry', 'reason': 'spec unreadable while harvesting deferrals (empty or invalid frontmatter): <plan-path>'}
rollback-manual-required {}
rc=0  (2s)

Pure reads of the same plan file (observed, four byte variants): read_frontmatter and verify._gate_frontmatter return {} for BOM+CRLF and BOM+LF, and the full mapping for CRLF-only and plain; tickettree._plan_fields, tickets_engine._plan_state and tickets.py find report built for all four.

Ablation (observed): with frontmatter.py:145 changed to encoding="utf-8-sig" in a scratch copy of the package (put first on PYTHONPATH), the BOM worktree run ends phase=done attempt=1 with one session.

Expected vs actual

Linux

The same script on Ubuntu (WSL2, ext4), bmad-loop at the same commit from a git archive copy with its own uv venv, Python 3.13.2, git 2.43.0; 2 of 2 runs, identical apart from hashes. Displayed plan paths are replaced with ; all other transcript fields are unchanged.

### uv run python repro_bom.py plain worktree
readers on the session's plan: {'read_frontmatter': {'ticket': 1, 'status': 'built', 'baseline_revision': '4b957ac7fb6d83190340a71831f8486955628a5e'}, 'tickettree._plan_fields': (True, 'built')}
sessions=1 phase=done attempt=1 commit=de4f2869548f7ca6c5d605446eb241728b95ee30 paused_stage=epic-boundary
dev-decision {'action': 'proceed', 'reason': ''}
story-done {}
rc=0
### uv run python repro_bom.py bom worktree
readers on the session's plan: {'read_frontmatter': {}, 'tickettree._plan_fields': (True, 'built')}
readers on the session's plan: {'read_frontmatter': {}, 'tickettree._plan_fields': (True, 'built')}
sessions=2 phase=deferred attempt=2 commit=None paused_stage=None
spec-read-failed {'site': 'spec-deferrals', 'error': 'empty or invalid frontmatter'}
dev-decision {'action': 'retry', 'reason': 'spec unreadable while harvesting deferrals (empty or invalid frontmatter): <plan-path>'}
spec-read-failed {'site': 'spec-deferrals', 'error': 'empty or invalid frontmatter'}
dev-decision {'action': 'defer', 'reason': 'spec unreadable while harvesting deferrals (empty or invalid frontmatter): <plan-path>'}
story-deferred {'reason': 'spec unreadable while harvesting deferrals (empty or invalid frontmatter): <plan-path>'}
rc=0
### uv run python repro_bom.py plain none
readers on the session's plan: {'read_frontmatter': {'ticket': 1, 'status': 'built', 'baseline_revision': 'f4978144bd5b8ae58da1bde227f39f381816cc4a'}, 'tickettree._plan_fields': (True, 'built')}
sessions=1 phase=done attempt=1 commit=f12e4065dd88a7fccf0833f8b17df9ead9d468df paused_stage=epic-boundary
dev-decision {'action': 'proceed', 'reason': ''}
story-done {}
rc=0
### uv run python repro_bom.py bom none
readers on the session's plan: {'read_frontmatter': {}, 'tickettree._plan_fields': (True, 'built')}
sessions=1 phase=dev-verify attempt=1 commit=None paused_stage=escalation
spec-read-failed {'site': 'spec-deferrals', 'error': 'empty or invalid frontmatter'}
dev-decision {'action': 'retry', 'reason': 'spec unreadable while harvesting deferrals (empty or invalid frontmatter): <plan-path>'}
rollback-manual-required {}
rc=0

The Ubuntu/WSL2 mock-adapter runs match the Windows runs. The shared decoder and delimiter check are platform-independent; no real coding-CLI or tmux session was exercised by this script.

Source

At a5a8f097. The shared reader (observed returning {} above):

# src/bmad_loop/frontmatter.py:141-153
def read_frontmatter(path: Path) -> dict[str, Any]:
    if not path.is_file():
        return {}
    try:
        text = path.read_text(encoding="utf-8")
    except UnicodeDecodeError:
        ...
        return {}
    return parse_frontmatter(text)

The first gate that rejects the session (observed in the journal): Engine deferral harvest, src/bmad_loop/engine.py:5952-5976 (fm = verify.read_frontmatter(spec_path) then if not fm: … retry("spec unreadable while harvesting deferrals (empty or invalid frontmatter)")). Behind it, source-derived and not observed (the harvest stops the session first): verify_dev_tickets reads the bound plan through _gate_frontmatter (src/bmad_loop/verify.py:9422), which wraps read_frontmatter, so fm.get("ticket") is None and :9425-9429 retries "plan … names ticket None"; verify_review_tickets reads it the same way (verify.py:10483), and so does the retro document read at tickets_engine.py:686.

The tickets readers that already skip a BOM: tickettree._plan_fields (tickettree.py:641, "A leading UTF-8 byte-order mark is skipped, as the script keeps one it finds"), discard_unbuilt_plan (tickettree.py:550-553, added by f36665a4 in PR #939, which fixed only that one check), tickets_engine._plan_state (tickets_engine.py:236).

Suggested fix

Not validated beyond the ablation above: decode with utf-8-sig in frontmatter.read_frontmatter (one reader for every gate), or strip a leading U+FEFF in parse_frontmatter.