[Harness / GPT-6 Astra] Explicit criteria fail to govern scope and continuation after a GPT-5.6-era harness migration

The payload names no source files or tests; it points to the harness skill and prompt instructions, the debugging trigger reworded to "without new evidence", and the runtime task_complete event that fires after a final response. Start by reproducing a turn that names unfinished scope and check what ends it. Done would need a reproducible cause and a validation method, which the issue does not yet define.

Summary

Codex repeatedly failed to turn explicit, acknowledged completion requirements into the next action or the decision to keep working. This happened after several rounds of Codex-assisted migration of a GPT-5.6-era harness to GPT-6, including narrower skill triggers and less prescriptive workflows. Some feedback controls then had to be restored or strengthened. This is a report of observed behavior and a migration-validation gap, not proof that shorter instructions or one deleted rule caused every incident.

The strongest incident is two premature turn endings in the same authorized task. In each final response, GPT-6 Astra explicitly named the unfinished full scope. The runtime then recorded task_complete. A supervisor had to send another continuation message, and work resumed without new source material, creative selections, or expanded authorization. After the first concrete correction, the behavior repeated.

The user had already specified end-to-end delivery and usable product outcomes. The instructions already distinguished local tests, milestones, and actual completion. “Define completion criteria” alone does not address this failure: those criteria were present, and the agent demonstrated awareness of them while ending the turn.

Environment and evidence boundaries

• Codex desktop on macOS 26.5.2, arm64, signed in with a ChatGPT account.
• Installed version when filing: 26.1002.52244, build 13536. This is not a claim that all earlier incidents ran on this exact build.
• Both premature-ending turns were recorded as gpt-6-astra, reasoning effort ultra. This does not describe every downstream content-generation call.
• Evidence: visible user/assistant messages, actual tool calls/results, instruction read-backs, runtime completion events, historical file diffs, and delivered artifacts. Raw logs contain private source material and remain local; no hidden reasoning was inspected.
• These are forensic observations from ongoing work, not a controlled old/new model or old/new harness experiment. Internal causes remain to be investigated.
• All times below are UTC.

Incident A: knowingly ending an unfinished authoring-tool task, twice

The user requested a complete-book analysis and an authoring tool that preserves character resources and relationships, reflects changed premises in downstream consequences and subsequent writing, and supports a later culturally grounded adaptation. A subsequent instruction ordered source analysis before adaptation; it did not cancel either obligation. Representative chapters were explicitly a method check, not the final scope.

Time	Observed event
Oct 7, 07:46	User explicitly requested the full scope and end-to-end delivery with self-verification.
Oct 8, 07:24, 11:13, 12:40	Tool results show the agent reading delivery-convergence and independent-validation instructions. These distinguish representative success from completion and require continuing authorized remaining work.
Oct 8, 15:03	Updated collaboration instructions were injected, again preserving remaining obligations and allowing independent work to continue around a local blocker.
Oct 8, 16:49	Final response reported a validated two-chapter workflow and explicitly said full-book assembly and cross-chapter work were unfinished. It was immediately followed by a runtime task_complete.
Oct 8, 17:02	Supervisor instructed the same task to preserve the valid milestone and continue the outstanding scope without waiting for another user “continue.” Work resumed.
Oct 8, 21:01	The task ended again, explicitly reporting only four chapters covered and the remaining book, distant dependencies, and adaptation unfinished. Another task_complete followed.
Oct 8, 21:03	A second supervisory continuation restarted work without new source material, a creative selection, or broader authority.

There were genuine pending author choices and engineering problems. Those constrained dependent creative work; the nearby records did not establish that all remaining work required a user or external action. The subsequent continuation immediately found authorized engineering work to perform.

This is not a false claim that the whole book was completed. It is a different failure: an accurate limitations paragraph accompanied a terminal response instead of controlling continued execution. Compaction events exist, but the final responses themselves retained the original scope. That is insufficient evidence for “the goal was forgotten” or a compaction defect.

A separate content-path defect was found in the same project: per-act adaptation calls explicitly assembled a generated design, current act, source chapter IDs, and the last two generated chapters, without explicitly assembling the corresponding source events and chapter-level resource/relationship changes. The initial design did receive a source blueprint, so it would be inaccurate to say all such information was absent. Chapter-ID coverage nevertheless did not establish that the required source semantics reached the consumer.

Latest recurrence, Oct 9 at 01:01: the user again challenged whether the author-facing outline served the intended writing use, and the disproportionate token volume. This was not a third premature ending: the task was active and began investigating. A saved plot job's usage record and the visible tool result agree on 164,589 input tokens + 10,983 output tokens = 175,572, with cached tokens recorded as zero. The downstream job's model field was empty; its model must not be inferred from the controller's Astra configuration. A later zero-send reconstruction of the current request assembly measured 303,493 characters, with 12 units and 195 unknown items for one scene. That reconstruction is not the saved original wire request and must not be used to extrapolate whole-book cost. The outputs did contain goals, pressure, choices, and consequences; the issue is not that all narrative structure was absent. The actual author-facing UI also showed internal fact identifiers and lengthy provenance/audit restrictions alongside the creative alternatives. This recurrence shows why the user's actual writing use and the assembled request must be inspected, rather than inferring success from an accepted candidate or a local context check.

Incident B: strategic-review scope can be narrowed despite a broader written specification

A separate task was to build a tool for reviewing an existing corporate strategic plan: whether market/customer/self-assessment supports strategic choices, whether those choices connect to implementation and resources, and whether the resulting commitments follow. On Oct 8 at 09:42 and 10:44, the user explicitly required each substantive assessment, choice, and implementation item to be expanded, with many-to-many relationships preserved. The user specifically required ontology, causal relationships, and time/resource constraints to contribute to the actual review. Later feedback requested readability and suggested working backward from financial outcomes, while retaining the wider strategy and non-financial obligations.

The delivered specification preserved that broader scope. The delivered review implementation, however, iterated over manually selected agenda.targets; the main report covered six selected questions. Keeping other statements in a graph or appendix was not the same as reviewing their substantive support. Tests and an independent review were recorded as passing, but the user again rejected the result as missing the original goal on Oct 9 at 00:49.

A read-only check of the delivered commit found a concrete consumer defect: changing a customer condition from unknown to false changed the saved underlying condition-execution result from UNKNOWN_PRECONDITION to PRECONDITION_FAILED, yet the entire reader-review object remained identical. The report builder consumed calculation/source data but not those condition-execution results. The declared integration therefore did not establish that a changed causal condition could affect the user's review. Another finding was a claim of a missing baseline coexisting with an already supplied baseline: “present but insufficiently verified” and “absent” had not been distinguished.

This is a scope/consumer-path question, not merely the user's preference for a different writing style. The investigation distinguishes the delivered commit from revisions started after that rejection. The private company, figures, documents, and raw output are not included here.

Related observation: a source-grounded interactive-world task

Another task explicitly required a source-grounded explorable world with character knowledge, action conditions, consequences, and save/restore. The controller wrote six completion conditions and actually read them back through a tool. On Oct 6 at 10:41, after inspecting missing actor-owned state, it explicitly required a real state-cut/private-view check before larger extraction batches. At 20:07 it dispatched another 25 windows using the old extraction contract. At 21:55 the repaired cut still had unknown status and no actor self-states; at 21:56 another 25-window batch started. At 23:33 the controller itself recognized that missing mental-state semantics in the old contract could require another full extraction, and changed the route.

The observed inconsistency is between the agent's stated evidence-based prerequisite and its later tool scheduling. Source extraction and real cut repair also continued in parallel; some extraction was valid independent work within the user's complete-book scope. The report does not classify every batch as wasted or every unfinished milestone as a failure. The relevant world-task configuration was Astra/max on Oct 6 and Astra/ultra from Oct 7 onward.

An earlier retrospective overstated the chronology: a cited independent critical-path review at 20:26 was produced after the 20:07 batch had already started. We have corrected that inference. That review did correctly say delivery was incomplete, but suggested usage validation after full compilation without establishing that the entire corpus was necessary for the early representative check. Three inspected compaction records retained the core user requirements. Neither missing criteria nor demonstrated deletion of the goal explains these observed scheduling inconsistencies.

What the harness migration actually changed

• Sep 5: Codex recommended preserving goals, evidence, permissions, and completion requirements while giving the model more freedom over methods. One debugging trigger changed from repeated failed fixes plus moving symptoms/shared-state coupling to reconsideration when attempts fail “without new evidence.” This is a real semantic change, not just fewer words. But the old failed-fix threshold might also miss local successes with an unusable overall result; no controlled comparison proves it would have prevented these incidents.
• Sep 12: A later revision followed the official GPT-6 Astra skills and prompts article. It narrowed triggers and reduced prescriptive loading. That article was published Sep 11, so it cannot be the cause of the Sep 5 change. It also recommends defining completion and following through; it does not recommend removing all safeguards.
• Sep 16 and Oct 3: Instructions already explicitly required revisiting a route when local success creates downstream failure, testing representative user outcomes early, and checking shared state and consumer contracts when failures move between stages.
• Oct 6: Independent validation was strengthened to use original user requirements and corrections rather than only the implementer's summary/checklist.
• Oct 8: Further rules required an unresolved user-facing counterexample to influence the actual next action, even after new logs, local passes, or technical recovery. They were loaded before the two endings above.

There were real installation checks, runtime tests, scenario reviews, and later two small synthetic actor tasks with continuation. It would be false to say no validation occurred. Their coverage did not establish preservation of the feedback function in long-running, evolving product tasks. Some checked that a reviewer could identify a defect, not that the working agent would change route or continue after a valid milestone. The later synthetic runs were not long-horizon migration A/B tests.

The local migration assistant therefore shares responsibility: it recommended and implemented the changes without demonstrating functional equivalence for this failure mode. Conversely, restoring selected feedback functions is not evidence for restoring every old approval, fixed retry count, timeout, or mandatory workflow. The practical distinction is removing unnecessary procedure versus preserving goal feedback and completion behavior.

Expected behavior and investigation requested

1. A valid milestone plus acknowledged remaining authorized work should not by itself produce terminal completion. If an external dependency blocks one branch, continue independent obligations. If the turn must end because of a real platform boundary, make that state distinguishable from task completion.
2. A plan, selected agenda, sample, or subagent checklist must not silently become the scope of the user's request. Check the delivered consumer against the original requirements and later corrections.
3. A known user-facing counterexample should affect the next dependent action and be rechecked through the same consumer path. A reviewer saying “not complete” while the actor follows an unchanged, unjustified route is not enough.
4. Evaluate migration advice by the function preserved, not prompt length or a reviewer-only score. Useful regressions include a valid partial milestone with unfinished scope, pending local choices with independent work available, technical recovery followed by renewed goal selection, and a readability correction that must not narrow the review universe.
5. Investigate model/harness interaction using visible action and stopping traces. These logs do not identify whether planning, long-context handling, stopping policy, review scoping, or a combination is the internal cause.

I am reporting observed failures despite clear criteria, not asking for unrestricted autonomy, bypassed safety boundaries, or a guarantee that every task always succeeds. Merely suggesting one more equivalent sentence in AGENTS.md would not establish a fix.

Related but not proven to share a cause: #44974 and #30937. An earlier local model-default issue was investigated separately; no evidence establishes it as the cause of these goal-preservation failures.