[Bug]: A delegated task's message held in its child's queue never tells its owner, and settlement cancels it as just "cancelled"

Aperta
#39,511 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
typescript

Direzione di ricerca

Start with the failing cases described for apps/server/src/orchestration-v2/DelegatedCompletionDelivery.test.ts, then trace OrchestratorMcpService.cancelTask, stopChild, holdStoppedThread, finalizeAppOwnedSubagentTask, dispatchMessage, and resultOfRun. Reproduce both gaps with the provided test command; done means a held follow-up notifies its owner and a settlement result says the message was never read.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What happened

A delegated task's message sat in its child thread's held queue for six days, and nobody was told. A person had settled the parent thread. After a server update, the parent got the notice "Child task ended with status cancelled." It did not say that the message never ran, or why.

Diagnosis

There are two gaps on the same path, and both reproduce on main (8e8ff9b243e5).

  1. A held task message gives its owner no signal. task_cancel stops the child with thread.stop (OrchestratorMcpService.cancelTask → stopChild). Stop holds every queued turn in the child (holdStoppedThread) and then cancels only the cancelled task's own queued turn. The child's other tasks stay held:

    • finalizeAppOwnedSubagentTask moves each of those tasks to idle.
    • idle is not active work, and nothing wakes the task's owner.
    • Per the comment in dispatchMessage, only a person's client message (or Resume) releases a held queue.

    So the owner waits forever, or forgets about the task. A user Stop and a provider-failure hold have the same effect.

    A send that has no sender thread (an external MCP client acting for the owner) is recorded by recordSubagentFollowup as a follow-up task of the child's parent. In that case the parent does not even know the task exists.

  2. Settlement cancels such a message without saying so. thread.settle cancels automatic (agent-sent) queued runs, held ones included. The task's result then falls through to Child task ended with status ${run.status}. in resultOfRun. That reads like the child stopped working. It does not say that the child never read the message.

    Since #19665, subagent threads settle as soon as their parent is settled. So settling a parent can now cancel these messages and wake the parent it just settled.

In the case above:

  • An external MCP client sent a follow-up to a delegated child while the child was busy.
  • Four hours later, the parent called task_cancel on two other tasks of that child. That stopped the child and held the follow-up. Its task went idle.
  • Six days later, the server was updated to a build that includes #19665. Its first settlement sweep settled 723 subagent threads of settled parents. This child was one of them, so the follow-up was cancelled, and the parent's result said only "ended with status cancelled".
Steps to reproduce

Add these two tests to describe("subagent follow-ups") in apps/server/src/orchestration-v2/DelegatedCompletionDelivery.test.ts, after "pauses queued follow-up cards and resumes them", then run:

cd apps/server && vp test run src/orchestration-v2/DelegatedCompletionDelivery.test.ts -t "held follow-up tells|cancelled unread by settlement"

Each test does the following:

  1. Seed a parent whose delegated child finished its first task.
  2. Send a follow-up with a sender thread so it starts running, then send one without a sender thread, as an MCP client does. The second one queues as a tracked follow-up.
  3. Dispatch thread.stop on the child, as task_cancel does. The queued follow-up's task becomes idle.
  4. First test: check whether the parent received any message about that task. It did not.
  5. Second test: end the running turn, settle the child, and read the task's result.

Both tests fail 3/3 on 8e8ff9b243e5:

AssertionError: the owner was never told its tracked message is held: expected false to be true
AssertionError: expected 'Child task ended with status cancelle…' to not equal 'Child task ended with status cancelle…'
Repro tests (diff against 8e8ff9b243e5)
@@ -2464,6 +2464,138 @@ it.layer(Layer.merge(layerTest, ProviderContinuationRequests.layer))(
       }),
     );
 
+    // A client (no sender thread) sends a follow-up that queues behind the child's running
+    // turn; Stop on the child (as task_cancel on another task does) then holds it.
+    const holdClientFollowUp = (name: string) =>
+      Effect.gen(function* () {
+        const orchestrator = yield* Orchestrator.OrchestratorV2;
+        const eventSink = yield* EventSink.EventSinkV2;
+        const now = yield* DateTime.now;
+        const parentThreadId = ThreadId.make(`thread:${name}-parent`);
+        const projectId = ProjectId.make(`project:${name}`);
+        const parentRunId = RunId.make(`run:${name}-parent`);
+        const rootNodeId = NodeId.make(`node:${name}-parent`);
+        yield* seedParentWithTerminalTask({
+          threadId: parentThreadId,
+          projectId,
+          runId: parentRunId,
+          rootNodeId,
+          taskId: NodeId.make(`node:${name}-seed`),
+          deliveryState: "acknowledged",
+          now,
+        });
+        const child = yield* seedRestartCancelledChild({
+          parentThreadId,
+          projectId,
+          parentRunId,
+          rootNodeId,
+          name: `${name}-child`,
+          completionWake: "always",
+          continuationPending: false,
+          runStatus: "completed",
+          now,
+        });
+        yield* orchestrator.recoverDelegatedTasks;
+        const send = (text: string, senderThreadId?: ThreadId) =>
+          orchestrator.dispatch({
+            type: "message.dispatch",
+            createdBy: "agent",
+            creationSource: "mcp",
+            commandId: CommandId.make(`command:${name}:${text}`),
+            threadId: child.childThreadId,
+            ...(senderThreadId === undefined ? {} : { senderThreadId }),
+            messageId: MessageId.make(`message:${name}:${text}`),
+            text,
+            attachments: [],
+            dispatchMode: { type: "queue_after_active" },
+          });
+        yield* send("active", parentThreadId);
+        yield* send("from-client");
+        const before = yield* orchestrator.getThreadProjection(child.childThreadId);
+        const active = before.runs.find((run) => run.ordinal === 2)!;
+        const queued = before.runs.find((run) => run.ordinal === 3)!;
+        assert.equal(queued.status, "queued");
+        assert.isDefined(queued.delegatedTaskId);
+        const waitForTask = (afterSequence: number, taskId: NodeId, status: string) =>
+          eventSink.stream({ afterSequence, eventType: "subagent.updated" }).pipe(
+            Stream.filter(
+              (stored) =>
+                stored.event.type === "subagent.updated" &&
+                stored.event.payload.id === taskId &&
+                stored.event.payload.status === status,
+            ),
+            Stream.take(1),
+            Stream.runHead,
+          );
+        const stopSequence = yield* eventSink.latestSequence();
+        yield* orchestrator.dispatch({
+          type: "thread.stop",
+          threadId: child.childThreadId,
+          commandId: CommandId.make(`command:${name}:stop`),
+        });
+        yield* waitForTask(stopSequence, queued.delegatedTaskId!, "idle");
+        return { parentThreadId, child, active, queued, waitForTask };
+      });
+
+    it.effect("a held follow-up tells the thread that owns its task", () =>
+      Effect.gen(function* () {
+        const orchestrator = yield* Orchestrator.OrchestratorV2;
+        const { parentThreadId, queued } = yield* holdClientFollowUp("held-client-followup");
+        const parent = yield* orchestrator.getThreadProjection(parentThreadId);
+        const task = parent.subagents.find((row) => row.id === queued.delegatedTaskId)!;
+        assert.equal(task.status, "idle");
+        assert.equal(task.completionWake, "always");
+        // Nothing but a person's send or Resume releases the hold, so the owner must hear of it.
+        const told = parent.messages.some(
+          (message) =>
+            message.notification?.source.kind === "delegated_task" &&
+            message.notification.source.taskIds.includes(task.id),
+        );
+        assert.isTrue(told, "the owner was never told its tracked message is held");
+      }),
+    );
+
+    it.effect("a follow-up cancelled unread by settlement says it never ran", () =>
+      Effect.gen(function* () {
+        const orchestrator = yield* Orchestrator.OrchestratorV2;
+        const eventSink = yield* EventSink.EventSinkV2;
+        const now = yield* DateTime.now;
+        const { parentThreadId, child, active, queued, waitForTask } =
+          yield* holdClientFollowUp("settled-client-followup");
+        const endedSequence = yield* eventSink.latestSequence();
+        yield* eventSink.write({
+          events: [
+            {
+              id: EventId.make("event:settled-client-followup-active-ended"),
+              type: "run.updated",
+              threadId: child.childThreadId,
+              runId: active.id,
+              occurredAt: now,
+              payload: { ...active, status: "interrupted", startedAt: now, completedAt: now },
+            },
+          ],
+        });
+        const activeTask = (yield* orchestrator.getThreadProjection(parentThreadId)).subagents.find(
+          (row) => row.id === active.delegatedTaskId,
+        )!;
+        if (activeTask.result === null) {
+          yield* waitForTask(endedSequence, active.delegatedTaskId!, "interrupted");
+        }
+        const settleSequence = yield* eventSink.latestSequence();
+        yield* orchestrator.dispatch({
+          type: "thread.settle",
+          threadId: child.childThreadId,
+          commandId: CommandId.make("command:settled-client-followup:settle"),
+        });
+        yield* waitForTask(settleSequence, queued.delegatedTaskId!, "cancelled");
+        const task = (yield* orchestrator.getThreadProjection(parentThreadId)).subagents.find(
+          (row) => row.id === queued.delegatedTaskId,
+        )!;
+        assert.notEqual(task.result, "Child task ended with status cancelled.");
+        assert.match(task.result ?? "", /never/);
+      }),
+    );
+
     it.effect("checkpoint retry releases a pending sibling after its wake offer failed", () =>
       Effect.gen(function* () {
         const orchestrator = yield* Orchestrator.OrchestratorV2;
Version

Seen on a fork build of 0.1.11-nightly.20261117.4825 (upgraded from 0.1.9-nightly.20261111.4501). Reproduced on main at 8e8ff9b243e5.

Environment

Linux x64, server run as a service, Claude provider. The repro uses the test harness's Codex stub and is provider-independent.

Evidence

Child thread runs, from the projections in the live database, opened read-only:

ordinal status queueHeld requested completed
2 interrupted – 11-14 04:45Z 11-14 14:11:40Z (task_cancel's Stop)
3 cancelled true 11-14 09:10Z 11-14 14:11:39Z (the cancelled task's own turn)
4 cancelled – 11-14 09:19Z 11-14 14:11:39Z (buffered notice dropped by Stop)
5 cancelled true 11-14 10:12Z 11-20 02:48:33Z (settlement)
  • Run 5's task, a follow-up from an external MCP client recorded on the parent with completionWake: "always", was idle from 11-14 14:11Z until 11-20 02:48Z. Its result was Child task ended with status cancelled.
  • The server trace shows thread.auto-settle on the child at 02:48:32.8Z, 35 s after the upgraded server started. That startup sweep dispatched 723 server:auto-settle: commands for subagent threads.
  • The parent's unsettledAt is 02:48:34.7Z, set by the wake.
Related issues
  • #22423: a queued message held after a provider crash is never delivered. That covers the same hold, from the user's side.
  • #26119 (closed): task_cancel docs on queued turns.
  • #19665: subagent threads settle with their parent.
  • #22925: the parent is not told when a child waits for an approval. That is the same kind of silent-child gap.
Fix applied or workaround

A fix is in our fork, for maintainers to use or replace as they see fit:

  • A small reactor, modelled on DelegatedTaskQuestionReactor, sends the owner one delegated_task notice (outcome updated) per hold. It skips the notice when the owner itself sent into an already-held queue.
  • The notice uses the question-notice guards, so Stop, task_cancel and archive drop it.
  • isStaleQuestionWake treats it as stale once the child's run is no longer held. That check runs at dispatch, at queue start and at the Claude tool-boundary offer.
  • resultOfRun reports a message cancelled before its run started as: "The child never read this message: it was cancelled before its run started[, while held in the child's queue][, when the child's thread settled]. Send it again if it is still needed."

Fork commit: https://github.com/Alb11747/t3code/commit/b4efbf983cefc40e2dcaa87b4edbc6458df09a56 (tests included).

The workaround: after task_cancel, check the child with task_status for other idle tasks, and cancel them or Resume the child.

Filed by

Filed by Claude Opus 5.5 (T3 Code) on behalf of Alb11747.

Lingua principale
TypeScript
Stelle
24.8k
Fork
6.4k
Merge medio
4h 29m
PR unite (30g)
488

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di pingdotgg/t3code

Tutte le issue di pingdotgg/t3code

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.