[Bug]: A delegated task's message held in its child's queue never tells its owner, and settlement cancels it as just "cancelled"

Đang mở
#39,511 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
45/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
typescript
Lĩnh vực
backend-api-design

Hướng nghiên cứu

Start with the failing cases described for apps/server/src/orchestration-v2/DelegatedCompletionDelivery.test.ts, then trace OrchestratorMcpService.cancelTask, stopChild, holdStoppedThread, finalizeAppOwnedSubagentTask, dispatchMessage, and resultOfRun. Reproduce both gaps with the provided test command; done means a held follow-up notifies its owner and a settlement result says the message was never read.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

What happened

A delegated task's message sat in its child thread's held queue for six days, and nobody was told. A person had settled the parent thread. After a server update, the parent got the notice "Child task ended with status cancelled." It did not say that the message never ran, or why.

Diagnosis

There are two gaps on the same path, and both reproduce on main (8e8ff9b243e5).

  1. A held task message gives its owner no signal. task_cancel stops the child with thread.stop (OrchestratorMcpService.cancelTask → stopChild). Stop holds every queued turn in the child (holdStoppedThread) and then cancels only the cancelled task's own queued turn. The child's other tasks stay held:

    • finalizeAppOwnedSubagentTask moves each of those tasks to idle.
    • idle is not active work, and nothing wakes the task's owner.
    • Per the comment in dispatchMessage, only a person's client message (or Resume) releases a held queue.

    So the owner waits forever, or forgets about the task. A user Stop and a provider-failure hold have the same effect.

    A send that has no sender thread (an external MCP client acting for the owner) is recorded by recordSubagentFollowup as a follow-up task of the child's parent. In that case the parent does not even know the task exists.

  2. Settlement cancels such a message without saying so. thread.settle cancels automatic (agent-sent) queued runs, held ones included. The task's result then falls through to Child task ended with status ${run.status}. in resultOfRun. That reads like the child stopped working. It does not say that the child never read the message.

    Since #19665, subagent threads settle as soon as their parent is settled. So settling a parent can now cancel these messages and wake the parent it just settled.

In the case above:

  • An external MCP client sent a follow-up to a delegated child while the child was busy.
  • Four hours later, the parent called task_cancel on two other tasks of that child. That stopped the child and held the follow-up. Its task went idle.
  • Six days later, the server was updated to a build that includes #19665. Its first settlement sweep settled 723 subagent threads of settled parents. This child was one of them, so the follow-up was cancelled, and the parent's result said only "ended with status cancelled".
Steps to reproduce

Add these two tests to describe("subagent follow-ups") in apps/server/src/orchestration-v2/DelegatedCompletionDelivery.test.ts, after "pauses queued follow-up cards and resumes them", then run:

cd apps/server && vp test run src/orchestration-v2/DelegatedCompletionDelivery.test.ts -t "held follow-up tells|cancelled unread by settlement"

Each test does the following:

  1. Seed a parent whose delegated child finished its first task.
  2. Send a follow-up with a sender thread so it starts running, then send one without a sender thread, as an MCP client does. The second one queues as a tracked follow-up.
  3. Dispatch thread.stop on the child, as task_cancel does. The queued follow-up's task becomes idle.
  4. First test: check whether the parent received any message about that task. It did not.
  5. Second test: end the running turn, settle the child, and read the task's result.

Both tests fail 3/3 on 8e8ff9b243e5:

AssertionError: the owner was never told its tracked message is held: expected false to be true
AssertionError: expected 'Child task ended with status cancelle…' to not equal 'Child task ended with status cancelle…'
Repro tests (diff against 8e8ff9b243e5)
@@ -2464,6 +2464,138 @@ it.layer(Layer.merge(layerTest, ProviderContinuationRequests.layer))(
       }),
     );
 
+    // A client (no sender thread) sends a follow-up that queues behind the child's running
+    // turn; Stop on the child (as task_cancel on another task does) then holds it.
+    const holdClientFollowUp = (name: string) =>
+      Effect.gen(function* () {
+        const orchestrator = yield* Orchestrator.OrchestratorV2;
+        const eventSink = yield* EventSink.EventSinkV2;
+        const now = yield* DateTime.now;
+        const parentThreadId = ThreadId.make(`thread:${name}-parent`);
+        const projectId = ProjectId.make(`project:${name}`);
+        const parentRunId = RunId.make(`run:${name}-parent`);
+        const rootNodeId = NodeId.make(`node:${name}-parent`);
+        yield* seedParentWithTerminalTask({
+          threadId: parentThreadId,
+          projectId,
+          runId: parentRunId,
+          rootNodeId,
+          taskId: NodeId.make(`node:${name}-seed`),
+          deliveryState: "acknowledged",
+          now,
+        });
+        const child = yield* seedRestartCancelledChild({
+          parentThreadId,
+          projectId,
+          parentRunId,
+          rootNodeId,
+          name: `${name}-child`,
+          completionWake: "always",
+          continuationPending: false,
+          runStatus: "completed",
+          now,
+        });
+        yield* orchestrator.recoverDelegatedTasks;
+        const send = (text: string, senderThreadId?: ThreadId) =>
+          orchestrator.dispatch({
+            type: "message.dispatch",
+            createdBy: "agent",
+            creationSource: "mcp",
+            commandId: CommandId.make(`command:${name}:${text}`),
+            threadId: child.childThreadId,
+            ...(senderThreadId === undefined ? {} : { senderThreadId }),
+            messageId: MessageId.make(`message:${name}:${text}`),
+            text,
+            attachments: [],
+            dispatchMode: { type: "queue_after_active" },
+          });
+        yield* send("active", parentThreadId);
+        yield* send("from-client");
+        const before = yield* orchestrator.getThreadProjection(child.childThreadId);
+        const active = before.runs.find((run) => run.ordinal === 2)!;
+        const queued = before.runs.find((run) => run.ordinal === 3)!;
+        assert.equal(queued.status, "queued");
+        assert.isDefined(queued.delegatedTaskId);
+        const waitForTask = (afterSequence: number, taskId: NodeId, status: string) =>
+          eventSink.stream({ afterSequence, eventType: "subagent.updated" }).pipe(
+            Stream.filter(
+              (stored) =>
+                stored.event.type === "subagent.updated" &&
+                stored.event.payload.id === taskId &&
+                stored.event.payload.status === status,
+            ),
+            Stream.take(1),
+            Stream.runHead,
+          );
+        const stopSequence = yield* eventSink.latestSequence();
+        yield* orchestrator.dispatch({
+          type: "thread.stop",
+          threadId: child.childThreadId,
+          commandId: CommandId.make(`command:${name}:stop`),
+        });
+        yield* waitForTask(stopSequence, queued.delegatedTaskId!, "idle");
+        return { parentThreadId, child, active, queued, waitForTask };
+      });
+
+    it.effect("a held follow-up tells the thread that owns its task", () =>
+      Effect.gen(function* () {
+        const orchestrator = yield* Orchestrator.OrchestratorV2;
+        const { parentThreadId, queued } = yield* holdClientFollowUp("held-client-followup");
+        const parent = yield* orchestrator.getThreadProjection(parentThreadId);
+        const task = parent.subagents.find((row) => row.id === queued.delegatedTaskId)!;
+        assert.equal(task.status, "idle");
+        assert.equal(task.completionWake, "always");
+        // Nothing but a person's send or Resume releases the hold, so the owner must hear of it.
+        const told = parent.messages.some(
+          (message) =>
+            message.notification?.source.kind === "delegated_task" &&
+            message.notification.source.taskIds.includes(task.id),
+        );
+        assert.isTrue(told, "the owner was never told its tracked message is held");
+      }),
+    );
+
+    it.effect("a follow-up cancelled unread by settlement says it never ran", () =>
+      Effect.gen(function* () {
+        const orchestrator = yield* Orchestrator.OrchestratorV2;
+        const eventSink = yield* EventSink.EventSinkV2;
+        const now = yield* DateTime.now;
+        const { parentThreadId, child, active, queued, waitForTask } =
+          yield* holdClientFollowUp("settled-client-followup");
+        const endedSequence = yield* eventSink.latestSequence();
+        yield* eventSink.write({
+          events: [
+            {
+              id: EventId.make("event:settled-client-followup-active-ended"),
+              type: "run.updated",
+              threadId: child.childThreadId,
+              runId: active.id,
+              occurredAt: now,
+              payload: { ...active, status: "interrupted", startedAt: now, completedAt: now },
+            },
+          ],
+        });
+        const activeTask = (yield* orchestrator.getThreadProjection(parentThreadId)).subagents.find(
+          (row) => row.id === active.delegatedTaskId,
+        )!;
+        if (activeTask.result === null) {
+          yield* waitForTask(endedSequence, active.delegatedTaskId!, "interrupted");
+        }
+        const settleSequence = yield* eventSink.latestSequence();
+        yield* orchestrator.dispatch({
+          type: "thread.settle",
+          threadId: child.childThreadId,
+          commandId: CommandId.make("command:settled-client-followup:settle"),
+        });
+        yield* waitForTask(settleSequence, queued.delegatedTaskId!, "cancelled");
+        const task = (yield* orchestrator.getThreadProjection(parentThreadId)).subagents.find(
+          (row) => row.id === queued.delegatedTaskId,
+        )!;
+        assert.notEqual(task.result, "Child task ended with status cancelled.");
+        assert.match(task.result ?? "", /never/);
+      }),
+    );
+
     it.effect("checkpoint retry releases a pending sibling after its wake offer failed", () =>
       Effect.gen(function* () {
         const orchestrator = yield* Orchestrator.OrchestratorV2;
Version

Seen on a fork build of 0.1.11-nightly.20261117.4825 (upgraded from 0.1.9-nightly.20261111.4501). Reproduced on main at 8e8ff9b243e5.

Environment

Linux x64, server run as a service, Claude provider. The repro uses the test harness's Codex stub and is provider-independent.

Evidence

Child thread runs, from the projections in the live database, opened read-only:

ordinal status queueHeld requested completed
2 interrupted – 11-14 04:45Z 11-14 14:11:40Z (task_cancel's Stop)
3 cancelled true 11-14 09:10Z 11-14 14:11:39Z (the cancelled task's own turn)
4 cancelled – 11-14 09:19Z 11-14 14:11:39Z (buffered notice dropped by Stop)
5 cancelled true 11-14 10:12Z 11-20 02:48:33Z (settlement)
  • Run 5's task, a follow-up from an external MCP client recorded on the parent with completionWake: "always", was idle from 11-14 14:11Z until 11-20 02:48Z. Its result was Child task ended with status cancelled.
  • The server trace shows thread.auto-settle on the child at 02:48:32.8Z, 35 s after the upgraded server started. That startup sweep dispatched 723 server:auto-settle: commands for subagent threads.
  • The parent's unsettledAt is 02:48:34.7Z, set by the wake.
Related issues
  • #22423: a queued message held after a provider crash is never delivered. That covers the same hold, from the user's side.
  • #26119 (closed): task_cancel docs on queued turns.
  • #19665: subagent threads settle with their parent.
  • #22925: the parent is not told when a child waits for an approval. That is the same kind of silent-child gap.
Fix applied or workaround

A fix is in our fork, for maintainers to use or replace as they see fit:

  • A small reactor, modelled on DelegatedTaskQuestionReactor, sends the owner one delegated_task notice (outcome updated) per hold. It skips the notice when the owner itself sent into an already-held queue.
  • The notice uses the question-notice guards, so Stop, task_cancel and archive drop it.
  • isStaleQuestionWake treats it as stale once the child's run is no longer held. That check runs at dispatch, at queue start and at the Claude tool-boundary offer.
  • resultOfRun reports a message cancelled before its run started as: "The child never read this message: it was cancelled before its run started[, while held in the child's queue][, when the child's thread settled]. Send it again if it is still needed."

Fork commit: https://github.com/Alb11747/t3code/commit/b4efbf983cefc40e2dcaa87b4edbc6458df09a56 (tests included).

The workaround: after task_cancel, check the child with task_status for other idle tasks, and cancel them or Resume the child.

Filed by

Filed by Claude Opus 5.5 (T3 Code) on behalf of Alb11747.

Ngôn ngữ chính
TypeScript
Star
24.8k
Fork
6.4k
Merge trung bình
4 giờ 29 phút
Pull request đã merge (30 ngày)
488

Chuẩn bị môi trường

Mở trong Codespaces

Khởi chạy dev container của dự án ngay trên trình duyệt, bằng tài khoản GitHub của bạn.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của pingdotgg/t3code

Tất cả issue của pingdotgg/t3code

Issue tương tự

Thêm issue về TypeScript

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.