RFC 001: Execution Record

This is an RFC for design review, not a task with a defined change. Read the linked RFC 002 and RFC 003, then the current queue and timer code in packages/sqlite-runtime/src/dao/task-queue-dao.ts, timers-dao.ts and sqlite-store.ts to see how the event is copied per row. Done means a maintainer-agreed design, since the conformance table ends in an unfinished "Outcome tests will ..." line.

Abstract

A workflow execution has no durable record of its own. It exists only as recorded steps and as copies of its event in every queued task, timer and store waiter. Therefore, after a restart nothing can report how an execution ended, nothing prevents its ID from being reused, and two wakes that arrive together deliver it twice.

We will solve this by giving every execution one durable record, created when it is admitted, which stores its event once along with its status, outcome and pending wake. Wakes will then name an execution instead of carrying a copy of its event, so any number of wakes will collapse into one delivery and a finished execution will never run again.

https://github.com/user-attachments/assets/f35dd1d0-97eb-4987-b9a3-71883d52c931

Video transcript

Yieldstar runs workflows: code that can pause for minutes or days, and survive a restart. Take a shop. When a customer places order 42, the shop triggers a workflow called charge-order, and passes it one param: the order number, 42. The workflow waits ten minutes, in case the customer cancels. Then it loads the order, and if the order is still open, it charges the card.

The workflow is made of steps: the wait, the load, and the charge. Each time the shop triggers the workflow, Yieldstar starts a new execution, and gives it an execution ID. As each step finishes, Yieldstar saves the step's result under that ID. If the process restarts, Yieldstar runs the workflow again from the top. Each finished step finds its saved result, and returns it instead of running again. So the execution ID is how a rerun finds its own saved steps, and does not charge the card a second time.

The shop can choose the execution ID itself. The param and the ID do different jobs. The param, 42, tells the workflow which order to charge. The ID names this one execution. The shop builds the ID from the order number, "order-42", for a reason. If the shop ever sends the same trigger twice, both triggers will carry the same ID, and the shop expects Yieldstar to treat them as one execution.

When the shop triggers the workflow, Yieldstar puts a task in the queue: run charge-order, with execution ID order-42 and param 42. A worker takes the task and runs the workflow. When the workflow reaches its ten minute wait, the worker stops, and Yieldstar writes a timer with the same details, set for 12:10. When the timer fires, it puts another task in the queue, and a worker runs the workflow from the top. The saved steps return their results, and the workflow carries on after the wait. We'll call the task and the timer wakes. So Yieldstar keeps the saved steps, and it keeps the wakes. But it keeps no record of the execution itself.

Problem 1. Yieldstar cannot answer a simple question: what happened to execution order-42? Say the shop sends the trigger, and the request times out. The shop does not know whether Yieldstar received it. Did the execution start? Is it waiting? Did it charge the card? To find out, the shop would look the execution up by its ID. But there is no record to look up. So the shop has one option that looks safe: send the trigger again, with the same ID.

Problem 2. Yieldstar does not treat the second trigger as the same execution. Say the first trigger did arrive, and order-42 is in its ten minute wait. The second trigger puts a new task in the queue, with the same ID. A worker runs it. The wait step finds its saved finish time, 12:10, so the workflow is still waiting, and Yieldstar writes another timer, for the same time. Now order-42 has two timers. At 12:10 both fire, and the queue holds two tasks. Say two workers take them at the same moment. Neither worker finds a saved charge, because neither has finished charging yet. So both workers charge the card.

Problem 3 is the same blindness, from the other side. Say the customer adds an item to order 42, and the shop needs to charge for it. The shop triggers charge-order again, and as before, it builds the ID from the order number: "order-42". This time, the shop means a new execution. But the workflow runs, and every step finds a saved result under order-42: the old wait, the old order, and the old receipt. So the execution returns the old receipt, and the new item is never charged.

Put the three problems side by side. The shop cannot ask what happened to order-42. A repeated trigger for order-42 runs the charge twice. A new trigger that reuses order-42 gets the old results. Each time, Yieldstar needs to answer the same question: does execution order-42 already exist? And each time, it has nowhere to look, because it keeps no record of the execution.

The fix will be to give each execution one record, created before anything runs. When the shop triggers the workflow, Yieldstar will first look for a record with that ID. If there is none, Yieldstar will create one, with the ID, the workflow name, the params, and a status: admitted. This step is called admission. If a record with that ID already exists, admission will fail with a conflict, and the HTTP trigger will return 409.

Now run the three problems again. The repeated trigger for order-42 will get a 409. No second task will be queued, and no second timer written, and the 409 tells the shop what it wanted to know: order-42 exists. The new trigger for the extra item will also get a 409, so the shop will learn straight away that it needs a new ID, like order-42-2, instead of getting the old receipt. And the question, what happened to order-42, will have a place to find its answer: the record.

The record will also change how wakes work. A wake will no longer copy the workflow name and params. It will only ask Yieldstar to run order-42 at a given time, and that request will be stored on the record, as one due time. If two wakes ask for order-42 at once, like the two timers in problem 2, they will merge into one due time, the earlier of the two. A worker will claim order-42 once. Yieldstar will not hand order-42 to another worker until the first one reports back, or takes too long. So order-42 will run in one place at a time.

When the worker reports back, the record will store how the run ended: suspended, if the workflow is waiting again; completed, with the receipt; or failed, with the error. Completed and failed will be final. If order-42 is delivered again later, Yieldstar will return the saved outcome, without running the workflow.

Before, Yieldstar knew an execution only through its saved steps and its wakes, so it could not say whether order-42 existed, or how it ended. With one record per execution, a repeated trigger will be rejected, a reused ID will be caught, wakes will merge into one run, and the outcome will stay on the record.

Problem

Yieldstar records each completed step in step_responses (step-response-dao.ts). The workflow result is recorded as one more step, keyed $$workflow-result$$ (workflow.ts).

The execution itself is never recorded. Its event (workflowId, executionId, params and context) is copied into each row that can wake it:

• a queued task in task_queue (task-queue-dao.ts);
• a timer in scheduled_tasks (timers-dao.ts); and
• a store waiter in store_waiters.event (sqlite-store.ts).

This causes three problems.

First, nothing durable can say whether an execution is running, suspended, completed or failed. triggerAndWait learns the result from an in-memory emitter (sdk-local.ts):

const result = await sdk.triggerAndWait({
  workflowId: "process-job",
  params: { jobId: "job-1" },
})

// If the runtime restarts before the workflow ends, this promise never settles,
// and no record says whether the execution completed or failed.

Second, every wake is an independent row with its own copy of the event. When a timer fires and a store change arrives for the same execution, both rows are delivered, and the execution runs twice.

Third, execution IDs are generated by the client with nanoid. Nothing prevents a caller from supplying an ID that is already in use.

RFC 003 already depends on such a record. It introduces ExecutionClient.admit and recorded terminal outcomes, but leaves "where the terminal workflow outcome is stored" unspecified.

Proposed solution

Each execution will have one durable record. Admission will create it, store the event once, and make the execution due immediately.

Timers, store wakes, effect waits and new triggers will request a wake for an executionId. A pending wake will be state on the record, not a separate copy of the event. Several wakes for one execution will therefore collapse into one delivery.

// Admission stores the event once and makes the execution due.
const { executionId } = await sdk.trigger({
  workflowId: "process-job",
  params: { jobId: "job-1" },
})

// Inside the workflow, a timer and a store change may both request a wake.
// The execution is delivered once, at the earlier due time.

When a delivery ends with completed or failed, the record will keep that outcome permanently. A later delivery will return it without running workflow code.

Proposed API

This RFC will add the execution record to @yieldstar/core:

type ExecutionStatus =
  | "admitted"
  | "suspended"
  | "completed"
  | "failed"

type ExecutionOutcome =
  | { status: "completed"; result: unknown }
  | { status: "failed"; error: Error }

interface ExecutionRecord {
  executionId: string
  workflowId: string
  params: EventParams
  context: EventContext
  status: ExecutionStatus
  outcome?: ExecutionOutcome
  admittedAt: number
  updatedAt: number
}

• admitted will mean that no delivery has been acknowledged yet.
• suspended will mean that the latest acknowledged delivery suspended the workflow.
• completed and failed will be terminal. outcome will be present only in these states.

Later RFCs will add states. Execution Leases will add a state for an execution that is being run, and a Cancellation RFC will add cancelled. This RFC does not specify them.

Runtime API

This RFC will define ExecutionClient in @yieldstar/core. It replaces the ExecutionClient in RFC 003.

type ExecutionAdmissionErrorCode =
  | "EXECUTION_ID_CONFLICT"
  | "INVALID_EXECUTION_EVENT"

type ExecutionDelivery = {
  executionId: string
  token: string
}

interface ExecutionClient {
  admit(request: {
    executionId?: string
    workflowId: string
    params: EventParams
    context: EventContext
  }): Promise<
    | { status: "accepted"; executionId: string }
    | {
        status: "rejected"
        code: ExecutionAdmissionErrorCode
        message: string
      }
  >

  get(executionId: string): Promise<ExecutionRecord | undefined>

  requestWake(request: {
    executionId: string
    wakeAt: number
  }): Promise<
    | { status: "requested" }
    | { status: "ignored" }
    | {
        status: "rejected"
        code: "EXECUTION_NOT_FOUND"
        message: string
      }
  >

  claimDue(request: { now: number }): Promise<ExecutionDelivery[]>

  acknowledge(request: {
    delivery: ExecutionDelivery
    outcome: WorkflowRunOutcome
  }): Promise<
    | { status: "suspended" }
    | ExecutionOutcome
  >
}

declare class ExecutionAdmissionError extends Error {
  readonly code: ExecutionAdmissionErrorCode
}

WorkflowRunOutcome is defined in RFC 003.

ExecutionDelivery.token will identify the wakes that a delivery covers. Its contents will be opaque to callers.

Required Semantics

Admission

• Execution IDs will be unique across the runtime and will never be reused.
• An unused caller-supplied ID will be reserved and returned unchanged. When no ID is supplied, the runtime will choose one.
• An ID already in use will return EXECUTION_ID_CONFLICT. The SDK will throw ExecutionAdmissionError, and the HTTP trigger route will return 409 Conflict.
• Admission will store workflowId, params and context once. Every delivery will read them from the record.
• Every delivery will read params and context exactly as they were admitted under the runtime's event serialization. context will still be a ReadonlyMap.
• A value that cannot round-trip will return INVALID_EXECUTION_EVENT. Admission will not store a lossy copy.
• Admission will create the record with status admitted and make the execution due immediately. Creating the record and its first wake will be all-or-nothing.
• "Due immediately" will not promise any delivery latency.
• If a delivery fails after admission, the execution ID will remain reserved. IDs will not be recycled according to delivery outcome.
• Direct trigger will resolve once admission commits. It will not wait for a worker to start.

Wakes

• A wake will name an execution and a due time. It will not carry a copy of the event.
• A wake requested for an execution that does not exist will return EXECUTION_NOT_FOUND. It will not create a record.
• Wakes not covered by a claimed delivery will merge into one due time: the earliest of their requested times.
• claimDue will return executions whose due time is at or before now. One execution will appear at most once in one result, however many wakes it has.
• A claimed delivery will cover every wake pending when it was claimed. Wakes requested after the claim will merge separately and will not inherit the claimed due time.
• After claimDue returns an execution, it will not return that execution again until the delivery is acknowledged or a redelivery timeout passes. RFC 002 will replace this timeout.
• Acknowledgement will clear the wakes the delivery covered. The execution's due time will then be the earliest wake requested after the claim, or none.
• If a delivery is not acknowledged, its wakes will stay pending, and the execution will be delivered again.
• A wake requested for a terminal execution will return ignored. Reaching a terminal status will clear any pending wake.
• Pending wakes will survive a runtime restart.
• A store commit will be all-or-nothing with the wake requests for the waiters it satisfies. After a crash, either the commit and every resulting wake are present, or neither is.

Outcomes

• acknowledge will record suspended, completed or failed from the delivery's WorkflowRunOutcome.
• completed and failed will be immutable. A later terminal acknowledgement will not change the record and will return the first outcome.
• A suspended acknowledgement on a terminal record will not change it and will return the terminal outcome.
• After the outcome is recorded, a delivery will return it without running workflow code.
• Every later delivery and read of a failed execution will return the same recorded error. At least its name and message will survive.
• An infrastructure error will not be acknowledged as an outcome. The record will keep its status, and its wakes will stay pending.

Relationship to RFC 003

RFC 003 defers to this RFC for admission, terminal outcomes and delivery. A direct trigger resolves once admission commits, and only the dispatcher delivers an execution.

Relationship to RFC 004

RFC 004's effect waits register an executionId and request a wake for it. They carry no workflow event.

Conformance tests

Runtime-adapter tests will exercise ExecutionClient:

Case	Observable outcome
Admission receives an unused caller-supplied ID	It reserves and returns that exact ID.
Admission receives an ID in use	It returns EXECUTION_ID_CONFLICT; the existing record is unchanged.
Delivery fails after admission	Repeating the admission with that ID returns EXECUTION_ID_CONFLICT.
Admission omits an ID	It returns a new ID that differs from every existing ID.
Admission stores an event	get returns the same workflowId, params and context; context is a ReadonlyMap.
A delivery runs	The worker reads workflowId, params and context from the record, not from the wake.
Admission receives a value that cannot round-trip	It returns INVALID_EXECUTION_EVENT; no record exists and nothing is due.
Admission commits	The execution is admitted and claimDue returns it.
Two wakes are requested before a delivery	claimDue returns the execution once.
Wakes are requested with different due times	The execution becomes due at the earlier time.
A wake is requested during a delivery	After that delivery acknowledges, the execution is still due at the requested time.
A delivery claims a past-due wake, and a future wake is requested during it	After acknowledgement, the execution is due at the future time, not immediately.
A wake is requested for an unknown execution ID	It returns EXECUTION_NOT_FOUND; no record is created and nothing is due.
A delivery is claimed and not yet acknowledged	claimDue does not return it again before the redelivery timeout.
A delivery is never acknowledged	Its wakes stay pending and the execution is delivered again.
A wake is requested for a terminal execution	It returns ignored; nothing becomes due.
A store commit and its wake requests are interrupted by a crash	After restart, either the commit and every wake are present, or neither is.
The runtime restarts with admitted and suspended executions	Their records and pending wakes survive and are delivered.

Outcome tests will cover acknowledgement and terminal records:

Case	Observable outcome
A delivery acknowledges suspended	The record is suspended and has no outcome.
A delivery acknowledges completed	The record is completed; get returns the result.
A delivery acknowledges failed	The record is failed; get returns an error with the same name and message.
A second terminal acknowledgement differs from the first	The record keeps the first outcome, and acknowledge returns it.
A suspended acknowledgement arrives after a terminal one	The record stays terminal, and acknowledge returns the terminal outcome.
A terminal execution is delivered again	The delivery returns the recorded outcome without running workflow code.
A delivery fails with an infrastructure error	The status is unchanged, and the execution is delivered again.

SDK tests will cover triggering:

Case	Observable outcome
Local or HTTP trigger omits an execution ID	The SDK returns the ID accepted by ExecutionClient.
Local or HTTP trigger supplies an ID in use	The SDK throws ExecutionAdmissionError; the HTTP route returns 409 Conflict.
Direct trigger is called	It resolves once admission commits, and the dispatcher delivers the execution.

Scope

This RFC will not specify:

• who is running an execution, leases, fencing or concurrent deliveries (RFC 002);
• a public observation API, such as poll, wait, list or idempotent triggers;
• cancellation;
• retention or pruning of records; or
• awaiting a child workflow's outcome.

Not Specified

This RFC will not choose:

• how records, pending wakes or delivery tokens are stored, or which tables a runtime adapter keeps;
• how acknowledgement tells claimed wakes from later ones, for example with a wake sequence number on the record;
• the redelivery timeout;
• how params, context and result are encoded. A later step-values RFC may require DurableValue;
• how a failed outcome's error is revived beyond its name and message. A later step-values RFC will define class, code and cause;
• whether the runner still writes the $$workflow-result$$ step; or
• how existing task_queue, scheduled_tasks and store_waiters rows are migrated. A migration will be required.