RFC 0011: Cycle resolution

Read the linked RFCs 0004, 0009, and 0010 alongside this proposal, then trace the reconciler's dependency planning, apply, update, destroy, and replacement paths. Start with cycle planning and placeholder parameter handling; the issue names no implementation files or tests. Done means deployable cycles can be created, settled, destroyed, and replaced as specified, with unsupported cycles and non-settling outputs reported clearly.

Abstract

Notation can't deploy resources that depend on each other, for example two Cloudflare Workers that call each other through service bindings.

We will solve this by first creating one of these resources as a placeholder: its config without the contributions that refer to its peers. Once its peers exist, it's updated with its full configuration. Two Workers that bind to each other each contribute a binding to the other, and the engine works out the order.

https://github.com/user-attachments/assets/ef03c43f-5baa-4c31-a145-6a254d9f1304

Video transcript

When you run notation deploy, the reconciler deploys resources in the order of their dependencies. Take a Cloudflare app with a Worker and a database. The Worker binds to the database, so the Worker depends on the database. The reconciler creates the database first. Then it uploads the Worker, with a binding to the database, which now exists.

Now say the app has two Workers. The api Worker handles requests, and calls the auth Worker to check each user. The auth Worker calls back to the api Worker to load the user's account. Each call goes through a service binding. A service binding is part of the Worker's script upload, and Cloudflare rejects an upload that binds to a Worker that does not exist yet. So the api Worker cannot be uploaded before the auth Worker, and the auth Worker cannot be uploaded before the api Worker.

Problem. The two Workers depend on each other, so their dependencies form a cycle, and the reconciler has no first resource to deploy. Today, when the reconciler finds a cycle, it fails, and nothing is deployed. This is important, because Workers that call each other are the normal shape of a Cloudflare app. A Worker and a Container form the same kind of cycle. The Container needs the ID of a Durable Object namespace, which exists only after the Worker uploads, and the Worker's upload must name the Container. Today, Notation cannot deploy either app.

RFC 4, on contributions, already removes many cycles. For example, an S3 bucket notifies a Lambda, and the Lambda reads from the bucket. The notification config moves into its own resource, which depends on both the bucket and the Lambda, so the cycle is gone. But a service binding cannot move out like that. The binding is part of the Worker itself, so there is no separate resource to move it into. The cycle stays.

The fix starts with how you write the cycle. Each Worker will be constructed with only its own config: its script name. Then each Worker will contribute a binding to the other. A contribution, from RFC 4, adds a value to another resource's config, after both resources are constructed. This matters, because a contribution will be the only thing that can close a cycle. A resource that you pass in config must already be constructed, so two constructors can never refer to each other.

So the engine can split each Worker into two parts: its config, and its contributions. A placeholder will be a Worker's config without its contributions. Say the engine picks the api Worker as the placeholder. Step 1: it creates the api Worker, with no bindings. Step 2: it creates the auth Worker, which binds to the api Worker, which now exists. Step 3: it updates the api Worker with its binding to the auth Worker. The engine creates the placeholder with the normal create operation, and finishes it with the normal update operation. So the Worker type will need no new operation.

But not every resource can start as a placeholder. The api Worker can start with no bindings only because the Worker schema marks bindings as optional. So the planner will allow a cycle only if every loop has a member whose contributed fields are all optional. If no member can start without its contributions, the plan fails as it does today, and names the resources in the loop.

Next, the cycle must settle. In step 2, the auth Worker saw the api Worker as a placeholder. If step 3 changed an output of the api Worker, the auth Worker's params are now out of date. So once every member has been applied, the engine will update again any member whose params changed. A cycle of two Workers gets three passes: each pass carries a change one hop, and the last pass checks that nothing changed. Now suppose each Worker binds to the other Worker's version ID. Every update makes a new version, which changes the other Worker, whose update makes another new version. This cycle never settles. So after three passes, the deploy fails, and names the version ID as the output that changes on every update.

Destroying the app has the same problem in reverse. Cloudflare will not delete a Worker while another Worker binds to it. So the engine will first revert each Worker to its placeholder, with an update that drops its bindings. Then the engine deletes both Workers. This also works when you delete both Workers from your code, because state records the edges between them.

Replacing a Worker is harder. Say the api Worker must be replaced. The reconciler usually deletes the old Worker first, but the auth Worker still binds to it, so Cloudflare refuses the delete. The auth Worker must stop referring to the old api Worker first. If the api Worker has a generated name, the old and new Workers can exist side by side. The engine creates the new api Worker with its full config, moves the auth Worker onto it, and then deletes the old one. The app stays up throughout. If the api Worker has a fixed name, the new Worker cannot exist beside the old one. So the engine reverts the auth Worker to its placeholder, deletes the old api Worker, creates the new one, and lets the cycle settle. Until then, the api Worker is down, so the plan warns you, and suggests a generated name behind a stable route.

Before, two Workers that called each other could not be deployed at all. Now, each Worker will contribute its binding to the other. The engine will create one Worker as a placeholder, and update it once its peer exists. One limit remains: a placeholder is live. Until the cycle settles, a call through a missing binding fails.

Problem

The reconciler orders resources by their dependencies. When the graph has a cycle, it fails and nothing is deployed.

0004 handles most cycles by moving the shared configuration into its own resource. For example, a bucket's notifications become a separate resource that depends on both the bucket and the Lambda. That works when the configuration that closes the loop can be written separately from either resource.

Some cycles can't be broken that way, because the reference that closes the loop is part of the resource itself:

• Worker ↔ Worker. Service bindings are part of a Worker's script upload. Cloudflare rejects an upload that binds to a script that doesn't exist yet, so A can't be uploaded before B, and B can't be uploaded before A.
• Worker ↔ Container. A Container application needs the ID of its Durable Object namespace, which only exists once the Worker that exports the class has uploaded. The Worker's upload must declare the class as backed by a container.

These cycles are the normal shape of a Cloudflare app, not edge cases. Without a way to deploy them, Notation can't support Workers that call each other.

Proposed solution

1. Cycles are closed by contributions. Only a contribution or dependOn from 0004 can close a cycle, never a constructor dependency. A resource passed in config must already be constructed, so two resources can never refer to each other that way.
2. A placeholder is config without contributions. The engine creates the placeholder with create(placeholderParams), through 0009 and the ownership rules of 0010, unchanged. No new operation is needed.
3. Cycles are allowed when they can be broken. The planner allows a cycle if every loop in it has a member whose contributed fields are all optional in its schema. Otherwise it fails as today, naming the resources in the loop.
4. A placeholder already exists. Once its peers exist, it's applied with update, like any other existing resource.
5. Cycles settle. Once every member has been applied, any member whose params changed because a peer's output changed is updated again. A cycle of N members gets N + 1 passes: each pass carries a change one hop, and one more confirms nothing changed. If it's still changing after that, the deploy fails, naming the output that changes on every update, e.g. a version ID.
6. Cycles are destroyed through placeholders. A provider may refuse to delete a resource another still refers to, so each member is first reverted to its placeholder with update(key, placeholderParams), dropping its references to its peers, and then deleted. This applies to a cycle removed from code as well, which is found from the edges recorded in state.
7. Replacing a member re-deploys the cycle around it. A replacement normally deletes the old instance first, but a peer that still refers to it may stop the delete, as Cloudflare does for a script another script binds to. Peers must therefore stop referring to the old instance before it is deleted. How depends on whether the old and new instances can exist side by side:

   • They can, e.g. with a generated name from 0010. The new instance is created with its full config, since its peers already exist. Peers move onto it, and the old one is deleted once nothing refers to it. The cycle stays up throughout, and no placeholder is needed.

     create A′ → update B → delete A
     

   • They can't, e.g. with a fixed name. Peers are reverted to their placeholders, which drops their references, then the old instance is deleted, the new one created, and the cycle settles.

     revert B → delete A → create A′ → settle
     

     Plan warns that the member will be unavailable while it's replaced, and suggests a generated name plus a stable route, alias or CNAME.

const workerSchema = worker.defineSchema({
  ScriptName: { propertyType: "param", presence: "required", value: v.string() },
  bindings: {
    propertyType: "param",
    presence: "optional",
    value: v.array(bindingSchema),
    collect: (items) => items.flat(),
  },
});

// infra/app.ts
const a = new Worker({ config: { ScriptName: "a" } });
const b = new Worker({ config: { ScriptName: "b" } });
a.contribute({ bindings: [{ type: "service", service: b.ScriptName }] });
b.contribute({ bindings: [{ type: "service", service: a.ScriptName }] });

a and b are constructed without referring to each other, and each binding arrives as a contribution. The engine picks a as the placeholder:

create a (no bindings) → create b (binds a) → update a (binds b) → settle

The Worker type keeps its usual create, update and delete.

• A placeholder has no contributions. Its peers don't exist yet, so nothing they would add is there.
• Peers see placeholder outputs, nobody else does. Resources outside the cycle wait for the final output, as today.
• A placeholder is state. If the deploy fails after the placeholder is created, the next deploy updates it, and destroy deletes it.

Plan

Plan shows a placeholder member as a create. Params that depend on a peer that doesn't exist yet are unknown until apply, and dry runs use plan's unknown tokens from 0004 for them. A fixed-name replacement in a cycle carries the unavailability warning, naming the peers it reverts.

Rationale

The engine already tracks contributions separately from config, and a cycle can only be closed by a contribution, so the placeholder needs no new operation. create builds it and update brings it to its desired state, as it does for any existing resource, including retries after a failed deploy. Whether a resource can break a cycle is read from its schema.

Related work

• Terraform, CloudFormation and Pulumi reject cycles. They break cycles by modelling the shared configuration as a separate resource, such as aws_s3_bucket_notification. Notation already does this through 0004.
• Alchemy has a similar mechanism: an optional precreate operation per resource type, rendezvous within a cycle, and a converge loop afterwards. This RFC adds no operation. The placeholder is the resource's config without contributions, created with create and reverted with update.

Limits

• A placeholder is briefly live. A placeholder Worker or Lambda can receive traffic before its peers' bindings are added, so calls through them fail until the cycle settles.
• Replacing a cycle member with a fixed name makes it unavailable until the cycle settles. Generated names avoid this.
• Settling writes a resource more than once in a deploy. Each write has its own key (0009), e.g. state:persist:<id>:settle:<n>.

Out of scope

• Cloudflare resources. This RFC only adds the mechanism. Which params are collected, such as a Worker's bindings, is decided when each provider is added.