RFC 0008: Runtime import attributes

未關閉
#55 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

評估

難度
5/5
預估耗時
一週以上
新手友好度
25/100
Issue 類型
功能
描述清晰度
基本清楚
活躍度
活躍
技術堆疊
aws, node.js, typescript

研究方向

Start by reading RFC 0012 and the proposed runtime/checkout.ts and infra/orders.ts examples to understand the existing boundary and the proposed import attributes. Then inspect the compiler and type generation entry points, including the notation types command and notation-env.d.ts, and check how runtime modules are loaded and bundled. Done means the RFC’s type views, lambda declaration, evaluation order, runtime bundle, and test loader behavior are implemented and covered by the relevant tests.

由索引模型根據 Issue 內容生成。

描述

enhancement

Abstract

Runtime modules and infra modules import each other, but each needs a different view of the other. Runtime code needs a queue's URL as a string, and infra needs a handler as a reference it can wire, not a function it can call. 0012 bridges the two with macros, which TypeScript can't check: $output and $grant rules are enforced only by the build, a handler is typed as both a function and a reference, and settings can't use infra values.

We will solve this by marking each import that crosses the boundary with a standard import attribute. Runtime code imports infra with { type: "infra" } and infra imports runtime code with { type: "runtime" }. Generated declarations give each side the view it needs, so TypeScript checks both directions, and calling lambda() in a runtime module declares the function it deploys as.

https://github.com/user-attachments/assets/a7c54f85-ae80-4934-a259-7304bbcb1471

Video transcript

A Notation app has two kinds of code. Infra code declares resources, such as an order queue. Runtime code runs inside a resource, such as the checkout Lambda. The checkout module and the orders module import each other. Checkout needs the URL of the queue, so it can send each new order. And the orders module needs the checkout handler, so it can connect the queue to it. But each side needs a different view of the other. Checkout needs the URL as a plain string. The orders module needs the handler as a reference that it can wire to the queue, not as a function that it can call.

Today, RFC 0012 bridges the two sides with macros. A macro is a function whose name starts with a dollar sign. The compiler reads each macro twice: once to build the code of the Lambda, and once to declare the Lambda in the infra phase. Dollar output reads the URL of the queue. Dollar grant gives the Lambda permission to send to the queue. And dollar lambda declares the Lambda itself, with its ID and its timeout.

Problem 1. The types lie. Each handler is typed as a function and a reference at the same time. So the orders module can call the checkout handler, as if it were a function. And runtime code can read the resource field of the handler, as if it were a reference. Both lines type-check. Both lines fail when they run.

Problem 2. The rules of macros live in the build. You cannot rename a macro on import, or store a macro in a variable. And the arguments of a macro must be literals or infra references. But TypeScript sees each macro as an ordinary function call. So the editor never reports a broken rule. You find out only when you build.

Problem 3. The two modules import each other, so they form a cycle. Dollar lambda runs when checkout is first imported. In the cycle, that can happen before infra has declared the queue. So the settings of the Lambda cannot use infra values. And it gets worse. The module that the compiler generates for checkout imports infra directly, so the plan depends on the order in which infra modules load. Reorder two imports, and the same app can produce a different plan. And because macros only exist in the compiler, a test cannot even load checkout without a stand-in for each macro.

The fix is to mark each import that crosses the boundary, with a standard import attribute. An import attribute is JavaScript syntax that says what the imported module is, as type json already does. Checkout will import the orders module with type infra. And the orders module will import checkout with type runtime. This is the key idea of the RFC. The boundary is written on the line where the code crosses it, and TypeScript will check both directions.

First, the runtime view. Inside checkout, the URL of the queue will be typed as a string, and the Lambda will read the deployed value when it runs. Infra functions will be uncallable from runtime code, except resource factories and grant, which the settings of the Lambda use. So no macro is needed to read a value: the code reads the URL like any other property.

Next, the infra view. Inside the orders module, each checkout handler will be a runtime reference. The consume method of the queue accepts a runtime reference, so wiring the handler type-checks. But a runtime reference is not callable, so calling the handler from infra will be a type error, in the editor, on the line you wrote. Other exports of checkout, such as a helper function, will be unusable from infra.

These two views come from a generated declarations file (notation-env.d.ts). The notation types command will write one block for each import specifier that has an attribute, using support for import attributes that arrives in TypeScript 7.1. Each block reads the types of the real module live. So when you add or rename an export, both views update, without a new run. And if an import has no block yet, a catch-all block gives it an error type that tells you to run notation types.

Dollar lambda will go away. Instead, checkout will call lambda: the same factory that infra code already uses. The call will declare the checkout Lambda, with its ID, its timeout, and its grants. So the grant to send to the queue will move into the settings of the Lambda. The call returns handle, and each handler that handle creates and checkout exports will belong to this Lambda. The build will find the Lambda by running the call, not by matching syntax.

But the settings now use the queue, so the cycle still matters. The fix is to run runtime modules after all infra. In the infra phase, four steps will happen. 1: the import of checkout resolves to a stand-in, and none of the code in checkout runs. 2: infra modules run, and consume records the stand-in. 3: checkout runs, after infra has fully loaded, so the settings can use any infra value. 4: each stand-in links to its real handler. This step order matters, because the plan no longer depends on the order in which infra modules load.

In step 3, the build will not run all of checkout. It will run a stripped copy, which keeps only the lambda call, the handlers that come from it, and the code that they reference. Handler bodies never run. SDK clients, file reads and logging are removed. The cost is that an effect the settings do not reference is lost. Say a side-effect import sets a global value, and the settings read it. The copy drops that import, so the setting is undefined, with no error.

The checkout bundle will keep only runtime code. The compiler will rewrite each read of the queue, such as the URL, into a read of a value that the Lambda receives when it deploys. The lambda call becomes the runtime half of the provider, with no settings. And if a handler calls infra, for example to make a grant, the build fails at that call. The bundle is tree-shaken, and only the reads that are left are delivered.

Tests will import checkout with no attribute, so the handlers are plain functions. A test sets the URL of the queue, and calls the handler directly. A loader hook from Notation serves each infra import from those values, so the test never loads infra, and needs no compiler.

This approach has limits. The generated declarations need TypeScript 7.1. TypeScript 6, Prettier and typescript-eslint cannot parse them, so you exclude the file from those tools. If you rename a handler in checkout, the editor does not rename it in the orders module, which then fails to type-check. A rename that starts in infra updates every file. Some rules are still build errors, such as a call to infra inside a handler. And plain Node cannot load a cycle of TypeScript imports with attributes, so tests need the loader hook from Notation.

Before, macros bridged runtime and infra. The types let infra call a handler, the editor missed broken rules, the plan depended on load order, and tests needed a compiler. With this RFC, each import that crosses the boundary will say which side it comes from. TypeScript will give each side the view it needs, the lambda call will declare the function, and the plan will be the same whatever the load order.

Problem

0012 gives runtime modules one sanctioned way to reach infra: $-prefixed macros that the compiler reads twice. That works, but:

  • The types lie. handle.apiRequest returns Handler & RuntimeRef. Infra can call a handler, and runtime code can read .resource. Both type-check and fail when they run.
  • Rules live in the build. Macros can't be aliased or stored, and their arguments must be literals or infra references. TypeScript sees ordinary calls, so the editor never reports a broken rule.
  • Settings can't use infra values. $lambda({ ... }) is evaluated when the runtime module is first imported, which in the infra–runtime cycle is before infra has declared anything.
  • The generated module depends on evaluation order. 0012's generated module imports infra statically, so the plan changes with the order infra modules load. A spike had to defer it with a lazy import().
  • Tests need a compiler. A module that calls a macro can't run without a stand-in for each macro.
  • It is a new concept. Readers have to learn which calls are macros and what each one becomes.

Proposed solution

  1. with { type: "infra" } marks an import in a runtime module as infra. Reading it inside runtime code reads the deployed value: orderQueue.url is the queue's URL, typed string.
  2. with { type: "runtime" } marks an import of a runtime module from infra. Infra sees each handler export as a RuntimeRef<Event, Result, Function>, which it can wire but not call.
  3. lambda() declares the function. The provider's ordinary factory, called in a runtime module, declares the function as an effect and returns { handle }. Handlers are the exports that handle creates. There is no resource export, and the build never matches syntax to find functions.
  4. Generated declarations type both views. notation types writes notation-env.d.ts, using TypeScript 7.1's support for import attributes in ambient module declarations.
  5. Runtime modules run last in the infra phase. Infra first sees each runtime import as a stand-in. The runtime modules then run, stripped of code the infra phase doesn't need, and the stand-ins link to their handlers.
  6. The runtime bundle keeps only runtime code. The compiler rewrites each use of an infra import. Any infra module left in the finished bundle is an error, as in 0012.
// runtime/checkout.ts
import { json } from "@notation/aws/lambda.fn";
import { lambda } from "@notation/aws/lambda" with { type: "infra" };
import { orderQueue } from "../infra/orders" with { type: "infra" };
import { sendMessage } from "./sqs-client";

const { handle } = lambda({ id: "checkout", timeout: 5, grants: [orderQueue.grant(["SendMessage"])] });

export const createOrder = handle.apiRequest(async (req) => {
  await sendMessage(orderQueue.url, req.body); // orderQueue.url: string
  return json({ ok: true });
});

export const processOrder = handle.queueMessage(async (event) => {
  for (const record of event.Records) console.log("processing", record.body);
});
// infra/orders.ts
import { queue } from "@notation/aws/sqs";
import * as checkout from "../runtime/checkout" with { type: "runtime" };

export const orderQueue = queue({ id: "orders" });
orderQueue.consume(checkout.processOrder); // RuntimeRef<QueueEvent, void, LambdaFunction>
// runtime/checkout.test.ts: no attribute, so handlers are plain functions
import { setOutputs } from "@notation/aws/testing";
import { createOrder } from "./checkout";

setOutputs({ "infra/orders#orderQueue.url": "https://sqs.test/orders" });
const res = await createOrder({ routeKey: "POST /orders", apiId: "api", body: "{}" });
Attributes

The attribute names what the imported module is, as type: "json" does. One key with two values keeps both sides of the boundary reading the same way.

  • type: "infra" is only valid in runtime modules, and its target must be an infra module or a provider's infra half.
  • type: "runtime" is only valid in infra modules, and its target must be a runtime module.
  • A missing attribute is a build error when a runtime module bundles infra, or infra imports a runtime module. TypeScript also catches it when the import is passed to route or consume, since a plain function isn't a RuntimeRef.
Types

notation types writes one block per attributed import specifier, plus catch-alls:

// notation-env.d.ts (generated)
declare module "../infra/orders*" with { type: "infra" } {
  const view: RuntimeView<typeof import("#root/infra/orders")>;
  export = view;
}
declare module "../runtime/checkout*" with { type: "runtime" } {
  const view: InfraView<typeof import("#root/runtime/checkout")>;
  export = view;
}
// an attributed import with no block above
declare module "../*" with { type: "runtime" } {
  const view: { readonly [name: string]: { readonly "no generated declaration for this { type: \"runtime\" } import": "run notation types" } };
  export = view;
}
  • RuntimeView<T> turns each OutputValue<U> into U, and each function into an uncallable InfraOnly. Resource factories and grant stay callable, since lambda() settings use them. A whole binding read as a value drops its methods, since they aren't delivered.
  • InfraView<M> turns each Handler<E, R, F> export into RuntimeRef<E, R, F>, an exported lambda() result into its function F, and every other export into an unusable RuntimeOnly<Name>.
  • Handler<E, R, F> is a callable type with a type-only mark, so InfraView can tell handlers from other functions. F is the provider's function type, so checkout.createOrder.function.logGroupName is typed.

Each block reads its module's types live, so adding, removing or renaming an export updates both views with no regeneration. The file changes only when an attributed import specifier appears, disappears or moves to another module.

Patterns. TypeScript picks the matching pattern with the longest prefix, so each block is the import specifier plus *. Suffix patterns such as */runtime/checkout can't work: "./runtime/checkout" and "../runtime/checkout" match the same suffix whatever its length. A pattern without * is ignored for relative specifiers. The same specifier naming two different modules from two directories can't be told apart by any pattern, so notation types fails and asks for a different specifier.

Catch-alls. ./* and ../* blocks, per attribute, give an import that has no block yet an error type that says to run notation types. Without them, the import silently falls back to the other side's real types, and errors appear later and indirectly, if at all.

notation types runs in notation build, before tsc, and in notation dev, which regenerates when a file is created, deleted or moved, or a file's attributed import specifiers change. It only rewrites the file when its content changes.

lambda()

lambda is the provider's existing factory, imported from its infra half. Its settings are the factory's config type, validated by its value types, plus grants.

  • In the infra phase, the call declares the Lambda, named by id, with its bundle at dist/functions/<id>/. Each handle.<kind>(fn) returns a handler reference carrying the function and the kind. It is named after its export when the module is linked, and a handler that isn't exported fails at plan, naming the handle call.
  • In the runtime bundle, the compiler replaces the call with the provider's runtime half, without its arguments. Each handle.<kind>(fn) then returns fn.
  • Several lambda() calls in one module, or a call in a helper module or behind a condition, each declare a function. Each function's bundle dispatches only to its own handlers.
  • Exporting the result gives infra the function itself: export const checkoutFn = lambda({...}) is seen by infra as LambdaFunction. Runtime code can't read its outputs, and infra can't use its handle. Infra can also reach the function through any handler, e.g. checkout.createOrder.function.
The infra phase

Infra and runtime modules import each other: infra/orders.ts imports checkout to wire it, and checkout's lambda() settings use orderQueue. To make the plan independent of load order, runtime modules run after all infra:

  1. A type: "runtime" import resolves to a stand-in. It is a generated module with one stand-in reference per export of the runtime module, following export *. It runs none of the module's code.
  2. Infra modules evaluate. Infra APIs that take a handler, such as route and consume, record the stand-in and apply their wiring once evaluation finishes, through 0004's end-of-evaluation hook.
  3. Runtime modules evaluate, as stripped copies (below). Infra has fully loaded, so lambda() settings are ordinary code: they can use infra values, local constants, helpers, and an id built from infra.
  4. Stand-ins link. Each stand-in resolves to the handler, or exported function, it names. Wiring with the wrong kind fails here, at the infra call site.

A stand-in's function is a lazy proxy that resolves after step 4, whatever the provider's function type. Passing checkout.createOrder.function.arn or grantInvoke() into another resource works, since outputs resolve later. Reading the function's identity while infra is evaluating, such as .function.id or turning an output into a string, fails with the call site: createOrder.function isn't known yet: infra used it as a value … while infra was being evaluated (infra/api.ts:5:70).

Deferring only the settings that use infra, as an earlier spike did, also gives a stable plan. But every provider factory must then accept deferred values, and infra values can't be used anywhere else, so derived constants, helpers and an id from infra all fail.

Stripped copies

The infra phase evaluates a copy of each runtime module that keeps only what declaring its functions needs. Per top-level statement, the build keeps:

  1. Calls to type: "infra" imports made while the module body runs, such as lambda(...).
  2. Declarations derived from those calls, and exports of them. A binding is derived if it comes from such a call, so handle and export const createOrder = handle.apiRequest(...) are kept. This carries across files, so import { handle } from "./fn" counts.
  3. Everything a kept statement references: declarations, imports, and what kept function bodies reference.
  4. Top-level statements that update a kept binding, such as memory *= 2 or Object.assign(settings, ...).

Calls on a derived binding have their arguments replaced with an inert placeholder, so handler bodies never run. Local modules the copy imports are stripped the same way, rooted at the names it imports, and their dropped exports become undefined. Everything else is removed: top-level await, I/O, SDK clients and logging run only in the function.

Settings must only depend on what they reference. Effects through statements the rule removes are lost without an error:

  • A side-effect import, e.g. import "./env-setup" setting a global the settings read, is removed, so the setting is undefined.
  • A mutation through a call, e.g. bump() changing a counter the settings read, is removed.

A class the settings use is kept whole, so its static blocks run at plan. Settings that do I/O, such as fetching a value, do it at plan, which is intended.

Content mappers can't do this: they only change what the type checker sees, and they can't claim .ts files.

The runtime bundle

The compiler rewrites each use of an infra import in runtime code:

  • A property read, e.g. orderQueue.url, becomes read("infra/orders#orderQueue", ["url"]). The function receives the value through 0007's Runtime map, under the key from 0012's output keys.
  • A whole binding used as a value, e.g. console.log(orderQueue), becomes read("infra/orders#orderQueue", []), and delivers all of the binding's data properties as one object: url, arn and id for a queue. Reads of its sub-paths are then not delivered twice. A whole namespace import used as a value is a build error.
  • A call to a provider factory, such as lambda(...), becomes the provider's runtime half with no arguments.
  • Top-level declarations used only by lambda() settings are removed.
  • Calling infra inside a handler, e.g. orderQueue.grant(), is a build error at the call.

Reads are marked pure, and only the reads left in the finished, tree-shaken bundle are delivered. Otherwise dead code would deliver extra values, or fail plan on values the function never reads.

Build order
  1. Evaluate the infra phase, with stand-ins and stripped copies. This declares the functions, and links each handler to its function.
  2. Bundle each function, with its handlers as the dispatch entry.
  3. Collect each bundle's reads and finish the plan with their values. A read whose value isn't an output fails at plan.

Bundling follows evaluation, because which handlers belong to which function is only known once lambda() has run.

Testing

Tests import runtime modules without an attribute, so handlers are plain functions. A Notation loader hook serves type: "infra" imports from values set with setOutputs, so tests never load infra. The hook also drops attributes before Node resolves, which avoids the Node bug below.

Rationale

Import attributes are standard JavaScript. The boundary is written where it is crossed, both runtimes and bundlers already parse it, and TypeScript 7.1 can type it. Both views come from the real modules, so every mistake that changes a type is an editor error at the line written, and the build only checks what types can't express.

Declaring the function with lambda() reuses the factory infra already calls, and reads like infra code. The function and its handlers are found by evaluating that call, not by matching syntax. Running runtime modules after infra removes the import cycle rather than deferring around it, so settings are ordinary code.

Consequences

  • No macros. $output, $grant and $lambda are removed, with the macro marker and its discovery.
  • Handler mistakes in infra, such as calling a handler or wiring it to the wrong source, are type errors.
  • Runtime and test code see plain functions and values.
  • The plan doesn't depend on the order infra modules load.
  • The resource export from 0007 is replaced by the lambda() call. A module that exports resource fails with a migration message.
  • Grants move into lambda() settings, away from the code that sends.
  • Infra APIs that take a handler apply their wiring at the end of evaluation.

Limits

  • Generated declarations are required. TypeScript 6, Prettier and typescript-eslint can't parse them, so they must be excluded from those tools.
  • Editor rename misses infra. Renaming createOrder in its own module updates that module and its tests, but not checkout.createOrder in infra, which then fails with TS2339 Property 'createOrder' does not exist. Starting the rename from infra updates every file. Other declaration shapes were tried, and none let rename cross; per-export blocks made it worse, renaming the declaration instead and leaving infra checking against an export that no longer exists.
  • Settings lose effects they don't reference, as described in stripped copies.
  • Infra can't read a function's identity while it evaluates, only pass it on.
  • Some rules are build or plan errors, not type errors: calling infra inside a handler, storing an infra method such as const g = orderQueue.grant, and a handler that isn't exported.
  • Plain Node can't load attributed .ts import cycles. Node skips checking attributes for TypeScript files, then recurses forever: any cycle, even a file importing itself with { type: "x" }, overflows the stack on Node 24 and 26. Tests and scripts that load runtime modules need Notation's loader hook.

Out of scope

  • Inferring grants from the SDK calls a handler makes, using the type checker.
  • Other providers' factories, such as worker() on Cloudflare.
  • Secrets, as in 0012.

Open questions

  • Should stripped copies warn when a removed statement has effects, such as a side-effect import in a runtime module?
  • Should infra's handler APIs other than route and consume, such as schedule, follow the same end-of-evaluation rule, and is that a contract of @notation/resource?
主要語言
TypeScript
星號
54
分支
1
平均合併
15 小時 22 分鐘
30 天內合併 PR
6

環境準備

  • 沒有 Dockerfile 或 Docker Compose 檔案
  • 沒有 Pull Request 範本
  • 閱讀貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

notationlabs/notation 的其他 Issue

查看 notationlabs/notation 的全部 Issue

相似的 Issue

更多 TypeScript Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。