Effect.callback keeps state from finished suspensions alive until the next full GC
Maintainers usually reply within 1 day
Assessment
This issue has not been assessed yet.
Description
What steps can reproduce the bug?
Each time a fiber suspends in Effect.callback, callbackOptions runs:
fiber._yielded = () => {
resumed = true
}
V8 pretenures a function literal assigned directly to a property, so this closure is allocated in the old generation. TurboFan ignores that flag, so this holds until TurboFan optimizes this code, and again after a deopt. Once the fiber resumes, the closure is dead but old, and every scavenge treats it as a root until the next full GC. Its context keeps onCancel alive. For Deferred.await, and for PubSub.take and PubSub.takeAll, which both wait in pollForItem, that reaches the completed Deferred and the value it delivered. Every SubscriptionRef.changes subscriber waits this way.
The script below sends a fresh value of about 75 KB every 50 ms for 20 s to a fiber that is waiting for it, and runs one scavenge every 4 values. It does this twice, once through a PubSub without replay and once through a new Deferred for each value. I ran it on Node 26.
npm install effect
node --expose-gc --min-semi-space-size=64 --max-semi-space-size=64 waitingFiber.mjs
The two semi-space flags make the young generation big enough that allocation does not trigger scavenges, so only the script's gc({ type: "minor" }) calls run them.
waitingFiber.mjs
// node --expose-gc --min-semi-space-size=64 --max-semi-space-size=64 waitingFiber.mjs
import v8 from "node:v8"
import { constants, performance, PerformanceObserver } from "node:perf_hooks"
import { Deferred, Effect, Exit, PubSub } from "effect"
const makeValue = () => Array.from({ length: 2000 }, (_, index) => ({ index })) // about 75 KB
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms))
const oldGenerationBytes = () =>
v8.getHeapSpaceStatistics()
.filter((space) => !space.space_name.startsWith("new_"))
.reduce((sum, space) => sum + space.space_used_size, 0)
const fullGcStarts = []
new PerformanceObserver((list) => {
for (const entry of list.getEntries()) {
if (entry.detail.kind === constants.NODE_PERFORMANCE_GC_MAJOR) fullGcStarts.push(entry.startTime)
}
}).observe({ entryTypes: ["gc"] })
// Sends a fresh value every 50 ms and runs one scavenge every 4 values, so a value that is
// dropped after its turn lives through at most one scavenge and should never be promoted.
const measure = async (name, send) => {
for (let index = 0; index < 20; index++) {
send(makeValue())
await sleep(50)
}
globalThis.gc()
const start = performance.now()
let promotedBytes = 0
for (let index = 1; index <= 400; index++) {
send(makeValue())
await sleep(50)
if (index % 4 === 0) {
const before = oldGenerationBytes()
globalThis.gc({ type: "minor", execution: "sync" })
promotedBytes += oldGenerationBytes() - before
}
}
await sleep(0)
const fullGcs = fullGcStarts.filter((time) => time > start).length
console.log(`${name}: ${(promotedBytes / 400 / 1024).toFixed(1)} KB promoted per value, ${fullGcs} full GCs in 20 s`)
}
// A fiber waits in PubSub.take for each value (a PubSub without replay).
const pubsub = Effect.runSync(PubSub.unbounded())
Effect.runFork(Effect.scoped(Effect.gen(function*() {
const subscription = yield* PubSub.subscribe(pubsub)
while (true) yield* PubSub.take(subscription)
})))
await measure("PubSub.take", (value) => Effect.runSync(PubSub.publish(pubsub, value)))
// A fiber awaits each value through a fresh Deferred.
let deferred = Deferred.makeUnsafe()
Effect.runFork(Effect.forever(Effect.suspend(() => Deferred.await(deferred))))
await measure("Deferred.await", (value) => {
const current = deferred
deferred = Deferred.makeUnsafe()
Deferred.doneUnsafe(current, Exit.succeed(value))
})
process.exit(0)
What is the expected behavior?
A value the fiber has received and dropped dies in the young generation, like any other short-lived value.
What do you see instead?
PubSub.take: 74.3 KB promoted per value, 3 full GCs in 20 s
Deferred.await: 74.8 KB promoted per value, 3 full GCs in 20 s
Every value is promoted. The old generation grows by the size of each value, and V8 runs full GCs to free it.
With --allow-natives-syntax, %InYoungGeneration(fiber._yielded) returns false right after a suspension. It returns true once TurboFan has optimized the Async primitive's evaluate, which took about 1,000 suspensions in a quick check. Code that suspends rarely, like the script, stays affected, and so does code that deopts.
Additional information
yieldNowWith already stores the same kind of closure through fiber.yieldWith(...). V8 does not pretenure a literal passed as an argument, so doing the same in callbackOptions fixes this:
-fiber._yielded = () => {
+fiber.yieldWith(() => {
resumed = true
-}
+})
With that change the script prints 0.3 and 0.2 KB promoted per value and no full GCs, and %InYoungGeneration returns true. Assigning the closure to a local const first also works before minification, but terser and Vite's minifier inline the const back into the property assignment. Neither can do that with the method call. I have the fix with a test and a changeset and will open a PR.
Other code also assigns a new closure to a property once per operation:
entry.onInterruptinCacheentry.completeUnsafeinRequestResolver- the cluster's
ShardingandentityManager
I have not measured those. Effect.callback runs on every suspension, so I kept the fix to it.
#9496 is a separate bug with the same effect on SubscriptionRef.
I investigated this with an AI coding agent, Claude Code. The agent read the Effect and V8 sources, wrote the repro, and ran it. I reviewed the analysis before filing.
- Dominant language
- TypeScript
- Stars
- 16.7k
- Forks
- 808
- Avg merge
- 10h 22m
- Merged PRs (30d)
- 771
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Effect-TS/effect
-
BrowserWorkerRunner: port finalizer throws when the worker global has no close() (Bun)Possibly taken @santiago-ramos-02 claimed this 18 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Effect-TS/effect#8635 · 3 comments ·
Maintainers usually reply within 1 day
-
Support {self: this} for fnUntracedMay be free again @ArjunCodess claimed this 30 days ago, and no pull request is open. Openenhancement
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Effect-TS/effect#8101 · 1 comment ·
Maintainers usually reply within 1 day
-
HashMap.set keeps the old value when the new one is Equal to itPossibly taken @schutzelaars claimed this today. Open
Difficulty 3/5 1-2 days Newbie friendliness 58/100
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
OpenApi: optional query and header parameters are documented as nullablePossibly taken @Marve10s claimed this 1 day ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 57/100
Maintainers usually reply within 1 day
All issues in Effect-TS/effect
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
ccch1mneyyy/dsh-TUI#1631 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
MiniMax-AI/minimax-code#507 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Transient OAuth login failure persists in getLoginStatus and masks active account in dashboardMay be free again A pull request for this issue was closed without being merged. Openaccount-pool bug gui
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
lidge-jun/opencodex#7378 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day