Replay-enabled PubSub keeps every published value alive until the next full GC
Los mantenedores suelen responder en 1 día
Evaluación
Este issue todavía no se ha evaluado.
Descripción
What steps can reproduce the bug?
Every SubscriptionRef is a PubSub.unbounded({ replay: 1 }). When the replay buffer is full, ReplayBuffer.offer drops its head with this.head = this.head.next!, but the dropped node keeps its value and its next.
If a full GC promoted that node while it was the head, it is now dead but in the old generation, and every scavenge treats its pointers into the young generation as roots until the next full GC. Through next it keeps every later node and its value alive. Those get promoted, and the last promoted node does the same to the nodes after it. So until the next full GC, every published value ends up in the old generation.
The script below sets a fresh value of about 75 KB every 50 ms for 20 s and runs one scavenge every 4 updates, so a replaced value lives through at most one scavenge. It compares a Ref with a SubscriptionRef that has no subscribers. I ran it on Node 26.
npm install effect
node --expose-gc --min-semi-space-size=64 --max-semi-space-size=64 subscriptionRef.mjs
The two semi-space flags make the young generation big enough that allocation does not trigger scavenges, so only the script's gc({ type: "minor" }) calls run them.
subscriptionRef.mjs
// node --expose-gc --min-semi-space-size=64 --max-semi-space-size=64 subscriptionRef.mjs
import v8 from "node:v8"
import { constants, performance, PerformanceObserver } from "node:perf_hooks"
import { Effect, Ref, SubscriptionRef } from "effect"
const makeValue = () => Array.from({ length: 2000 }, (_, index) => ({ index })) // about 75 KB
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms))
const oldGenerationBytes = () =>
v8.getHeapSpaceStatistics()
.filter((space) => !space.space_name.startsWith("new_"))
.reduce((sum, space) => sum + space.space_used_size, 0)
const fullGcStarts = []
new PerformanceObserver((list) => {
for (const entry of list.getEntries()) {
if (entry.detail.kind === constants.NODE_PERFORMANCE_GC_MAJOR) fullGcStarts.push(entry.startTime)
}
}).observe({ entryTypes: ["gc"] })
// Sets a fresh value every 50 ms and runs one scavenge every 4 updates, so a value that is
// replaced after one update lives through at most one scavenge and should never be promoted.
const measure = async (name, set) => {
for (let index = 0; index < 20; index++) {
set(makeValue())
await sleep(50)
}
globalThis.gc()
const start = performance.now()
let promotedBytes = 0
for (let index = 1; index <= 400; index++) {
set(makeValue())
await sleep(50)
if (index % 4 === 0) {
const before = oldGenerationBytes()
globalThis.gc({ type: "minor", execution: "sync" })
promotedBytes += oldGenerationBytes() - before
}
}
await sleep(0)
const fullGcs = fullGcStarts.filter((time) => time > start).length
console.log(`${name}: ${(promotedBytes / 400 / 1024).toFixed(1)} KB promoted per update, ${fullGcs} full GCs in 20 s`)
}
const ref = Effect.runSync(Ref.make(makeValue()))
await measure("Ref", (value) => Effect.runSync(Ref.set(ref, value)))
const subscriptionRef = Effect.runSync(SubscriptionRef.make(makeValue()))
await measure("SubscriptionRef", (value) => Effect.runSync(SubscriptionRef.set(subscriptionRef, value)))
What is the expected behavior?
A SubscriptionRef promotes no more than a Ref. A value that has been replaced dies in the young generation.
What do you see instead?
Ref: 0.0 KB promoted per update, 0 full GCs in 20 s
SubscriptionRef: 74.5 KB promoted per update, 3 full GCs in 20 s
Every value is promoted. The old generation grows by the size of each value, and V8 runs full GCs that a Ref holding the same values never needs. An app that keeps large state in a SubscriptionRef and replaces it often pays for this with full-GC pauses.
Additional information
The other buffers in PubSub.ts already clear what they drop:
- the bounded implementations reset a slot to
AbsentValue UnboundedPubSubclears the value of each node once every subscriber has taken itReplayWindowImplclears what it hands out
ReplayBuffer is the only one that does not. Nothing reads a node after offer drops it. Since #6805, which fixed #6804, a different leak in this code, ReplayWindowImpl copies the values it needs. So offer can clear the node it drops:
if (this.size === this.capacity) {
- this.head = this.head.next!
+ const dropped = this.head
+ this.head = dropped.next!
+ dropped.next = null
+ dropped.value = AbsentValue
}
With that change the script prints SubscriptionRef: 0.0 KB promoted per update, 0 full GCs in 20 s. I have the fix with a test and a changeset and will open a PR.
A SubscriptionRef.changes subscriber that is waiting for the next value keeps values alive in a second, separate way, reported in #9497.
I investigated this with an AI coding agent, Claude Code. The agent read the Effect and V8 sources, wrote the repro, and ran it. I reviewed the analysis before filing.
- Lenguaje dominante
- TypeScript
- Estrellas
- 16.7k
- Forks
- 808
- Merge medio
- 10 h 52 min
- PR fusionados (30 d)
- 836
Preparar el entorno
Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de Effect-TS/effect
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
Los mantenedores suelen responder en 1 día
-
RpcServer: with disableFatalDefects, a handler that throws before returning its Effect still fails every pending callPosiblemente ocupada @effect-bot la tomó hoy. Abierto
Effect-TS/effect#9892 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Duration: ISO 8601 format/parse and a Schema codec (exact units only)Posiblemente ocupada @jayhemnani9910 la tomó hoy. Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 25/100
Effect-TS/effect#9888 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
HttpApiClient: `Redacted` query and header values appear in plain text in client spansPosiblemente ocupada @effect-bot la tomó hoy. Abierto
Dificultad 3/5 1-2 días Aptitud para principiantes 30/100
Los mantenedores suelen responder en 1 día
-
Effect.fn stack frames keep the call's arguments alive in forked fibersPosiblemente ocupada @Hona la tomó hace 1 día. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 25/100
Los mantenedores suelen responder en 1 día
Todos los issues de Effect-TS/effect
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
awesome-dsh-plugin/awesome-dsh-plugin#9184 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
farion1231/cc-switch#8829 ·
Los mantenedores suelen responder en 1 día
-
[core-client] Remove unused hand-maintained transport helpers after the definition-table cutoverAbiertodoc-audit enhancement
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
-
neon functions new: Next: hint says to export env vars for neon dev, but dev loads .env.localAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
neondatabase/neon-pkgs#941 ·
Los mantenedores suelen responder en 1 día