`light-webserver` reads each annotated screen synchronously inside the HTTP handler, stalling every other request
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 80/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- javascript, node.js
- Domain
- backend, performance
Research direction
Read skills/ce-brainstorm/scripts/light-webserver.js around line 1135 and compare the matching branch in skills/ce-prototype/scripts/light-webserver.js. Start by tracing the annotate document request path and how it reads and processes the HTML; reproduce the delay with the provided 32 MiB screen and concurrent /version requests. Done means the screen is served while unrelated requests stay responsive, with both copies addressed.
Written by the indexing model from the issue text.
Description
Environment
EveryInc/compound-engineering-pluginat commit122f9f90330df0b54da865e12cbecb81ffd0ae53- Node.js 24, Windows/macOS/Linux
skills/ce-brainstorm/scripts/light-webserver.jsand itsce-prototypecopy (same file content)
Summary
In annotate mode, the light-webserver reads a navigated HTML screen with fs.readFileSync from inside the async http.createServer handler, then runs annotateScreen over the full string before writing a byte. Nothing is awaited first, so the read and the string walk both run on the event-loop thread. While a large screen is being read and annotated, every other socket waits: a 3 ms /version request that normally answers in about 3 ms takes as long as the navigation's time-to-first-byte. At 32 MiB the stall is about 1.5 s; at 128 MiB about 3.3 s.
Root cause
// light-webserver.js:1129-1141, inside `if (options.annotate)` in async handleRequest()
if (req.method === "GET") {
touch()
const filePath = resolveContainedFile(options.screensDir, req, res)
if (!filePath) return
if (contentType(filePath) === CONTENT_TYPES[".html"] && isDocumentNavigation(req)) {
const renderedKey = screensChangeKey(options)
serveAnnotateDocument(req, res, annotateScreen(fs.readFileSync(filePath, "utf8"), requestOrigin(req), urlPath), renderedKey) // line 1135
return
}
...
handleRequest is async and invoked from the http.createServer callback, but there is no await before line 1135. The synchronous read and the subsequent annotateScreen pass (html.replace, isFullDocument with toLowerCase()) are linear in the screen size and block the single event loop shared by /version, /events, /wait and /annotation.
Affected code
skills/ce-brainstorm/scripts/light-webserver.js, line 1135 (annotate document branch). The same code exists at skills/ce-prototype/scripts/light-webserver.js:1135; both files are one blob at the audited commit.
Reproduction
- Create a full HTML document of the target size, for example
head -c 33554432 /dev/zero | tr '\0' 'a'wrapped in<!doctype html><html><body>...</body></html>, and place it atscreens/big.htmlunder the server root. - Start the real server in annotate mode with a fixed port:
node skills/ce-brainstorm/scripts/light-webserver.js serve \
--root <run>/root --host 127.0.0.1 --port 4180 --annotate
- Send one document navigation for
/big.html(Accept: text/html,Sec-Fetch-Dest: document) and, from the same client, four/versionrequests 3 ms apart on independent sockets. - Repeat the four requests with no navigation in flight to get the control.
Observed with Node 24 on Windows 11:
| screen | navigation TTFB | request max | requests over 50 ms | control request max |
|---|---|---|---|---|
| 1 KiB | 4.4 ms | 2.7 ms | 0 / 4 | 4.3 ms |
| 8 MiB | 94.7 ms | 71.6 ms | 2 / 4 | 3.5 ms |
| 32 MiB | 1508.9 ms | 1528.5 ms | 4 / 4 | 5.2 ms |
| 128 MiB | 3342.7 ms | 3351.8 ms | 4 / 4 | 4.3 ms |
At 32 MiB the four requests issued at +20/+40/+60/+80 ms all complete at about 1.5 s, in a single burst at the moment the handler returns; the same endpoint answers in 3鈥? ms when no navigation is in flight. A repeat run measured 1280.9 ms and 3309.8 ms TTFB for 32/128 MiB.
Expected behavior
A slow screen read should not stop unrelated clients. The handler should read asynchronously (await fs.promises.readFile) and, if the annotation pass is expensive, yield or perform it off the request path, so /version, /events and other sockets stay responsive while a page is being prepared.
Actual behavior
The handler performs the whole read and annotation synchronously before writing the first body byte. All other requests are suspended for the duration; the delay grows linearly with the screen size and the same 3 ms endpoint becomes roughly 500x slower at 32 MiB.
Suggested fix
const raw = await fs.promises.readFile(filePath, "utf8");
const body = annotateScreen(raw, requestOrigin(req), urlPath);
serveAnnotateDocument(req, res, body, renderedKey);
If annotateScreen remains a long synchronous string pass, consider chunking it or moving it to a worker for large documents.
- Dominant language
- TypeScript
- Stars
- 25.6k
- Forks
- 2.1k
- Avg merge
- 3h 57m
- Merged PRs (30d)
- 170
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from EveryInc/compound-engineering-plugin
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
EveryInc/compound-engineering-plugin#2085 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
EveryInc/compound-engineering-plugin#795 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
EveryInc/compound-engineering-plugin#2083 ·
Maintainers usually reply within 1 day
-
Host support: DeepSeek Harness (dsh)Possibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 3/5 1-2 days Newbie friendliness 48/100
EveryInc/compound-engineering-plugin#2072 ·
Maintainers usually reply within 1 day
-
Respect a user policy of working on the default branch (ce-commit, ce-debug, ce-worktree, ce-test-browser, ce-dogfood)Possibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 5/5 Over a week Newbie friendliness 22/100
EveryInc/compound-engineering-plugin#2071 ·
Maintainers usually reply within 1 day
All issues in EveryInc/compound-engineering-plugin
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
hurttlocker/o8#6535 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 1 day
-
area: backend bug priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
snapotter-hq/SnapOtter#4856 ·
Maintainers usually reply within 1 day
-
bug follow-up
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
spellshift/realm#2509 ·
Maintainers usually reply within 1 day