10.1.4 regression: path spelling validation calls readdir() on every request, O(n) latency for large directories
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- javascript, node.js
- Domain
- backend, performance
Research direction
Start with getPathSpellingStatus() and compare its behavior between versions 10.1.3 and 10.1.4. Run the supplied repro.mjs against both versions, then trace the default wildcard route to identify where readdir() is called. Done means restoring request latency that does not scale with directory size while preserving the spelling-validation behavior where it is needed.
Written by the indexing model from the issue text.
Description
Prerequisites
- I have written a descriptive issue title
- I have searched existing issues to ensure the regression has not already been reported
Last working version
10.1.3
Stopped working in version
10.1.4
Node.js version
24.21.0
Operating system
Linux
Operating system version (i.e. 20.04, 11.3, 10)
Ubuntu, kernel 6.8.0 (app runs in node:24-alpine; static root is an ext4 bind mount)
💥 Regression Report
Since 10.1.4 (the fix for GHSA-r799-r9gc-m956), getPathSpellingStatus() runs on every request served through the default wildcard: true route. For each path segment it calls readdir() on the parent directory and then entries.includes(segment). So every request costs time proportional to the number of entries in each directory along the path.
We serve product photos from one flat directory with ~815k files. After upgrading from 10.1.3 to 10.1.4, median latency for /static/product/photo/.webp in production went from ~3 ms to ~1.8 s (p90 ~4 s, p99 ~10 s). That includes 304 Not Modified responses and requests for non-existent files. Files in small directories under the same root are unaffected (~7 ms). A single stat() of the same file takes <1 ms on the host and inside the container, so the filesystem is not the bottleneck.
The spelling check only matters on case-insensitive filesystems. On case-sensitive ones (ext4, the default on Linux), a case alias cannot resolve to a different file, yet the full directory scan still runs on every request.
It is also an amplification vector: any unauthenticated, well-formed request under a large directory triggers a full readdir() of it.
Steps to Reproduce
// repro.mjs — node repro.mjs 10.1.3|10.1.4 (after installing that version) import Fastify from 'fastify' import fastifyStatic from '@fastify/static' import { mkdirSync, writeFileSync } from 'node:fs' import { join } from 'node:path' import { tmpdir } from 'node:os'const root = join(tmpdir(), 'fastify-static-large-dir')
const dir = join(root, 'photo')
mkdirSync(dir, { recursive: true })
for (let i = 0; i < 300_000; i++) writeFileSync(join(dir,f${i}.webp), '')const app = Fastify()
await app.register(fastifyStatic, { root, prefix: '/static/' })
await app.ready()
const n = 20
const t = performance.now()
for (let i = 0; i < n; i++) {
const res = await app.inject(/static/photo/f${i * 1000 + 7}.webp)
if (res.statusCode !== 200) throw new Error(String(res.statusCode))
}
console.log(process.argv[2], 'avg ms/request:', ((performance.now() - t) / n).toFixed(1))
Results (macOS, 300k files, sequential requests):
| version | avg per request |
|---|---|
| 10.1.3 | 1.9 ms |
| 10.1.4 | 419 ms |
Expected Behavior
Serving a file should not scale with the size of its directory, as in 10.1.3. Some possible approaches:
Detect once per root at startup whether the filesystem is case-insensitive (e.g. stat an upper-cased variant of a known entry), and skip the spelling check on case-sensitive filesystems.
Or cache the directory listing per directory, invalidated by mtime, and use a Set instead of Array#includes.
Or provide an option to opt out of the spelling check, for deployments on a known case-sensitive filesystem.
As a workaround we pinned @fastify/[email protected].
- Dominant language
- JavaScript
- Stars
- 499
- Forks
- 131
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 1
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from fastify/fastify-static
-
bug: directory index route root priority is overwritten when wildcard is falsePossibly taken @TalhaHunter101 claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
fastify/fastify-static#617 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
fastify/fastify-static#612 · 1 comment ·
-
Use built-in glob from `node:fs`Possibly taken @Ankitraj-sharma claimed this 13 days ago. Opengood first issue
Difficulty 3/5 1-2 days Newbie friendliness 25/100
fastify/fastify-static#551 · 8 comments · 1 reaction ·
-
feature request
Difficulty 5/5 Over a week Newbie friendliness 25/100
fastify/fastify-static#432 · 2 comments ·
-
discussion
Difficulty 5/5 Over a week Newbie friendliness 25/100
fastify/fastify-static#385 · 2 comments ·
All issues in fastify/fastify-static
Similar issues
-
Language MVP task
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
saayam-for-all/webapp#1914 ·
Maintainers usually reply within 1 day
-
[Bug] Dev image build fails: `npm install` in the root Dockerfile runs out of memory (exit 134)Possibly taken @andest01 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
good first issue hacktoberfest help wanted
Difficulty 2/5 Under an hour Newbie friendliness 76/100
pacocartones/free-llm-api-hub#262 ·
Maintainers usually reply within 1 day
-
agent-task risk:low
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
guyellis/http-status-check#512 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
microsoft/playwright#43327 ·
Maintainers usually reply within 1 day