stats.sonicjs.com: /admin/content 500 — D1 overloaded by CI playwright ingest bursts
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- playwright, typescript
- Domain
- backend-api-design, databases, performance
Research direction
Start by tracing playwrightIngest and the bootstrap KV skip-marker behavior described in the issue, then inspect the D1 batch insert path and its tests. Reproduce the overload with a large ingest batch and check how retries, batch sizing, and bootstrap behavior affect D1 usage. Done means ingest bursts no longer cause repeated bootstrap work or 500 responses on admin routes; the issue does not identify specific files or tests.
Written by the indexing model from the issue text.
Description
Summary
/admin/content (and likely all admin routes) return {"error":"Internal Server Error","status":500} on stats.sonicjs.com due to D1 resource exhaustion caused by high-volume CI playwright ingest traffic.
Root cause (confirmed from wrangler tail)
The stats D1 hits Cloudflare's per-10-second resource limit when the CI ships large playwright test batches. Every cold-start isolate re-runs bootstrap (because bootstrap never caches its KV skip-marker when D1 is overloaded), creating a feedback loop:
D1_ERROR: D1 DB is overloaded. Too many resources consumed within a 10 second window.
Errors seen on:
[MigrationService] auth_two_factor lockout column repair failed[Bootstrap] Error registering document types[Bootstrap] Error auto-registering collection document types[PluginBootstrap] Error ensuring plugin Authentication System / Media Manager / Lexical Editor[playwright] /v1/playwright-runs insert failed
When a user hits /admin/content during a burst, any D1 query inside the route handler throws the same overload error. It escapes the route's try/catch (RBAC middleware runs before the route handler), hits the global app.onError → JSON 500.
Trigger
CI is processing multiple playwright runs in parallel (logs show simultaneous POST /v1/playwright-runs/{id}/tests from run 44587182379-1). Each batch inserts ~750 test rows via D1 batch(). At peak parallelism this saturates the stats D1's 10s resource window.
Secondary issue: JWT_SECRET not set
Every cold-start logs:
[SonicJS Security] JWT_SECRET is not set — using hardcoded fallback.
The stats worker is running without a JWT_SECRET Wrangler secret — admin sessions are signed with the public fallback key (security risk).
Fixes
Immediate:
- Set
JWT_SECRETon the stats worker:wrangler secret put JWT_SECRET --name sonicjs-stats
Short-term (D1 overload):
- Throttle playwright ingest: add a small delay/retry between batch inserts in
playwrightIngest - Cap per-request D1 batch size below 500 statements to stay under D1's row-write budget per 10s window
- Reduce CI playwright ingest parallelism
Long-term:
- Bootstrap KV skip-marker never writes when D1 is overloaded → every cold-start isolate retries full bootstrap during the same burst. Add in-memory backoff or circuit-breaker to avoid compounding the overload.
- Consider upgrading the stats D1 from the free tier to a paid plan for higher resource limits.
Steps to reproduce
Trigger a large CI playwright run while tailing wrangler tail sonicjs-stats --format=pretty, then visit /admin/content during the ingest burst.
- Dominant language
- TypeScript
- Stars
- 1.8k
- Forks
- 219
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 67
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from SonicJs-Org/sonicjs
-
bug(admin/users): server error returns bare alert HTML causing black-shape visual artifactPossibly taken @lane711 claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
SonicJs-Org/sonicjs#1211 ·
Maintainers usually reply within 1 day
-
area: api bug priority: P2
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
SonicJs-Org/sonicjs#1200 ·
Maintainers usually reply within 1 day
-
ResendProvider cannot send on Workers: unbound `fetch` stored as an instance property throws "Illegal invocation"Possibly taken @lane711 claimed this 23 days ago. Openarea: email bug priority: P1
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
SonicJs-Org/sonicjs#1099 ·
Maintainers usually reply within 1 day
-
Email plugin settings are never read: loadDbEmailSettings queries a v2 `plugins` table that does not exist in v3Possibly taken @lane711 claimed this 23 days ago. Openarea: email bug priority: P1
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
SonicJs-Org/sonicjs#1098 ·
Maintainers usually reply within 1 day
-
area: documentation bug documentation priority: P2
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
SonicJs-Org/sonicjs#1012 ·
Maintainers usually reply within 1 day
All issues in SonicJs-Org/sonicjs
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
AloisSeckar/elrh-cosca#36 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
cursor/community-plugins#572 ·
Maintainers usually reply within 1 day
-
[Plugin] calendarOpen
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
MUKE-coder/grit#228 ·
-
bug status:needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
dongdongbh/Mindwtr#1587 ·
Maintainers usually reply within 1 day