stats.sonicjs.com: /admin/content 500 — D1 overloaded by CI playwright ingest bursts
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 35/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- playwright, typescript
- Ambito
- backend-api-design, databases, performance
Direzione di ricerca
Start by tracing playwrightIngest and the bootstrap KV skip-marker behavior described in the issue, then inspect the D1 batch insert path and its tests. Reproduce the overload with a large ingest batch and check how retries, batch sizing, and bootstrap behavior affect D1 usage. Done means ingest bursts no longer cause repeated bootstrap work or 500 responses on admin routes; the issue does not identify specific files or tests.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
/admin/content (and likely all admin routes) return {"error":"Internal Server Error","status":500} on stats.sonicjs.com due to D1 resource exhaustion caused by high-volume CI playwright ingest traffic.
Root cause (confirmed from wrangler tail)
The stats D1 hits Cloudflare's per-10-second resource limit when the CI ships large playwright test batches. Every cold-start isolate re-runs bootstrap (because bootstrap never caches its KV skip-marker when D1 is overloaded), creating a feedback loop:
D1_ERROR: D1 DB is overloaded. Too many resources consumed within a 10 second window.
Errors seen on:
[MigrationService] auth_two_factor lockout column repair failed[Bootstrap] Error registering document types[Bootstrap] Error auto-registering collection document types[PluginBootstrap] Error ensuring plugin Authentication System / Media Manager / Lexical Editor[playwright] /v1/playwright-runs insert failed
When a user hits /admin/content during a burst, any D1 query inside the route handler throws the same overload error. It escapes the route's try/catch (RBAC middleware runs before the route handler), hits the global app.onError → JSON 500.
Trigger
CI is processing multiple playwright runs in parallel (logs show simultaneous POST /v1/playwright-runs/{id}/tests from run 44587182379-1). Each batch inserts ~750 test rows via D1 batch(). At peak parallelism this saturates the stats D1's 10s resource window.
Secondary issue: JWT_SECRET not set
Every cold-start logs:
[SonicJS Security] JWT_SECRET is not set — using hardcoded fallback.
The stats worker is running without a JWT_SECRET Wrangler secret — admin sessions are signed with the public fallback key (security risk).
Fixes
Immediate:
- Set
JWT_SECRETon the stats worker:wrangler secret put JWT_SECRET --name sonicjs-stats
Short-term (D1 overload):
- Throttle playwright ingest: add a small delay/retry between batch inserts in
playwrightIngest - Cap per-request D1 batch size below 500 statements to stay under D1's row-write budget per 10s window
- Reduce CI playwright ingest parallelism
Long-term:
- Bootstrap KV skip-marker never writes when D1 is overloaded → every cold-start isolate retries full bootstrap during the same burst. Add in-memory backoff or circuit-breaker to avoid compounding the overload.
- Consider upgrading the stats D1 from the free tier to a paid plan for higher resource limits.
Steps to reproduce
Trigger a large CI playwright run while tailing wrangler tail sonicjs-stats --format=pretty, then visit /admin/content during the ingest burst.
- Lingua principale
- TypeScript
- Stelle
- 1.8k
- Fork
- 219
- Merge medio
- 2g 21h
- PR unite (30g)
- 67
Preparare l'ambiente
- Include un Dockerfile o un file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di SonicJs-Org/sonicjs
-
bug(admin/users): server error returns bare alert HTML causing black-shape visual artifactForse già presa @lane711 l’ha presa oggi. Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
SonicJs-Org/sonicjs#1211 ·
I maintainer di solito rispondono entro 1 giorno
-
area: api bug priority: P2
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
SonicJs-Org/sonicjs#1200 ·
I maintainer di solito rispondono entro 1 giorno
-
ResendProvider cannot send on Workers: unbound `fetch` stored as an instance property throws "Illegal invocation"Forse già presa @lane711 l’ha presa 23 giorni fa. Apertaarea: email bug priority: P1
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
SonicJs-Org/sonicjs#1099 ·
I maintainer di solito rispondono entro 1 giorno
-
Email plugin settings are never read: loadDbEmailSettings queries a v2 `plugins` table that does not exist in v3Forse già presa @lane711 l’ha presa 23 giorni fa. Apertaarea: email bug priority: P1
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
SonicJs-Org/sonicjs#1098 ·
I maintainer di solito rispondono entro 1 giorno
-
area: documentation bug documentation priority: P2
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
SonicJs-Org/sonicjs#1012 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di SonicJs-Org/sonicjs
Issue simili
-
awaiting-final-review request-icon
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 70/100
homebridge/plugins#1376 · 1 commento ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
mindfold-ai/Trellis#672 ·
-
p1 state:active track
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
StellarCanary/ProtocolCanary-Action#698 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
pear-devs/pear-desktop#4839 ·
I maintainer di solito rispondono entro 1 giorno