Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Docs and operating-model requests 504: content hydration crawls 2,179 GitHub blobs serially inside a 30s Lambda

Aperta
  • Qualcosa non va?
#230 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
aws, github, typescript
Ambito
backend, cloud, observability, performance

Direzione di ricerca

Start with backend/src/docs/githubStore.ts and trace how ContentsApiGithubStore.sync() is called by operatingModel/loader.ts. Reproduce the direct docs endpoint timeout, then choose and scope a hydration approach that completes within the Lambda budget and makes GitHub requests fail fast. Also inspect seedRuntimeUsers for the existing-user role backfill; done means docs and operating-model endpoints recover, hangs return promptly, and missing roles cannot recur.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Symptom

The portal Today page shows "Process documents are unavailable — HTTP 504", an empty action queue, and My Plan / operating-model data missing. Work API calls were also failing (fixed separately, see below).

Diagnosis (verified against production 2026-09-26)

  • GET /docs/process-quality invoked directly on the backend Lambda returns Sandbox.Timedout after 30.00s; /health on the same container returns 200 in ~2s. CloudFront surfaces the timeout as 504.
  • Every docs/operating-model request calls ContentsApiGithubStore.sync() (backend/src/docs/githubStore.ts), which on a cold container hydrates the whole content/ tree one GitHub API call per blob, serially, with no fetch timeout.
  • DataTalksClub/dataops-knowledge content/ currently holds 2,179 blobs (443 md, 1,516 jpg, 220 png). At ~0.3–0.5s per blobs call that is ~10–15 minutes of work inside a 30s Lambda, so hydration never completes in one invocation. Each new container restarts the crawl (progress persists only in per-container /tmp), so no container ever becomes warm.
  • Blast radius: /docs/*, /search, /api/operating-model, /api/my-plan, and the Today page's process-quality call (operatingModel/loader.ts calls the same store.sync()).
  • Side damage: each doomed crawl burns up to ~2,200 GitHub requests; the knowledge token's rate limit already showed 1,301/5,000 used within one hour.

Work API 403s (fixed in data on 2026-09-26, needs a repo-level guard)

  • The #164 role gate denies users without a supported role. The three live user items predate the role attribute (created 2026-06-28, no role), and seedRuntimeUsers skips existing users as "unchanged", so the role was never backfilled. Every team read/work mutation returned 403 ("Team access requires an active admin or operator role").
  • Applied the declared seed state (role: 'admin' for grace/valeriia/alexey) directly to dataops-v1-users via UpdateItem; /api/cards and /api/tasks verified 200 after.
  • Follow-up needed: make seedRuntimeUsers update missing attributes on existing users (or migrate in place), so this cannot recur. Data migration was done by hand this time only because the gate shipped without it.

Fix directions (for grooming)

  1. Stop eager per-blob hydration. Candidates: hydrate only .md eagerly and fetch images lazily via ensureFile; or download a single tarball and extract content/; or sync the knowledge repo to S3 from CI and hydrate from S3 in one/few calls.
  2. Add an explicit timeout (AbortSignal.timeout) to request()/fetchImpl in githubStore.ts so a GitHub hang fails fast with 502 instead of eating the whole Lambda budget.
  3. Consider a startup/deploy-time hydration step (offline snapshot in the artifact) instead of on-demand crawling.

Related observability gap

Backend Lambda log streams stop on 2026-08-11 even though the function is actively invoked (CloudWatch metrics show hundreds of invocations/day; direct invokes today produced no streams). The execution role still has AWSLambdaBasicExecutionRole attached. Root cause unknown — needs its own investigation; until fixed, production debugging is blind (this diagnosis had to be reproduced by direct invokes).

Lingua principale
TypeScript
Stelle
2
Fork
0
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Nessun modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di DataTalksClub/dataops

Tutte le issue di DataTalksClub/dataops

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.