Bot Detection Signing Key Rotation triggers synchronous WASM reload causing CPU spikes every 5 minutes

Aperta
  • Qualcosa non va?
#4,795 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
38/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
go, wasm
Ambito
backend, security

Direzione di ricerca

Start by tracing the AppSec engine’s Bot Detection signing-key rotation timer and the synchronous WASM reload it triggers; the issue does not name specific files or tests. Confirm the reload’s timing and CPU impact, then determine which proposed direction fits the project. Done means rotation no longer causes the reported recurring CPU spike while preserving key rotation behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

needs/kind needs/triage

Description

When CrowdSec AppSec Bot Detection is enabled, the Signing Key Rotation mechanism triggers a synchronous WASM module reload every 5 minutes (default rotation_interval=5m0s). This causes a predictable CPU spike lasting 4-5 seconds on small-instance deployments, during which the WASM runtime consumes 100% of a single core. On 2-core machines under concurrent load, this can cause request latency degradation or brief service unavailability.

The CPU spike is traffic-independent — it occurs at fixed intervals regardless of request volume, confirming it is an internal timer-driven operation rather than a response to incoming traffic.

Cause Analysis

The CPU spike is caused by the Bot Detection Signing Key Rotation timer (rotation_interval=5m0s), which triggers a synchronous WASM module reload in the AppSec engine. Key evidence:

  1. Spike timing aligns exactly with rotation_interval (not with traffic patterns or cron jobs)
  2. CrowdSec log shows: WAF challenge runtime initialized ... rotation_interval=5m0s
  3. Memory profile shows full WASM module deallocation + reallocation cycle
  4. The operation is single-threaded and blocks the AppSec runner during reload
  5. Multiple signing keys would make this WORSE (more frequent reloads), not better

Suggested Improvements

  1. Async WASM hot-reload: Perform signing key rotation and WASM module swap in a background goroutine, swapping the active module atomically after preparation completes
  2. Configurable rotation interval: Allow users to tune rotation_interval based on their security/performance tradeoff (e.g., 15m or 30m for small instances)
  3. Lazy rotation: Only rotate when the current key has been used for N challenges, rather than on a fixed timer
  4. WASM module pooling: Pre-allocate the new WASM instance before tearing down the old one to reduce memory allocation pressure
Lingua principale
Go
Stelle
15.1k
Fork
726
Merge medio
2g 51m
PR unite (30g)
18

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di crowdsecurity/crowdsec

Tutte le issue di crowdsecurity/crowdsec

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.