refactor: consider migrating from hashicorp/golang-lru/v2/expirable to jellydator/ttlcache/v3

Abierto
  • ¿Algo no está bien?
#1,014 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Refactorización
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
go
Área
backend, performance, testing

Línea de trabajo

Start by reviewing the listed cache call sites, beginning with pkg/processtree/process_tree_manager.go and pkg/rulebindingmanager/cache/cache.go, then inspect the existing golang-lru usage and run the relevant tests. Compare golang-lru with ttlcache using the requested parallel, allocation, lifecycle, and end-to-end benchmarks. Done means the benchmark criteria are met, remaining call sites are migrated, imports are removed where applicable, and leak coverage is added for stoppable caches.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Problem Statement

Several subsystems across node-agent currently use github.com/hashicorp/golang-lru/v2/expirable for in-memory TTL/LRU caching:

  • Process Tree: pkg/processtree/process_tree_manager.go (containerProcessTreeCache)
  • Rule Manager & CEL Libraries:
    • pkg/rulemanager/rule_manager.go (alertLogDedup)
    • pkg/rulebindingmanager/cache/cache.go (rulesForPod)
    • pkg/rulemanager/rulecooldown/rulecooldown.go (cooldownMap)
    • pkg/rulemanager/ruleadapters/creator.go (hashCache)
    • pkg/rulemanager/cel/libraries/cache/function_cache.go (CEL function cache)
    • pkg/rulemanager/cel/libraries/containerprofile/prestop_cache.go (prestop_cache)
  • SBOM Manager: pkg/sbommanager/v1/sbom_manager.go (failureRetries, crashLoopRetries, busyRetries)
  • Host FIM Sensor: pkg/hostfimsensor/v1/dedup_cache.go (dedupCache)
Limitations of golang-lru/v2/expirable:
  1. Uncloseable Background Goroutines: In golang-lru v2.0.7, expirable.NewLRU spawns a background ticker goroutine (deleteExpired) that runs indefinitely because the package does not provide a Close() method (explicitly noted in upstream comments in expirable_lru.go and upstream issue #159 / PR #161). While acceptable for node-agent startup singletons, any stoppable or dynamically created component leaks goroutines on shutdown.
  2. Release Stagnation: The upstream repository is in low-touch maintenance; the last tagged release (v2.0.7) was published in September 2023.
Proposed Solution

Evaluate and migrate from hashicorp/golang-lru/v2/expirable to github.com/jellydator/ttlcache/v3.

Why jellydator/ttlcache/v3?
  • Explicit Lifecycle Control: Exposes Start() and Stop() methods for background expiration, or can run in purely passive/lazy eviction mode without spawning any background goroutines.
  • Modern Go Generics: Type-safe (ttlcache.Cache[K, V]).
  • Flexible Eviction & Sizing: Supports capacity bounds with LRU / FIFO eviction policies.
  • Active Maintenance: Widely adopted in the Go ecosystem as a standard replacement for go-cache and unmaintained expirable caches.
Benchmarking & Performance Requirements

Because several of these caches sit on critical, high-frequency event processing paths (e.g. ProcessTreeManager on every event, rulesForPod on every pod evaluation), the migration must satisfy the following benchmarking criteria:

  1. Hot-Path Concurrency & Latency (Benchmark*):
    • Implement comparative Go microbenchmarks using testing.B.RunParallel comparing golang-lru/v2 vs jellydator/ttlcache/v3.
    • Test under representative access patterns:
      • Read-heavy (90% read / 10% write) hit scenarios.
      • Miss-heavy and high-churn insertion scenarios.
    • Measure lock contention and ensure p95/p99 operation latencies do not regress.
  2. Memory Footprint & Allocation Overhead:
    • Measure B/op and allocs/op with go test -benchmem.
    • Profile heap allocations and verify that memory usage remains stable under sustained maximum entry capacity.
    • Compare performance and memory overhead between active background sweeper mode vs passive (lazy on-access) eviction mode.
  3. Goroutine Lifecycle & Leak Verification:
    • Automated tests must verify that stopping or discarding caches leaves zero running background goroutines (e.g. using go.uber.org/goleak or runtime.NumGoroutine()).
  4. End-to-End System Benchmark:
    • Validate against the existing node-agent performance benchmark suite (make benchmark / CI performance workflow) to ensure no regressions in overall node-agent CPU utilization and event throughput under load.
Implementation Plan
  • Implement comparative microbenchmarks (golang-lru vs ttlcache/v3) covering read/write contention and memory allocations.
  • Prototype ttlcache/v3 in one subsystem (e.g. pkg/processtree or pkg/rulebindingmanager).
  • Validate microbenchmark and CI benchmark results against the baseline.
  • Migrate remaining expirable.LRU call sites across the codebase.
  • Remove github.com/hashicorp/golang-lru/v2/expirable imports where applicable.
  • Add goroutine leak tests for all stoppable cache instances.
Lenguaje dominante
Go
Estrellas
39
Forks
42
Merge medio
1 d 18 h
PR fusionados (30 d)
40

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de kubescape/node-agent

Todos los issues de kubescape/node-agent

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.