refactor: consider migrating from hashicorp/golang-lru/v2/expirable to jellydator/ttlcache/v3

Đang mở
  • Có gì không ổn?
#1,014 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
35/100
Loại issue
Tái cấu trúc
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
go
Lĩnh vực
backend, performance, testing

Hướng nghiên cứu

Start by reviewing the listed cache call sites, beginning with pkg/processtree/process_tree_manager.go and pkg/rulebindingmanager/cache/cache.go, then inspect the existing golang-lru usage and run the relevant tests. Compare golang-lru with ttlcache using the requested parallel, allocation, lifecycle, and end-to-end benchmarks. Done means the benchmark criteria are met, remaining call sites are migrated, imports are removed where applicable, and leak coverage is added for stoppable caches.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Problem Statement

Several subsystems across node-agent currently use github.com/hashicorp/golang-lru/v2/expirable for in-memory TTL/LRU caching:

  • Process Tree: pkg/processtree/process_tree_manager.go (containerProcessTreeCache)
  • Rule Manager & CEL Libraries:
    • pkg/rulemanager/rule_manager.go (alertLogDedup)
    • pkg/rulebindingmanager/cache/cache.go (rulesForPod)
    • pkg/rulemanager/rulecooldown/rulecooldown.go (cooldownMap)
    • pkg/rulemanager/ruleadapters/creator.go (hashCache)
    • pkg/rulemanager/cel/libraries/cache/function_cache.go (CEL function cache)
    • pkg/rulemanager/cel/libraries/containerprofile/prestop_cache.go (prestop_cache)
  • SBOM Manager: pkg/sbommanager/v1/sbom_manager.go (failureRetries, crashLoopRetries, busyRetries)
  • Host FIM Sensor: pkg/hostfimsensor/v1/dedup_cache.go (dedupCache)
Limitations of golang-lru/v2/expirable:
  1. Uncloseable Background Goroutines: In golang-lru v2.0.7, expirable.NewLRU spawns a background ticker goroutine (deleteExpired) that runs indefinitely because the package does not provide a Close() method (explicitly noted in upstream comments in expirable_lru.go and upstream issue #159 / PR #161). While acceptable for node-agent startup singletons, any stoppable or dynamically created component leaks goroutines on shutdown.
  2. Release Stagnation: The upstream repository is in low-touch maintenance; the last tagged release (v2.0.7) was published in September 2023.
Proposed Solution

Evaluate and migrate from hashicorp/golang-lru/v2/expirable to github.com/jellydator/ttlcache/v3.

Why jellydator/ttlcache/v3?
  • Explicit Lifecycle Control: Exposes Start() and Stop() methods for background expiration, or can run in purely passive/lazy eviction mode without spawning any background goroutines.
  • Modern Go Generics: Type-safe (ttlcache.Cache[K, V]).
  • Flexible Eviction & Sizing: Supports capacity bounds with LRU / FIFO eviction policies.
  • Active Maintenance: Widely adopted in the Go ecosystem as a standard replacement for go-cache and unmaintained expirable caches.
Benchmarking & Performance Requirements

Because several of these caches sit on critical, high-frequency event processing paths (e.g. ProcessTreeManager on every event, rulesForPod on every pod evaluation), the migration must satisfy the following benchmarking criteria:

  1. Hot-Path Concurrency & Latency (Benchmark*):
    • Implement comparative Go microbenchmarks using testing.B.RunParallel comparing golang-lru/v2 vs jellydator/ttlcache/v3.
    • Test under representative access patterns:
      • Read-heavy (90% read / 10% write) hit scenarios.
      • Miss-heavy and high-churn insertion scenarios.
    • Measure lock contention and ensure p95/p99 operation latencies do not regress.
  2. Memory Footprint & Allocation Overhead:
    • Measure B/op and allocs/op with go test -benchmem.
    • Profile heap allocations and verify that memory usage remains stable under sustained maximum entry capacity.
    • Compare performance and memory overhead between active background sweeper mode vs passive (lazy on-access) eviction mode.
  3. Goroutine Lifecycle & Leak Verification:
    • Automated tests must verify that stopping or discarding caches leaves zero running background goroutines (e.g. using go.uber.org/goleak or runtime.NumGoroutine()).
  4. End-to-End System Benchmark:
    • Validate against the existing node-agent performance benchmark suite (make benchmark / CI performance workflow) to ensure no regressions in overall node-agent CPU utilization and event throughput under load.
Implementation Plan
  • Implement comparative microbenchmarks (golang-lru vs ttlcache/v3) covering read/write contention and memory allocations.
  • Prototype ttlcache/v3 in one subsystem (e.g. pkg/processtree or pkg/rulebindingmanager).
  • Validate microbenchmark and CI benchmark results against the baseline.
  • Migrate remaining expirable.LRU call sites across the codebase.
  • Remove github.com/hashicorp/golang-lru/v2/expirable imports where applicable.
  • Add goroutine leak tests for all stoppable cache instances.
Ngôn ngữ chính
Go
Star
39
Fork
42
Merge trung bình
1 ngày 18 giờ
Pull request đã merge (30 ngày)
40

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của kubescape/node-agent

Tất cả issue của kubescape/node-agent

Issue tương tự

Thêm issue về Go

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.