refactor: consider migrating from hashicorp/golang-lru/v2/expirable to jellydator/ttlcache/v3

オープン
  • 何か問題がありますか？
#1,014 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
35/100
issue の種類
リファクタリング
明瞭さ
おおむね明確
活発さ
活発
技術スタック
go
領域
backend, performance, testing

調査の方向性

Start by reviewing the listed cache call sites, beginning with pkg/processtree/process_tree_manager.go and pkg/rulebindingmanager/cache/cache.go, then inspect the existing golang-lru usage and run the relevant tests. Compare golang-lru with ttlcache using the requested parallel, allocation, lifecycle, and end-to-end benchmarks. Done means the benchmark criteria are met, remaining call sites are migrated, imports are removed where applicable, and leak coverage is added for stoppable caches.

索引モデルが issue の本文から書いたものです。

説明

Problem Statement

Several subsystems across node-agent currently use github.com/hashicorp/golang-lru/v2/expirable for in-memory TTL/LRU caching:

  • Process Tree: pkg/processtree/process_tree_manager.go (containerProcessTreeCache)
  • Rule Manager & CEL Libraries:
    • pkg/rulemanager/rule_manager.go (alertLogDedup)
    • pkg/rulebindingmanager/cache/cache.go (rulesForPod)
    • pkg/rulemanager/rulecooldown/rulecooldown.go (cooldownMap)
    • pkg/rulemanager/ruleadapters/creator.go (hashCache)
    • pkg/rulemanager/cel/libraries/cache/function_cache.go (CEL function cache)
    • pkg/rulemanager/cel/libraries/containerprofile/prestop_cache.go (prestop_cache)
  • SBOM Manager: pkg/sbommanager/v1/sbom_manager.go (failureRetries, crashLoopRetries, busyRetries)
  • Host FIM Sensor: pkg/hostfimsensor/v1/dedup_cache.go (dedupCache)
Limitations of golang-lru/v2/expirable:
  1. Uncloseable Background Goroutines: In golang-lru v2.0.7, expirable.NewLRU spawns a background ticker goroutine (deleteExpired) that runs indefinitely because the package does not provide a Close() method (explicitly noted in upstream comments in expirable_lru.go and upstream issue #159 / PR #161). While acceptable for node-agent startup singletons, any stoppable or dynamically created component leaks goroutines on shutdown.
  2. Release Stagnation: The upstream repository is in low-touch maintenance; the last tagged release (v2.0.7) was published in September 2023.
Proposed Solution

Evaluate and migrate from hashicorp/golang-lru/v2/expirable to github.com/jellydator/ttlcache/v3.

Why jellydator/ttlcache/v3?
  • Explicit Lifecycle Control: Exposes Start() and Stop() methods for background expiration, or can run in purely passive/lazy eviction mode without spawning any background goroutines.
  • Modern Go Generics: Type-safe (ttlcache.Cache[K, V]).
  • Flexible Eviction & Sizing: Supports capacity bounds with LRU / FIFO eviction policies.
  • Active Maintenance: Widely adopted in the Go ecosystem as a standard replacement for go-cache and unmaintained expirable caches.
Benchmarking & Performance Requirements

Because several of these caches sit on critical, high-frequency event processing paths (e.g. ProcessTreeManager on every event, rulesForPod on every pod evaluation), the migration must satisfy the following benchmarking criteria:

  1. Hot-Path Concurrency & Latency (Benchmark*):
    • Implement comparative Go microbenchmarks using testing.B.RunParallel comparing golang-lru/v2 vs jellydator/ttlcache/v3.
    • Test under representative access patterns:
      • Read-heavy (90% read / 10% write) hit scenarios.
      • Miss-heavy and high-churn insertion scenarios.
    • Measure lock contention and ensure p95/p99 operation latencies do not regress.
  2. Memory Footprint & Allocation Overhead:
    • Measure B/op and allocs/op with go test -benchmem.
    • Profile heap allocations and verify that memory usage remains stable under sustained maximum entry capacity.
    • Compare performance and memory overhead between active background sweeper mode vs passive (lazy on-access) eviction mode.
  3. Goroutine Lifecycle & Leak Verification:
    • Automated tests must verify that stopping or discarding caches leaves zero running background goroutines (e.g. using go.uber.org/goleak or runtime.NumGoroutine()).
  4. End-to-End System Benchmark:
    • Validate against the existing node-agent performance benchmark suite (make benchmark / CI performance workflow) to ensure no regressions in overall node-agent CPU utilization and event throughput under load.
Implementation Plan
  • Implement comparative microbenchmarks (golang-lru vs ttlcache/v3) covering read/write contention and memory allocations.
  • Prototype ttlcache/v3 in one subsystem (e.g. pkg/processtree or pkg/rulebindingmanager).
  • Validate microbenchmark and CI benchmark results against the baseline.
  • Migrate remaining expirable.LRU call sites across the codebase.
  • Remove github.com/hashicorp/golang-lru/v2/expirable imports where applicable.
  • Add goroutine leak tests for all stoppable cache instances.
主要言語
Go
スター
39
フォーク
42
平均マージ
1日 18時間
マージ済み PR（30日）
40

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

kubescape/node-agent のほかの issue

kubescape/node-agent の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。