Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Inference Operator v3.2.0 ignores l2CacheBackend: redis, hardcodes LMCACHE_REMOTE_URL=sagemaker-hyperpod://...

Aperta
  • Qualcosa non va?
#431 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
52/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
aws, kubernetes, python, redis
Ambito
cloud, databases, infrastructure

Direzione di ricerca

Inizia riproducendo la InferenceEndpointConfig e ispeziona il worker pod renderizzato con il comando kubectl jsonpath fornito. Traccia il modo in cui l’operatore gestisce l2CacheBackend, l2CacheLocalUrl e worker.environmentVariables; il lavoro è completo quando Redis produce LMCACHE_REMOTE_URL=redis://..., il worker segnala un backend LMCache integro e il percorso di deployment di tieredstorage è documentato o chiarito.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Environment
  • EKS add-on amazon-sagemaker-hyperpod-inference v1.3.0-eksbuild.1 (latest available in ap-northeast-1)
  • Inference Operator image hyperpod-inference-operator:v3.2.0
  • Worker image lmcache/vllm-openai:v0.4.7 (LMCache 0.4.7, vLLM 0.23.0)
  • Instance type ml.g7e.4xlarge, HyperPod EKS-orchestrated cluster
What happened

Setting kvCacheSpec.l2CacheSpec.l2CacheBackend: redis (with l2CacheLocalUrl: redis://...:6379) in the InferenceEndpointConfig has no effect. The operator instead injects into the worker pod:

LMCACHE_REMOTE_URL=sagemaker-hyperpod://$(NODE_IP):9200
LMCACHE_EXTRA_CONFIG={"sagemaker_hyperpod_shared_memory_name": "ai_toolkit_cache"}

The redis value from the CRD is silently dropped — the operator never emits LMCACHE_REMOTE_URL=redis://.... Attempting to override LMCACHE_REMOTE_URL via worker.environmentVariables does not work either: those LMCACHE_* entries do not appear in the rendered pod (the operator strips/overrides them).

Because the worker is forced onto the sagemaker-hyperpod connector, and that connector requires a host-side ai-toolkit daemon (POSIX shared memory /ai_toolkit_cache + TCP :9200) which is not present on the cluster, LMCache enters degraded mode and the L2 cache never stores anything:

LMCache ERROR: Failed to initialize shared memory: [Errno 22] Invalid argument: '/ai_toolkit_cache'
LMCache WARNING: Health check failed: RemoteBackendHealthCheck(sagemaker-hyperpod://<NODE_IP>:9200)
LMCache WARNING: HealthMonitor: System unhealthy, entering degraded mode
LMCache WARNING: LMCache is unhealthy, skipping store operation
... LMCache hit tokens: 0     (External prefix cache hit rate: 0.0%)
Expected behavior

With l2CacheBackend: redis + l2CacheLocalUrl: redis://...:6379, the worker's LMCache should be configured with LMCACHE_REMOTE_URL=redis://..., connect to the specified Redis endpoint, report healthy, and perform L2 KV-cache store/lookup. redis is listed as a supported L2 backend in the docs (KV cache & intelligent routing), so the operator overriding it contradicts the documentation.

Reproduction
  1. Deploy an InferenceEndpointConfig with: 
    kvCacheSpec:
  enableL1Cache: true
  enableL2Cache: true
  l2CacheSpec:
    l2CacheBackend: redis
    l2CacheLocalUrl: redis://<redis-svc>.<ns>.svc.cluster.local:6379
  2. (Optionally) add worker.environmentVariables entries for LMCACHE_REMOTE_URL to try to override.
  3. Inspect the rendered worker pod: 
    kubectl get pod <worker> -o jsonpath='{range .spec.containers[*].env[*]}{.name}={.value}{"\n"}{end}' | grep LMCACHE
  4. Observe LMCACHE_REMOTE_URL=sagemaker-hyperpod://$(NODE_IP):9200 (not redis://...), and the worker logs showing the LMCache unhealthy/degraded loop above.
Additional question (tieredstorage)

Separately: l2CacheBackend: tieredstorage requires the host-side ai-toolkit daemon (shm /ai_toolkit_cache + :9200). On our cluster this daemon is not installed as any DaemonSet, and the add-on configuration schema (aws eks describe-addon-configuration) exposes no toggle for it (only alb, enableCustomServiceAccounts, executionRoleArn, hyperpodClusterArn, jumpstartGatedModelDownloadRoleArn, keda, tlsCertificateS3Bucket). How is tiered storage meant to be enabled — a cluster-creation flag, and can it be enabled on an existing cluster?

References
Lingua principale
Python
Stelle
41
Fork
95
Merge medio
18h 29m
PR unite (30g)
5

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di aws/sagemaker-hyperpod-cli

Tutte le issue di aws/sagemaker-hyperpod-cli

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.