Splunk Operator: Missing Hedless service or incorrect use of ClusterIP service.

Aperta
  • Qualcosa non va?
#1,885 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
66/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
go, kubernetes
Ambito
backend, devops

Direzione di ricerca

Start by tracing checkLicenseRelatedPodFailures and how the LicenseManager StatefulSet gets its spec.serviceName; compare that with the service creation paths for Standalone, IndexerCluster, and MonitoringConsole. Reproduce the missing DNS name in a Kubernetes deployment and identify whether the controller should create the referenced headless service or query the existing ClusterIP service. Done means the license check succeeds without recurring DNS errors for LicenseManager and ClusterManager.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Please select the type of request

Bug

Tell us more

Describe the request

Bug: In Splunk Operator 3.2.0 the LicenseManager controller logs recurring errors from
checkLicenseRelatedPodFailures, because it queries the license manager pod through its
headless-service DNS name (splunk-lm-license-manager-0.splunk-lm-license-manager-headless...).
The operator does not create a headless service for LicenseManager, so the name never resolves:

{"level":"ERROR","msg":"failed to get license information from Splunk API","controller":"LicenseManager",
 "func":"checkLicenseRelatedPodFailures","error":"Get \"https://splunk-lm-license-manager-0.splunk-lm-license-manager-headless.<namespace>.svc.cluster.local:8089/services/licenser/licenses?output_mode=json\":
 dial tcp: lookup splunk-lm-license-manager-0.splunk-lm-license-manager-headless.<namespace>.svc.cluster.local: no such host"}

The LicenseManager StatefulSet references the headless service in spec.serviceName, but only the
ClusterIP service exists. This matches the documented service set (docs/Ingress.md lists only
*-license-manager-service and *-cluster-manager-service, no headless services for these components):

$ kubectl get sts splunk-lm-license-manager -o jsonpath='{.spec.serviceName}'
splunk-lm-license-manager-headless

$ kubectl get svc | grep -E "lm-|cm-"
splunk-cm-cluster-manager-service   ClusterIP   ...   8000/TCP,8089/TCP
splunk-lm-license-manager-service   ClusterIP   ...   8000/TCP,8089/TCP

The same gap exists for ClusterManager (spec.serviceName: splunk-cm-cluster-manager-headless, service
not created), though we have not observed errors caused by it yet. Headless services are created for
Standalone, IndexerCluster and MonitoringConsole.

Expected behavior

The license check succeeds without errors. Either:

  1. The operator creates the headless service referenced in the StatefulSet's spec.serviceName for
    LicenseManager (and ClusterManager), or
  2. checkLicenseRelatedPodFailures reaches the license manager via splunk-<name>-license-manager-service
    instead of the pod's headless DNS name.

Splunk setup on K8S

  • Splunk Operator 3.2.0 (Helm chart; CRDs applied from the 3.2.0 release splunk-operator-crds.yaml)
  • Splunk Enterprise 9.4.16 and 10.2.8 (behaviour identical on both)
  • CRs: LicenseManager, ClusterManager, IndexerCluster (3 peers, RF2/SF2), MonitoringConsole,
    Standalone (search head) and Standalone (3 heavy forwarders)
  • TLS on splunkd with certificates from cert-manager

Reproduction/Testing steps

  1. Deploy a LicenseManager CR with Splunk Operator 3.2.0.
  2. Check the StatefulSet: kubectl get sts splunk-<name>-license-manager -o jsonpath='{.spec.serviceName}'
    → splunk-<name>-license-manager-headless
  3. Check services: kubectl get svc | grep license-manager → only splunk-<name>-license-manager-service
  4. Watch the operator log:
    kubectl logs deploy/splunk-operator-controller-manager -n <operator-ns> | grep checkLicenseRelatedPodFailures
    → no such host errors.

K8s environment

  • OpenShift 4.22.15 (v1.35.6)
  • Operator deployed via Helm, managed by Argo CD
  • Reproduced on two separate clusters

Proposed changes(optional)

Either create the headless service for LicenseManager/ClusterManager (consistent with spec.serviceName),
or have checkLicenseRelatedPodFailures use the existing ClusterIP service.

K8s collector data(optional)

Can be provided on request.

Additional context(optional)

Workaround: creating the headless service manually resolves the error (verified: no further
no such host errors after creation):

apiVersion: v1
kind: Service
metadata:
  name: splunk-lm-license-manager-headless
spec:
  clusterIP: None
  publishNotReadyAddresses: true
  selector:
    app.kubernetes.io/component: license-manager
    app.kubernetes.io/instance: splunk-lm-license-manager
    app.kubernetes.io/managed-by: splunk-operator
    app.kubernetes.io/name: license-manager
    app.kubernetes.io/part-of: splunk-lm-license-manager
  ports:
    - name: https-splunkd
      port: 8089
      targetPort: 8089

A manually created service could conflict if a future operator version starts creating this service
itself, so an upstream fix would be preferred.

Impact: licensing itself works (all components use the ClusterIP service). Only the operator's
license-related pod failure check is non-functional and produces recurring ERROR log entries.

Lingua principale
Go
Stelle
260
Fork
134
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di splunk/splunk-operator

Tutte le issue di splunk/splunk-operator

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.