kubeflow/sdk

SparkConnect CR cleanup behavior on failure and SDK state mismatch

Offen

#476 geöffnet am 30.04.2026

 (6 Kommentare) (1 Reaktion) (1 zugewiesene Person)Python (196 Forks)auto 404
area/sparkgood first issuehelp wantedkind/bug

Repository-Metriken

Stars
 (124 Sterne)
PR-Merge-Metriken
 (PR-Metriken ausstehend)

Beschreibung

Summary

While working on port-forward cleanup in the SDK #460 , it surfaced that there may be a broader gap in how SparkConnect resources are handled on failure.

Observations

  • SparkConnect CR does not appear to clean up resources when connection/setup fails
  • This can leave resources running if failure happens mid-connection

There is a related issue in the Spark Operator: kubeflow/spark-operator#2927

Additional Observation

There also appears to be a mismatch in state mapping:

  • The SDK exposes a "Running" state
  • SparkConnect CR does not expose a corresponding state

Contributor Guide