Application driver Pod is not terminated when Spark is OOMKilled
Maintainers usually reply within 1 day
@razvan is already working on this.
Since Oct 2, 2026.
Assessment
This issue has not been assessed yet.
Description
Affected Stackable version
all
Affected Apache Spark-on-Kubernetes version
No response
Current and expected behavior
When a Spark driver is OOMKilled, the Pod doesn't terminate.
The vector container keeps running even after the Spark process is gone because no Vector shutdown file is generated. This container then prevents the Pod from being terminated.
A shutdown file for vector cannot be generated in this case because SIGKILL cannot be trapped.
A workaround for this problem is to use a livenessProbe on the vector container that connects to the driver's communication port (7078 by default).
Possible solution
A more robust and elegant solution is to turn the vector container into a native sidecar container.
Then Vector would be terminated whenever the main container is terminated. To still give it time collect any remainnig log files, a preStop hook can be used instead of the existing STACKABLE_POST_HOOK.
This solution is a breaking change for anyone using podOverrides to change the vector container. This needs to be documented in the changelog and the release notes.
This change requires Kubernetes >1.33 (which SDP doesn't officially support anymore) but there might still be users running it.
Additional context
No response
Environment
No response
Would you like to work on fixing this bug?
yes
- Dominant language
- Rust
- Stars
- 74
- Forks
- 6
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 15
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from stackabletech/spark-k8s-operator
-
type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
stackabletech/spark-k8s-operator#775 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
stackabletech/spark-k8s-operator#806 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
stackabletech/spark-k8s-operator#805 ·
Maintainers usually reply within 1 day
-
Pin versions/ranges for spark-connect-client python dependencies using a constraints fileMay be free again @Maleware claimed this 33 days ago, and no pull request is open. Opentype/bug
stackabletech/spark-k8s-operator#786 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
Allow defining a time to live for finished SparkApplication driver podsPossibly taken @sweb claimed this 18 days ago. Opencustomer-request
Difficulty 4/5 3-5 days Newbie friendliness 55/100
stackabletech/spark-k8s-operator#774 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
All issues in stackabletech/spark-k8s-operator
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
storytold/photocraft#14822 ·
Maintainers usually reply within 1 day
-
bug css
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
dginev/latexml-oxide#853 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 2 days
-
[Bug] export_build_environment admits its output directory only after the export ran in the build computerPossibly taken @viniciusdsandrade claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day