Disable verbose containerdebug logging in Spark Kubernetes operator
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 52/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- kubernetes, rust, spark
- Domain
- devops, infrastructure
Research direction
Start in rust/operator-binary/src/crd/mod.rs around line 781, where _STACKABLE_PRE_HOOK is hardcoded, and trace how the listed environment and pod overrides are applied. Done means a supported configuration can disable the containerdebug hook without being overwritten, while preserving the default behavior when no override is provided.
Written by the indexing model from the issue text.
Description
Affected Stackable version
v26.3.0
Affected Apache Spark-on-Kubernetes version
No response
Current and expected behavior
Hi all,
Is there any way to disable containerdebug when running Stackable Spark on Kubernetes?
By default, containerdebug appears to always be enabled, and it generates a large amount of unnecessary and verbose logging! Currently it seems like the spark-k8s-operator hardcodes the variable _STACKABLE_PRE_HOOK to containerdebug --output={VOLUME_MOUNT_PATH_LOG}/containerdebug-state.json --loop &.
We have tried overriding the variable _STACKABLE_PRE_HOOK by:
- spec.env with _STACKABLE_PRE_HOOK=""
- job.podOverrides.spec.containers[].env with _STACKABLE_PRE_HOOK=""
- envOverrides at role level or role group level.
However, none of these approaches appear to disable containerdebug. Is there a supported configuration option to turn it off, or is disabling it currently not possible?
Any help would be appreciated!
Possible solution
No response
Additional context
No response
Environment
No response
Would you like to work on fixing this bug?
None
- Dominant language
- Rust
- Stars
- 72
- Forks
- 4
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 15
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from stackabletech/spark-k8s-operator
-
type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
stackabletech/spark-k8s-operator#774 · 1 comment ·
-
type/bug
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
type/bug
Difficulty 3/5 1-2 days Newbie friendliness 55/100
-
customer-request type/bug
Difficulty 4/5 3-5 days Newbie friendliness 45/100
stackabletech/spark-k8s-operator#701 · 1 comment ·
All issues in stackabletech/spark-k8s-operator
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug core
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
JIT-compiled number -> Decimal conversion silently overflows instead of raising DECIMAL_OVERFLOW Openfuzz
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
ClickHouse/ClickHouse#122114 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
linebender/vello_svg#90 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100