apache/dolphinscheduler

[Improvement][Helm Chart] Support HDFS HA

Open

#16,390 opened on Jul 29, 2024

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Java (4,324 forks)batch import
help wantedimprovementkubernetes

Repository metrics

Stars
 (11,659 stars)
PR merge metrics
 (Avg merge 1d 17h) (22 merged PRs in 30d)

Description

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

The helm chart support HDFS HA.

Use case

Now, the config option has "resource.hdfs.fs.defaultFS". And the comment say "if resource.storage.type=HDFS and namenode HA is enabled, you need to copy core-site.xml and hdfs-site.xml to conf dir", but the template not use the "conf" dir or "config" dir. The pod only mount common.properties.

I want some ways to support HDFS enabled HA. Maybe some option for the xml files in common.properties, or some mount.

And, i m using HDFS deployed by stackable hdfs operator, the core-site.xml and hdfs-site.xml are ConfigMaps. Maybe some way can use this.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide

Research direction
Modify the Helm chart to allow mounting additional configuration files (core site.xml, hdfs site.xml) from ConfigMaps or secrets. Update the template to reference these files and add an option in values.yaml for specifying the ConfigMap names. Test with a real HDFS HA cluster.
Tech stack
javakubernetes
Domain
backendinfrastructuredevops
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
KubernetesHelmHDFS
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60