apache/beam

Python HDFS: add Kerberos authentication support

Open

#20,719 opened on 2022年6月4日

GitHub で見る
 (4 comments) (0 reactions) (1 assignee)Java (4,097 forks)batch import
P3good first issuehadoopimprovementiopython

Repository metrics

Stars
 (7,313 stars)
PR merge metrics
 (平均マージ 6d) (30d で 243 merged PRs)

説明

The HDFS client used by Beam supports Kerberos.

Initial idea: add a flag --hdfs_client that defaults to "INSECURE" and also accepts "KERBEROS". This flag will control initialization of self._hdfs_client.

HDFS client docs: https://hdfscli.readthedocs.io/en/latest/api.html#module-hdfs.ext.kerberos

The HDFS client seems to use this Kerberos library: https://pypi.org/project/requests-kerberos/

Imported from Jira BEAM-11750. Original Jira may contain additional context. Reported by: udim.

コントリビューターガイド