Support DIGEST-MD5 / delegation token authentication for HMS

Open

#3,145 opened on Mar 12, 2026 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 5/5
Estimated time: Over a week
Newbie friendliness: 45/100

Issue type: Feature
Clarity: Mostly clear
Activity status: Active
Tech stack: hadoop, python
Domain: backend, databases

Research direction

Start with _HiveClient and its existing hive.kerberos-authentication path, then inspect how TSaslClientTransport is configured. The work is done when DIGEST-MD5 reads the Hadoop Writable credentials from $HADOOP_TOKEN_FILE_LOCATION and uses the token identifier and password for HMS authentication.

Written by the indexing model from the issue text.

Description

stale

Feature Request / Improvement

Summary

PyIceberg's HiveCatalog supports Kerberos (GSSAPI) authentication via hive.kerberos-authentication, but does not support DIGEST-MD5 SASL authentication with Hadoop delegation tokens. In many production Hadoop environments, pods/containers authenticate to HMS using delegation tokens (read from $HADOOP_TOKEN_FILE_LOCATION) rather than Kerberos keytabs. This means PyIceberg's Hive catalog cannot be used in these environments without building a custom client.

Proposed Enhancement

Extend _HiveClient to support DIGEST-MD5 delegation token auth:

Add a new config property (e.g. hive.metastore.authentication=DIGEST-MD5)
When DIGEST-MD5 is configured, read credentials from $HADOOP_TOKEN_FILE_LOCATION (Hadoop Writable credentials format)
Use TSaslClientTransport with mechanism=DIGEST-MD5 and the extracted token identifier/password

Dominant language: Python
Stars: 1.1k
Forks: 589
Avg merge: 2d 2h
Merged PRs (30d): 70

Contributor guide

No contributing guide indexed for this repository

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

create_predictable_zip member order differs on Windows for directory input Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 90/100

learningequality/ricecooker#747 · Opened Sep 23, 2026
[Anon] Mechanicum: skitarii as mechanicum allies rule Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

BSData/horus-heresy-3rd-edition#3171 · Opened Sep 23, 2026
CHANGELOG.md entries are too long to maintain, and their length is what makes every PR conflict expensive Open

enhancement

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

lanl/PyBNF#809 · Opened Sep 23, 2026
[Bug]: PrevNextNodePostprocessor loses retrieved scores when neighbor windows overlap Open

Difficulty 2/5 1-3 hours Newbie friendliness 76/100

run-llama/llama_index#23199 · Opened Sep 23, 2026
Cannot export point cloud without radius attribute Open

Difficulty 2/5 1-3 hours Newbie friendliness 84/100

KhronosGroup/glTF-Blender-IO#2769 · Opened Sep 23, 2026

More Python issues