Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Support reading HDFS files with Kerberos authentication

Open
  • Is something wrong?
#4,654 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
38/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
hadoop
Domain
data-engineering, distributed-systems, security

Research direction

Start by tracing how graphscope.session() configures existing non-secure HDFS input and output, then inspect how Engine pods receive storage settings. Define how keytab, principal, and optional krb5.conf are supplied and authenticated across all pods; done means Kerberized HDFS reads and writes work alongside existing local, object-storage, and non-secure HDFS access.

Written by the indexing model from the issue text.

Description

stale

Currently, GraphScope supports reading graph data from local files, object storage, and non-secure HDFS. However, many production Hadoop clusters require Kerberos authentication for security compliance.

When attempting to load graph data from a Kerberized HDFS (e.g., hdfs://namenode:8020/path/to/data), there is no built-in mechanism to provide Kerberos credentials (keytab, principal, krb5.conf), making it impossible to access such data sources directly.

I would like GraphScope to support reading graph data from HDFS clusters with Kerberos authentication enabled. The expected solution could include:

  1. Configuration parameters in graphscope.session() to specify Kerberos settings, e.g.:

    sess = graphscope.session(
    hdfs_kerberos_enabled=True,
    hdfs_kerberos_principal="user/host@REALM",
    hdfs_kerberos_keytab="/path/to/keytab",
    hdfs_krb5_conf="/etc/krb5.conf"  # optional
)

  2. Automatic authentication before accessing HDFS files, ensuring all Engine pods can authenticate to the NameNode and DataNodes.

  3. Support for both HDFS input (loading graph data) and HDFS output (storing results).

Dominant language
C++
Stars
3.6k
Forks
470
No merged PRs in 30d

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from alibaba/GraphScope

All issues in alibaba/GraphScope

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.