hudi-cli: metadata delete-record-index NPEs unless another metadata command ran first

Open Beginner friendly
#19,878 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
88/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
java, spark
Domain
cli

Research direction

Open MetadataCommand.java and compare deleteRecordIndex with the sibling metadata create, init, and list-partitions commands. Run the reproduction with delete-record-index as the first metadata command, then use the tests being written in issue #19877 to verify the command initializes its Spark context and deletes the record index without an ordering dependency.

Written by the indexing model from the issue text.

Description

type:bug
Bug Description

What happened:
metadata delete-record-index throws a bare NullPointerException when it is the first metadata command run in a hudi-cli session. MetadataCommand.deleteRecordIndex (master 6ac27904, MetadataCommand.java:154-156) builds new HoodieSparkEngineContext(jsc) without calling initJavaSparkContext(...) first, unlike metadata create (line 136), metadata init (line 181) and metadata list-partitions (line 219). The command only works if another metadata command initialised the shared jsc field earlier in the same session.

What you expected:
The command initialises its own Spark context like the sibling commands and deletes the record index.

Steps to reproduce:

  1. connect --path <table with record index>
  2. metadata delete-record-index
  3. NPE from HoodieSparkEngineContext construction. Run metadata list-partitions first and step 2 succeeds.

Found while writing the tests in #19877; the command is left untested there rather than pinning the ordering dependency. Fix is a one-liner: call initJavaSparkContext(Option.of(master)) (adding the --sparkMaster option the other commands take) before constructing the engine context.

Environment

Hudi version: master (6ac27904), also present in 1.0.x
Query engine: hudi-cli (Spark)
Relevant configs: none

Logs and Stack Trace
java.lang.NullPointerException
  at org.apache.hudi.client.common.HoodieSparkEngineContext.<init>(HoodieSparkEngineContext.java)
  at org.apache.hudi.cli.commands.MetadataCommand.deleteRecordIndex(MetadataCommand.java:156)
Dominant language
Java
Stars
6.3k
Forks
2.5k
Avg merge
2d 5h
Merged PRs (30d)
111

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/hudi

All issues in apache/hudi

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.