The error message "is not a Parquet file" doesn't show the Parquet path anymore

Open Beginner friendly
#3,354 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
68/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
java
Domain
data-engineering

Research direction

Start by comparing the linked ParquetFileReader.java locations from versions 1.13.1 and 1.15.2, focusing on how the error message obtains the path. Reproduce the reported short-file case using the shown parquet path and verify that the resulting exception identifies that path instead of the ParquetIOUtils object representation.

Written by the indexing model from the issue text.

Description

Type: bug
Describe the bug, including details regarding any error messages, version, and platform.

Version 1.13.1, used by Spark 3.5: https://github.com/apache/parquet-java/blob/db4183109d5b734ec5930d870cdae161e408ddba/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java#L548

used to show the parquet file path: Cause: java.lang.RuntimeException: file:/tmp/spark-37..../00000000000000000000.checkpoint.parquet is not a Parquet file (length is too low: 10)

In version 1.15.2, used by Spark 4: https://github.com/apache/parquet-java/blob/859eac165b08f927fa14590c33bc5f476405fb68/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java#L584

The error message says: Cause: java.lang.RuntimeException: io.delta.kernel.defaults.internal.parquet.ParquetIOUtils$1@63d4ffeb is not a Parquet file (length is too low: 10)

Component(s)

No response

Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 23h
Merged PRs (30d)
30

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/parquet-java

All issues in apache/parquet-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.