Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

TestParquetReader fails with hadoop 3.4.2

Open
#3,356 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
hadoop, java
Domain
data-engineering

Research direction

Start with the failing TestParquetReader cases, especially testRangeFiltering, and inspect the vectored IO reading path. Check how returned buffers are handled by the releaser, then rerun the suite with Hadoop 3.4.2 and parquet-mr 1.16.0. Done means the tests pass without TrackingByteBufferAllocator leak errors.

Written by the indexing model from the issue text.

Description

Type: bug
Describe the bug, including details regarding any error messages, version, and platform.

A couple of test cases of TestParquetReader suite started failing with errors like:

- testRangeFiltering[0] *** FAILED ***
  org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException: 24 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
  at org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:161)
  at org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:569)

with hadoop 3.4.2 and parquet-mr 1.16.0
The leaks are happening when reading using vectored IO because we never pass the buffers to the releaser.

Component(s)

No response

Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
4d 12h
Merged PRs (30d)
28

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/parquet-java

All issues in apache/parquet-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.