Cannot read parquet file that was generated from nanoparquet

Open

#3,043 opened on Nov 6, 2024 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 4/5
Estimated time: 3-5 days
Newbie friendliness: 35/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Stale
Tech stack: java
Domain: data-engineering

Research direction

Start with the stack trace at org.apache.parquet.column.values.ValuesReader.readInteger and the ColumnReaderBase integer-reading path, using the linked mtcars_np.zip file as the reproduction case. Inspect the file's encoding metadata and compare it with what parquet-java 1.14.3 expects; done means MessageColumnIO can read the nanoparquet-generated file without UnsupportedOperationException.

Written by the indexing model from the issue text.

Description

Type: bug

Describe the bug, including details regarding any error messages, version, and platform.

When trying to use MessageColumnIO to get a record reader, the following error occurs:

Exception in thread "main" java.lang.UnsupportedOperationException
	at org.apache.parquet.column.values.ValuesReader.readInteger(ValuesReader.java:178)
	at org.apache.parquet.column.impl.ColumnReaderBase$ValuesReaderIntIterator.nextInt(ColumnReaderBase.java:830)
	at org.apache.parquet.column.impl.ColumnReaderBase.checkRead(ColumnReaderBase.java:663)
	at org.apache.parquet.column.impl.ColumnReaderBase.consume(ColumnReaderBase.java:801)
	at org.apache.parquet.column.impl.ColumnReaderImpl.consume(ColumnReaderImpl.java:30)
	at org.apache.parquet.column.impl.ColumnReaderImpl.<init>(ColumnReaderImpl.java:43)
	at org.apache.parquet.column.impl.ColumnReadStoreImpl.getColumnReader(ColumnReadStoreImpl.java:80)
	at org.apache.parquet.io.RecordReaderImplementation.<init>(RecordReaderImplementation.java:282)
	at org.apache.parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:141)
	at org.apache.parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:105)
	at org.apache.parquet.filter2.compat.FilterCompat$NoOpFilter.accept(FilterCompat.java:180)
	at org.apache.parquet.io.MessageColumnIO.getRecordReader(MessageColumnIO.java:105)

I expected that this would work. Pyarrow for example can read the file.

Sample file can be found here:

mtcars_np.zip

Nanoparquet can be found here:

https://github.com/r-lib/nanoparquet

Version: 1.14.3
Platform: Linux

Component(s)

Core

Dominant language: Java
Stars: 3.1k
Forks: 1.6k
Avg merge: 6d 16h
Merged PRs (30d): 36

Contributor guide

No contributing guide indexed for this repository

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

chore: aspectjweaver(EPL-2.0)와 AGPL-3.0 호환성 정리 Open

documentation

Difficulty 2/5 1-3 hours Newbie friendliness 65/100

inu-appcenter/memorIN-backend#288 · Opened Sep 24, 2026
Vulnerability Nessus 12085 - Apache Tomcat Default Files (with killbill/killbill:0.24.21) Open

Difficulty 2/5 1-3 hours Newbie friendliness 65/100

killbill/killbill#2314 · 1 comment · Opened Sep 24, 2026
Work list, Cases and Compliance overlap, and nothing says which one to use Open

frontend maui-pilot pilot-ask question

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

Taleef7/workwell#698 · Opened Sep 24, 2026
[Bug][UI] Replace the status of the button in the Sessions Tab Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

Netcracker/qubership-integration-platform#918 · Opened Sep 24, 2026
executions.Query — startDate and timeRange filters are sent with inverted comparison operators Open

area/plugin

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

kestra-io/plugin-kestra#190 · Opened Sep 24, 2026

More Java issues