ProtoSchemaConverter renders invalid schema for oneof in unwrap mode

Open

#3,039 opened on Oct 31, 2024 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 3/5
Estimated time: 1-2 days
Newbie friendliness: 45/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Stale
Tech stack: java
Domain: data

Research direction

Start in parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java at the unwrap repetition handling, then review the TestProto3.OneOfTestMessage fixture and its tests. Enable validation in parquet-protobuf/src/test/java/org/apache/parquet/proto/TestUtils.java and add coverage for unwrap mode. Done means the oneof conversion produces a valid schema and the relevant read path is tested.

Written by the indexing model from the issue text.

Description

Type: bug

Describe the bug, including details regarding any error messages, version, and platform.

When unwrap is enabled all fields of the TestProto3.OneOfTestMessage will be required.

message TestProto3.OneOfTestMessage {
  required int32 first = 1;
  required int32 second = 2;
}

https://github.com/apache/parquet-java/blob/73a4430af6c40f8eb246ad4911eb6d103c9a2abe/parquet-protobuf/src/test/resources/TestProto3.proto#L116

This will never work but tests are missing for unwrap of TestProto3.OneOfTestMessage

The required repetition is added here:
https://github.com/apache/parquet-java/blob/73a4430af6c40f8eb246ad4911eb6d103c9a2abe/parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java#L278

This could be caught early by adding withValidation(true) here:
https://github.com/apache/parquet-java/blob/73a4430af6c40f8eb246ad4911eb6d103c9a2abe/parquet-protobuf/src/test/java/org/apache/parquet/proto/TestUtils.java#L221

If validation is disabled it will fail when trying to read one of the non-existing fields in the oneof.

Is there really a need for setting all primitive fields in unwrap to required?

Component(s)

No response

Dominant language: Java
Stars: 3.1k
Forks: 1.6k
Avg merge: 6d 16h
Merged PRs (30d): 36

Contributor guide

No contributing guide indexed for this repository

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

chore: aspectjweaver(EPL-2.0)와 AGPL-3.0 호환성 정리 Open

documentation

Difficulty 2/5 1-3 hours Newbie friendliness 65/100

inu-appcenter/memorIN-backend#288 · Opened Sep 24, 2026
Vulnerability Nessus 12085 - Apache Tomcat Default Files (with killbill/killbill:0.24.21) Open

Difficulty 2/5 1-3 hours Newbie friendliness 65/100

killbill/killbill#2314 · 1 comment · Opened Sep 24, 2026
Work list, Cases and Compliance overlap, and nothing says which one to use Open

frontend maui-pilot pilot-ask question

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

Taleef7/workwell#698 · Opened Sep 24, 2026
[Bug][UI] Replace the status of the button in the Sessions Tab Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

Netcracker/qubership-integration-platform#918 · Opened Sep 24, 2026
executions.Query — startDate and timeRange filters are sent with inverted comparison operators Open

area/plugin

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

kestra-io/plugin-kestra#190 · Opened Sep 24, 2026

More Java issues