ParquetMetadataConverter should never log at level WARN
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- java
- Domain
- data-engineering
Research direction
Start by reading parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java at the referenced line and inspect the surrounding metadata conversion behavior. Check relevant converter tests and logging conventions, then confirm with maintainers whether the warning should be removed, downgraded, or replaced with an exception. Done means the agreed behavior is implemented without unintended compatibility changes and covered by an appropriate regression test.
Written by the indexing model from the issue text.
Description
Describe the bug, including details regarding any error messages, version, and platform.
When ParquetMetadataConverter encounters a mismatch between logical and converted type, it logs a message at level WARN. https://github.com/apache/parquet-java/blob/eed26b4ea96881cc543b4233cdddece5d1d608d7/parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java#L1864
This behavior does not make sense because one of two things must be true:
- This is an issue that can affect normal operation and the application needs to (potentially) respond to it
- This is not a concern for the calling application and it won't impact normal operation
If item 1 is true, then an exception should be thrown and the application choose to respond or not (possibly by logging). If item item 2 is true, then the log message should either be omitted entirely or be logged at level DEBUG (INFO level would still be too spammy)
I would be happy to submit a patch if there is consensus about which approach to take. Perhaps there should be a flag to control whether the exception should be thrown to avoid breaking existing code.
Component(s)
No response
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 6d 16h
- Merged PRs (30d)
- 36
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/parquet-java
-
Type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/parquet-java#3792 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
apache/parquet-java#3767 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/parquet-java#3695 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/parquet-java#3667 ·
-
Type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/parquet-java#3574 · 1 comment ·
All issues in apache/parquet-java
Similar issues
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
inu-appcenter/memorIN-backend#288 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
frontend maui-pilot pilot-ask question
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
area/plugin
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
kestra-io/plugin-kestra#190 ·