InternalParquetRecordWriter flushRowGroupToStore not actually writing to file
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 45/100
Research direction
Start by reading InternalParquetRecordWriter, especially flushRowGroupToStore and close, then trace how parquetFileWriter.end writes buffered data. Confirm the expected flush boundary and add or update coverage for file contents after flushRowGroupToStore; done means the file contains the flushed row group before close is invoked.
Written by the indexing model from the issue text.
Description
Describe the bug, including details regarding any error messages, version, and platform.
Description:
The flushRowGroupToStore method in InternalParquetRecordWriter purports to "Flush mem columnStore to file," but no data is actually written to the file system until the close method is invoked. This behavior is unexpected, as one would anticipate data being flushed.
Expected Result:
Data should be written to the file when flushRowGroupToStore is invoked, effectively flushing the in-memory column store contents to the file.
Actual Result:
File contents remain empty until the close method is called, where parquetFileWriter.end is invoked to write the data.
Version : parquet-hadoop 1.13.1
Component(s)
No response
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 6d 16h
- Merged PRs (30d)
- 36
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/parquet-java
-
Type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/parquet-java#3792 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
apache/parquet-java#3767 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/parquet-java#3695 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/parquet-java#3667 ·
-
Type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/parquet-java#3574 · 1 comment ·
All issues in apache/parquet-java
Similar issues
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
inu-appcenter/memorIN-backend#288 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
frontend maui-pilot pilot-ask question
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
area/plugin
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
kestra-io/plugin-kestra#190 ·