Add example Variant objects where field_id_size and field_offset_size differ
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- testing
Research direction
Start by inspecting the existing object examples in variant/ and how their headers and metadata dictionaries are represented. Add the two requested cases with differing field ID and offset widths, using header bytes 0x06 and 0x12; done means both examples are present and exercise independent decoding of the two widths.
Written by the indexing model from the issue text.
Description
The object value_header holds field_offset_size_minus_one in bits 0-1 and field_id_size_minus_one in bits 2-3. Writers compute the two independently -- field ID width from the largest field ID in the object, offset width from the size of the field data -- so the two routinely differ.
Every object example in variant/ uses 1-byte field IDs and 1-byte field offsets. When the two are equal, the two header fields hold the same value, so a reader that reads them from each other's bit positions still decodes correctly. The corpus therefore cannot detect that class of bug. arrow-dotnet shipped exactly this bug and validated cleanly against this corpus the whole time (apache/arrow-dotnet#420).
Two examples would close the gap:
- Few fields (1-byte field IDs) but more than 256 bytes of field data (2-byte field offsets) -- header byte
0x06. - A metadata dictionary of more than 256 entries (2-byte field IDs) containing a small object whose own field data is under 256 bytes (1-byte field offsets) -- header byte
0x12.
Related to #78 but not covered by it: #78 is framed around field count and treats the two widths as one knob ("a 3 byte field id / offset"). The first
example above has only two fields. The second overlaps with #78's ">2^8 fields" case.
- Dominant language
- Python
- Stars
- 90
- Forks
- 79
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/parquet-testing
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/parquet-testing#112 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
apache/parquet-testing#114 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
apache/parquet-testing#107 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 32/100
apache/parquet-testing#97 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
apache/parquet-testing#89 · 5 comments ·
All issues in apache/parquet-testing
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100