[Go][Parquet] reader panics when selectively decrypting columns
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start with go/parquet/encryption_read_config_test.go around the linked reproduction and compare it with the reported selective-column-decryption implementation. Determine the expected behavior when a column key is absent, including whether encrypted values can be retained, then define regression coverage for the panic and the intended read result.
Written by the indexing model from the issue text.
Description
Describe the bug, including details regarding any error messages, version, and platform.
Hello, following up on issue apache/arrow#14940, I am leveraging the go arrow library to encrypt/decrypt parquet files.
My goal is to read parquet files with encrypted columns, decrypt some of the columns and produce a new parquet file with the decrypted column while keeping the encrypted column as part of the file.
Unfortunately, when reading parquet files with an encryption key per column, the reader panics with the error hidden column exception if one column is missing from the decryption configuration.
To reproduce this behavior, I have forked the repo and added a decryption configuration (no 5), which you can try here: https://github.com/ahmb84/arrow/blob/bf1964552ca4d62988ca306f4b98db08e18ac7d8/go/parquet/encryption_read_config_test.go#L158
I've then tried to implement a version of the reader that can bypass the encryption context if no key is present in the decryption configuration for a column encrypted with its key.
It seems to be working, as the reader does not panic, but the encrypted column appears to be skipped as no values are read (I believe that's the expected behavior?).
Ideally, the reader would be able to return the encrypted value so that I could rebuild a parquet file with the encrypted data and some decrypted data.
You can find my implementation here: https://github.com/ahmb84/arrow/tree/selective-column-decryption-impl
Let me know if my understanding is correct. I appreciate your help!
Component(s)
Go, Parquet
- Dominant language
- Assembly
- Stars
- 406
- Forks
- 146
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 93
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/arrow-go
-
Type: enhancement
Difficulty 5/5 Over a week Newbie friendliness 30/100
-
Type: usage
Difficulty 4/5 3-5 days Newbie friendliness 38/100
-
Difficulty 3/5 1-2 days Newbie friendliness 90/100
-
good-first-issue Type: enhancement Type: usage
Difficulty 4/5 3-5 days Newbie friendliness 52/100
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
cisagov/cyhy-reports#149 · 3 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
open-compass/VLMEvalKit#1698 ·