[Bug] Empty array/map field in Hive text is read as NULL (or errors) instead of an empty collection
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Start in be/src/core/data_type_serde/ at DataTypeArraySerDe::deserialize_one_cell_from_hive_text and DataTypeMapSerDe::deserialize_one_cell_from_hive_text, and inspect how empty slices and the explicit \N marker are handled. Verify the change with Hive text deserialization coverage: empty fields should produce [] or {}, while \N remains NULL and non-nullable loads no longer fail.
Written by the indexing model from the issue text.
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
master / 4.0
What's Wrong?
When reading a Hive text-format table, an empty field for an ARRAY or MAP column is not deserialized as an empty collection. In DataTypeArraySerDe::deserialize_one_cell_from_hive_text and DataTypeMapSerDe::deserialize_one_cell_from_hive_text (be/src/core/data_type_serde/), an empty slice returns Status::InvalidArgument("slice is empty!"):
- For a nullable column,
DataTypeNullableSerDeswallows that error and inserts NULL, so an empty field silently becomes NULL. - For a non-nullable column, the error propagates and fails the load.
Neither matches Spark/Hive semantics, where an empty complex-type field is an empty collection ([] / {}), distinct from an explicit NULL (\N).
What You Expected?
An empty Hive-text field for an ARRAY/MAP column should deserialize to an empty collection ([] / {}). An explicit NULL marker (\N) should remain NULL.
How to Reproduce?
Query a Hive text table (or use the Hive text serde) where an ARRAY/MAP column has an empty field. The value comes back as NULL (nullable column) or the read fails (non-nullable column), instead of an empty collection.
Anything Else?
Fix: on an empty slice, append an empty collection (a zero-length element) and return OK, instead of returning InvalidArgument. This changes empty nullable array/map fields from NULL to []/{}; explicit \N is unchanged. (Struct is intentionally left out — an "empty struct" has no canonical representation; that can be a follow-up.)
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
- Dominant language
- Java
- Stars
- 16k
- Forks
- 4k
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 569
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/doris
-
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100