[Bug] Empty array/map field in Hive text is read as NULL (or errors) instead of an empty collection

Open Beginner friendly
#66,039 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
cpp
Domain
databases

Research direction

Start in be/src/core/data_type_serde/ at DataTypeArraySerDe::deserialize_one_cell_from_hive_text and DataTypeMapSerDe::deserialize_one_cell_from_hive_text, and inspect how empty slices and the explicit \N marker are handled. Verify the change with Hive text deserialization coverage: empty fields should produce [] or {}, while \N remains NULL and non-nullable loads no longer fail.

Written by the indexing model from the issue text.

Description

Search before asking
  • I had searched in the issues and found no similar issues.
Version

master / 4.0

What's Wrong?

When reading a Hive text-format table, an empty field for an ARRAY or MAP column is not deserialized as an empty collection. In DataTypeArraySerDe::deserialize_one_cell_from_hive_text and DataTypeMapSerDe::deserialize_one_cell_from_hive_text (be/src/core/data_type_serde/), an empty slice returns Status::InvalidArgument("slice is empty!"):

  • For a nullable column, DataTypeNullableSerDe swallows that error and inserts NULL, so an empty field silently becomes NULL.
  • For a non-nullable column, the error propagates and fails the load.

Neither matches Spark/Hive semantics, where an empty complex-type field is an empty collection ([] / {}), distinct from an explicit NULL (\N).

What You Expected?

An empty Hive-text field for an ARRAY/MAP column should deserialize to an empty collection ([] / {}). An explicit NULL marker (\N) should remain NULL.

How to Reproduce?

Query a Hive text table (or use the Hive text serde) where an ARRAY/MAP column has an empty field. The value comes back as NULL (nullable column) or the read fails (non-nullable column), instead of an empty collection.

Anything Else?

Fix: on an empty slice, append an empty collection (a zero-length element) and return OK, instead of returning InvalidArgument. This changes empty nullable array/map fields from NULL to []/{}; explicit \N is unchanged. (Struct is intentionally left out — an "empty struct" has no canonical representation; that can be a follow-up.)

Are you willing to submit PR?
  • Yes I am willing to submit a PR!
Code of Conduct
Dominant language
Java
Stars
16k
Forks
4k
Avg merge
2d 12h
Merged PRs (30d)
569

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/doris

All issues in apache/doris

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.