DictionaryEncoder.decode accepts out-of-range dictionary indices

Open Beginner friendly
#1,261 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
74/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
java
Domain
security

Research direction

Start at DictionaryEncoder.retrieveIndexVector and trace its use from DictionaryEncoder.decode, ListSubfieldEncoder.decodeListSubField, and StructSubfieldEncoder.decode. Verify that negative indices and indices equal to dictionaryCount are rejected before transfer.copyValueSafe; done means crafted dictionary indices cannot reach the dictionary copy with an invalid position.

Written by the indexing model from the issue text.

Description

DictionaryEncoder.retrieveIndexVector guards each index from the index vector with indexAsInt > dictionaryCount before transfer.copyValueSafe(indexAsInt, i). Valid indices are 0..dictionaryCount-1, so the check is off by one: an index equal to dictionaryCount is accepted and reads one slot past the dictionary vector, and a negative index (a signed index type with the high bit set) is not rejected either and also reaches copyValueSafe. The index vector is decoded from an IPC/C-data payload, so a crafted dictionary-encoded batch yields an out-of-bounds read of the dictionary vector, exposing adjacent off-heap memory when bounds checking is disabled via arrow.enable_unsafe_memory_access.

The same helper backs DictionaryEncoder.decode, ListSubfieldEncoder.decodeListSubField and StructSubfieldEncoder.decode.

The bound should be indexAsInt < 0 || indexAsInt >= dictionaryCount.

Dominant language
Java
Stars
95
Forks
154
Avg merge
2d 16h
Merged PRs (30d)
9

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/arrow-java

All issues in apache/arrow-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.