How to iterate over a record batch?

Open
#468 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
25/100
Issue type
Documentation
Clarity
Needs clarification
Activity status
Stale
Tech stack
java
Domain
data

Research direction

Start with the VectorSchemaRoot, getVector, getObject, and contentToTSVString usage described in the issue, then compare them with the Java documentation linked there. Identify the documented way to read each vector with its concrete Java type, and consider the work done when a newcomer can iterate over a batch without reverse-engineering type mappings or reparsing strings.

Written by the indexing model from the issue text.

Description

Type: usage
Describe the usage question you have. Please include as many useful details as possible.

Now I have a VectorSchemaRoot. I can see to iterate over the batch with getVector and then getObject.

But the return value is of type Obejct. And I wonder how I can downcast it for some useful class I can retrieve the real value (string, int, float, etc.).

I know we have the field info of each vetcor, but I don't know the mapping between field type to real Java class. It looks over challenge to remember all the mapping by reverse engineering the code, and it may change as version evolves.

I checked https://arrow.apache.org/docs/java/index.html but all the pages tell about constructing a batch and how to move it from one place to another, rather than tell about how to read and dump a batch to a typed two-dimensional matrix.

The most trivial usage, contentToTSVString, call Object::toString on each cell. But I don't thing we should convert all the values to String and reparse it to concrete type.

Dominant language
Java
Stars
95
Forks
154
Avg merge
2d 10h
Merged PRs (30d)
11

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/arrow-java

All issues in apache/arrow-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.