[Go] Allow adding existing arrays into structs
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- go
- Domain
- data-engineering
Research direction
Start at StructBuilder and inspect how its field builders currently construct child arrays. Compare that path with the requested SetColumn behavior and trace how existing Arrow arrays are attached or copied. Done means a struct can reuse existing child arrays without manual value-by-value copying, with coverage for the supported array types.
Written by the indexing model from the issue text.
Description
Describe the enhancement requested
Hi! We have a use case, in which we have existing Arrow arrays and we want to compose them to struct. Unfortunately, currently StructBuilder does not support it. So currently we have to copy all values from the specific Arrow array to the similar array in struct one by one. It would be great if we can just reuse existing array without manual copying.
What we do now?
Something like this:
for j := 0; j < int(numRows); j++ {
sb.Append(true)
for i := 0; i < structCol.NumField(); i++ {
newInternalCol := internalCols[i]
if newInternalCol.DataType().ID() == arrow.STRING {
sb.FieldBuilder(i).(*array.StringBuilder).Append(newInternalCol.(*array.String).Value(j))
} else if newInternalCol.DataType().ID() == arrow.INT8 {
sb.FieldBuilder(i).(*array.Int8Builder).Append(newInternalCol.(*array.Int8).Value(j))
What we want:
sb.SetColumn(i, newInternalCol[i])
The main problem here is not this few lines of code that we need to write, but manual memory copying. Is it doable?
Component(s)
Go
- Dominant language
- Assembly
- Stars
- 406
- Forks
- 146
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 93
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/arrow-go
-
Type: enhancement
Difficulty 5/5 Over a week Newbie friendliness 30/100
-
Type: usage
Difficulty 4/5 3-5 days Newbie friendliness 38/100
-
Difficulty 3/5 1-2 days Newbie friendliness 90/100
-
good-first-issue Type: enhancement Type: usage
Difficulty 4/5 3-5 days Newbie friendliness 52/100
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
open-compass/VLMEvalKit#1698 ·
-
triage:deciding
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
open-telemetry/otel-arrow#4123 · 1 reaction ·