The construction of arrays should be more flexible
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- csharp
- Domain
- data-engineering
Research direction
Start by reviewing the type-specific Builder and IArrowArray construction APIs described in the issue, then compare the linked Apache Arrow issues for a bounded scope. The issue lists several independent goals, so done would require selecting one improvement, defining its behavior, and adding corresponding coverage in the affected array construction code.
Written by the indexing model from the issue text.
Description
Today, arrays can only be constructed using a type-specific Builder. Builders start with some default initial size and grow either incrementally or by setting a new capacity. Once construction is complete, an immutable IArrowArray is produced. This mode of operation is safe but inflexible and potentially inefficient.
Some ideas for improving this are to
- Allow an initial size to be specified for builders
- Support (fixed-size) read-write arrays -- at least for fixed-width types -- in addition to read-only arrays
- Allow ownership of memory to pass from the builder to the array upon construction so as not to require an additional allocation.
- Allow native memory to be allocated without clearing it first
- Allow a null value to be set directly in the builder, just as a non-null value can be
- More flexible builders for nested types
- Validating that the nullability of the type matches the values in the array
Inspired by:
https://github.com/apache/arrow/issues/21093
https://github.com/apache/arrow/issues/21386
https://github.com/apache/arrow/issues/21746
https://github.com/apache/arrow/issues/22309
https://github.com/apache/arrow/issues/25449
https://github.com/apache/arrow/issues/30567
https://github.com/apache/arrow/issues/35285
https://github.com/apache/arrow/issues/36133
https://github.com/apache/arrow/issues/40188
- Dominant language
- C#
- Stars
- 40
- Forks
- 30
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 13
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/arrow-dotnet
-
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
apache/arrow-dotnet#410 · 4 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 38/100
apache/arrow-dotnet#409 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
apache/arrow-dotnet#399 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
apache/arrow-dotnet#398 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
apache/arrow-dotnet#397 · 2 comments ·
All issues in apache/arrow-dotnet
Similar issues
-
bug needs response
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Adyen/adyen-dotnet-api-library#1869 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
difficulty/starter 🚀 good first issue kind/bug platform/ios 🍎 project/non-ui ⚙️ triage/untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
unoplatform/uno#24650 ·
-
area-ai untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
dotnet/extensions#7783 ·
-
untriaged
Difficulty 1/5 Under an hour Newbie friendliness 88/100
dotnet/dotnet-api-docs#13095 ·